LLM Stuff

Hodgepodge of experiments, tests, etc. with several Large Language Models (LLMs), courtesy of GitHub Models.

Current models:

• OpenAI GPT-4o
• Meta Llama 3.2 (90B parameters)
• Cohere Embed V3

Refer to the TODO section for a list of models I plan to add.

TL;DR LLM playground

Requirements

• Anaconda OR Miniconda

Tip

If you have trouble deciding between Anaconda and Miniconda, please refer to the table below

Anaconda	Miniconda
New to conda and/or Python	Familiar with conda and/or Python
Not familiar with using terminal and prefer GUI	Comfortable using terminal
Like the convenience of having Python and 1,500+ scientific packages automatically installed at once	Want fast access to Python and the conda commands and plan to sort out the other programs later
Have the time and space (a few minutes and 3 GB)	Don't have the time or space to install 1,500+ packages
Don't want to individually install each package	Don't mind individually installing each package

Typing out entire Conda commands can sometimes be tedious, so I wrote a shell script (conda_shortcuts.sh on GitHub Gist) to define shortcuts for commonly used Conda commands.

Example: Delete/remove a conda environment named test_env

• Shortcut command

  rmenv test_env
  

• Manually typing out the entire command

  conda env remove -n test_env && rm -rf $(conda info --base)/envs/test_env

The shortcut has 80.8% less characters!

Installation

1. Verify that conda is installed

   conda --version
   

2. Ensure conda is up to date

   conda update conda
   

3. Enter the directory where you want the repository (llm-stuff) to be cloned
   • POSIX

     cd ~/path/to/directory

   • Windows

     cd C:\path\to\directory

4. Clone the repository (llm-stuff), then enter (i.e. cd command) llm-stuff directory

   git clone https://github.com/lynkos/llm-stuff.git && cd llm-stuff

5. Create a conda virtual environment from environment.yml

   conda env create -f environment.yml
   

6. Activate the virtual environment (llm_env)

   conda activate llm_env
   

7. Confirm that the virtual environment (llm_env) is active
   • If active, the virtual environment's name should be in parentheses () or brackets [] before your command prompt, e.g.

     (llm_env) $
     

   • If necessary, see which environments are available and/or currently active (active environment denoted with asterisk (*))

     conda info --envs
     

     OR

     conda env list
     

Usage

Depending on the LLM you want to use, execute any of the following commands in your preferred Terminal.

OpenAI GPT-4o

python -m src.OpenAI.GPT-4o

Meta Llama 3.2 (90B)

python -m src.Meta.Llama3_2

Cohere Embed V3

python -m src.Cohere.EmbedV3

TODO

• Add more models
  • Meta Llama 3.1 Instruct (405B)
  • Microsoft Phi 3.5 MoE Instruct (128k)
  • Microsoft Phi 3.5 Vision Instruct (128k)
  • Cohere Embed V3 English
  • Mistral Nemo
  • AI21 Jamba 1.5 Large
• Add command line functionality
• Separate classes for vision and language models
• Logging for debugging
• Save convo transcript option