Instead of just following along, I broke everything down step by step, adding explanations and extra context to help me understand what was happening. This walkthrough is my way of organizing what I learned\u2014and hopefully making it easier for anyone else figuring this out for the first time!<\/p>\n
Fair warning\u2014this is long. Like, really<\/em> long. If you\u2019re looking for a quicker, more to-the-point version, go watch the video\u2014it\u2019s a great, concise, and easy to follow along. But if you\u2019re like me and want to really understand what\u2019s happening under the hood, then roll up your sleeves.<\/p>\n This tutorial will walk through how to build a simple LangChain app that:<\/strong><\/p>\n In this tutorial, you will learn how to:<\/strong><\/p>\n In this section, we\u2019ll set up our development environment to run a local large language model (LLM)<\/strong>. By the end, you\u2019ll have a working Jupyter Notebook<\/strong> where you can seamlessly switch between OpenAI\u2019s API<\/strong> and a locally running model like LLaMA 2 or Mixtral<\/strong>.<\/p>\n Already familiar with this setup?<\/strong> You might want to skip ahead and check out Santiago Valdarrama\u2019s GitHub project<\/a> (llm)<\/strong>, which accompanies his YouTube tutorial on building a Retrieval-Augmented Generation (RAG) system with LangChain<\/strong>. Otherwise, let\u2019s get started!<\/p>\n This tutorial assumes you have a basic familiarity with programming concepts, command-line usage, and Visual Studio Code (VS Code). It also uses Jupyter notebooks inside VS Code.<\/p>\n \u26d4 Dependencies:<\/strong> Here\u2019s what you\u2019ll need to get up and running:<\/p>\n Ollama is a lightweight tool that acts as a wrapper around several open-source LLMs (like Llama 2, Mixtral, etc.) to run them via a common interface.<\/p>\n Download and Install Ollama:<\/strong><\/p>\n You can now interact with your model through the command line. Try typing a prompt like \u201ctell me a joke\u201d. For other models, view the list here: https:\/\/ollama.com\/search<\/a><\/p>\n First we need to set up a project workspace in VS Code, ensuring there is a clean and isolated environment to work with. We’ll create a dedicated directory, set up a Jupyter Notebook, and configure a virtual environment along with environment variables for sensitive data like your API keys.<\/p>\n Create a new directory (e.g., local-model) and open it in VS Code. You can do this manually or via the command line: We\u2019re using a virtual environment to keep dependencies isolated and ensure that installing packages doesn\u2019t interfere with other projects.<\/p>\n Now, let\u2019s confirm that everything is set up correctly. Open your Jupyter Notebook and run the following simple Python command to check if your environment is working properly. Add Then run your file:<\/p>\n Create an environment file(e.g., In your notebook, load these variables using Python\u2019s In order to read environment variables, install Python dotenv in the terminal by running \u2754Why use a local LLM?<\/strong><\/p>\n Before diving into the code, here are some reasons why you might want to run a local LLM:<\/p>\n At this point, we have:<\/strong><\/p>\n Before we move on to the next step, let\u2019s review some vocabulary.<\/p>\n LangChain<\/strong> is an open-source framework designed to help developers build applications powered by Large Language Models (LLMs) like GPT, Llama, and Claude. In LangChain<\/strong>, a Chain<\/strong> is a structured sequence of operations that process inputs (e.g., user queries) through one or more steps before producing an output. Chains allow you to combine multiple components<\/strong>, such as LLMs, retrieval systems, APIs, and logic<\/strong>, into a pipeline.<\/p>\n\n
\n
\nPart One: Project Setup in VS Code<\/h2>\n
\n\n
1. Installing Ollama and Downloading Models<\/h3>\n
\n
# List available commands by typing:\r\nollama --help --help\r\n\r\n# Install a model (for example, Llama 2) by running:\r\nollama pull llama2\r\n\r\n# Verify your installed models:\r\nollama list\r\n\r\n#To start serving a model locally, run:\r\nollama run llama2\r\n<\/code><\/pre>\n2. Creating a Project Directory:<\/strong><\/h3>\n
dev mkdir local-model<\/code><\/p>\n3. Set Up a Jupyter Notebook:<\/strong><\/h3>\n
\n
notebook.ipynb<\/code>) in your project.<\/li>\n4. Creating a Virtual Environment:<\/strong><\/h3>\n
\n
python3 -m venv .venv<\/code><\/li>\nsource .venv\/bin\/activate<\/code><\/li>\n<\/ul>\n5. Making Sure It All Runs:<\/h3>\n
print(\"Hello World\")<\/code> to your file and Run. When this runs, you\u2019ll be prompted to select a kernel. Select Python Environment, then select the Python .venv<\/p>\n![\"\"](\"https:\/\/spin.atomicobject.com\/wp-content\/uploads\/Screenshot-2025-03-04-at-1.46.29\u202fPM.png\")
<\/p>\n![\"\"](\"https:\/\/spin.atomicobject.com\/wp-content\/uploads\/Screenshot-2025-03-04-at-1.49.40\u202fPM.png\")
<\/p>\n6. Setting Up Libraries & Environment Variables:<\/h3>\n
.env<\/code>) inside of your folder to store your API keys and other configuration. Get your OpenAI API key and store it in the variable.<\/p>\nOPENAI_API_KEY=your_openai_key_here\r\n<\/code><\/pre>\nos<\/code> library or a dedicated library (like dotenv<\/code>).<\/p>\n#################################################################################\r\n### IMPORT LIBRARIES\r\n#################################################################################\r\nimport os \r\n\r\n# Library that reads environment variables in the .env files\r\nfrom dotenv import load_dotenv \r\nload_dotenv()\r\n\r\n# print(\"Hello\")\r\n\r\n#################################################################################\r\n### IMPORT - Load environment variables\r\n#################################################################################\r\n\r\n# Get the OpenAI Key\r\n# Why do we need to use an openai api when we are running models locally?\r\n# We want to test everything we are doing locally with openai\/gpt to see how they compare. \r\nOPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\")\r\n\r\n#################################################################################\r\n### DEFINE MODELS\r\n#################################################################################\r\nMODEL = \"gpt-3.5-turbo\" #Chat GPT Model\r\n# MODEL = \"mixtral;8x7b\" #Locally running open source model\r\n# MODEL = \"llama2\"\r\n<\/code><\/pre>\npip install python_dotenv<\/code><\/p>\n\n
\n
\nDid You Know That\u2026<\/h3>\n
\n
\nPart Two: Setting Up LangChain<\/h2>\n
What is LangChain?<\/h3>\n