Tutorial
Create a LangChain RAG system in Python with watsonx
Fetch data to create a vector store as context for an LLM to answer questionsIn this tutorial, we’ll use LangChain to walk through a step-by-step Retrieval Augmented Generation (RAG) example in Python.
RAG is a technique in natural language processing (NLP) that combines information retrieval and generative models to produce more accurate, relevant and contextually aware responses.
For our use case, we’ll set up a RAG system for IBM Think 2024. IBM Think 2024 is a conference where IBM announces new products, technologies, and partnerships. We will fetch content from several ibm.com websites, which will make up a knowledge base from, which we will provide to an LLM with context to answer some questions about Think 2024.
Watch me demonstrate the steps of this tutorial for how to leverage LangChain RAG to achieve this use case in this video:
Video will open in new tab or window
More about RAG and LangChain
In traditional language generation tasks, large language models (LLMs) like OpenAI’s GPT-3.5 (Generative Pre-trained Transformer) or IBM’s Granite Models are used to construct responses based on an input prompt. However, these models might struggle to produce responses that are contextually relevant, factually accurate or up to date. The models might not know the latest information on IBM Think 2024.
RAG applications address this limitation by incorporating a retrieval step before response generation. During retrieval, vector search can be used to identify contextually pertinent information, such as relevant information or documents from a large corpus of text, typically stored in a vector database. Finally, an LLM is used to generate a response based on the retrieved context.
LangChain is a powerful open-source framework that facilitates the development of applications using LLMs for various NLP tasks. In the context of RAG, LangChain plays a critical role by combining the strengths of retrieval-based methods and generative models to enhance the capabilities of NLP systems.
Prerequisites
You need an IBM Cloud account to create a watsonx.ai project.
Steps
Step 1. Set up your environment
While you can choose from several tools, this tutorial walks you through how to set up an IBM account to use a Jupyter Notebook. Jupyter Notebooks are widely used within data science to combine code, text, images, and data visualizations to formulate a well-formed analysis.
Log in to watsonx.ai using your IBM Cloud account.
Create a watsonx.ai project.
Create a Jupyter Notebook.
This step will open a Notebook environment where you can copy the code from this tutorial to implement a RAG application for Think 2024. Alternatively, you can download this notebook to your local system and upload it to your watsonx.ai project as an asset. This Jupyter Notebook is available on GitHub.
Step 2. Set up the watsonx.ai Runtime service and API key
- Create a watsonx.ai Runtime service instance (choose the Lite plan, which is a free instance).
- Generate an API Key.
- Associate the watsonx.ai Runtime service to the project that you created in watsonx.ai.
Step 3. Install and import relevant libraries and set up credentials
We'll need a few libraries for this tutorial. Make sure to import the ones below, and if they're not installed, you can resolve this with a quick pip install.
#installations
%pip install langchain
%pip install langchain_chroma
%pip install langchain-community
%pip install -U langchain_ibm
%pip install unstructured
%pip install "ibm-watson-machine-learning>=1.0.327"
%pip install nltk
Import the relevant libraries:
#imports
import os
import getpass
from ibm_watson_machine_learning.foundation_models.utils.enums import ModelTypes
from ibm_watson_machine_learning.metanames import GenTextParamsMetaNames as GenParams
from ibm_watsonx_ai.foundation_models.utils.enums import EmbeddingTypes
from langchain_ibm import WatsonxEmbeddings, WatsonxLLM
from langchain.vectorstores import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_community.document_loaders import UnstructuredURLLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
import nltk
nltk.download('averaged_perceptron_tagger_eng')
Set up your credentials and input your API Key:
credentials = {
"url": "https://us-south.ml.cloud.ibm.com",
"apikey": getpass.getpass("Please enter your WML api key (hit enter): ")
}
Set up your project_id as part of your environment variables or input it:
try:
project_id = os.environ["PROJECT_ID"]
except KeyError:
project_id = input("Please enter your project_id (hit enter): ")
Step 4. Index the URLs to create the knowledge base
We’ll index our Think 2024 specific articles from URLs to create a knowledge base as a vectorstore. The content from these URLs will be our data sources and context for this exercise. The context will then be provided to an LLM to answer any questions we have about the Think 2024 conference.
The first step to building vector embeddings is to clean and process the raw data set. This might involve the removal of noise and the standardization of the text. For our example, we won’t do any cleaning since the text is already cleaned and standardized.
First, let's establish URLS_DICTIONARY. URLS_DICTIONARY is a dict that helps us map the URLs from which we will be extracting the content. Let's also set up a name for our collection: askibm_think_2024.
URLS_DICTIONARY = {
"ibm.com_events_think_faq.html": "https://www.ibm.com/events/think/faq",
"events_think_agenda.html": "https://www.ibm.com/events/think/agenda",
"products_watsonx_ai.html": "https://www.ibm.com/products/watsonx-ai",
"products_watsonx_ai_foundation_models.html": "https://www.ibm.com/products/watsonx-ai/foundation-models",
"watsonx_pricing.html": "https://www.ibm.com/watsonx/pricing",
"watsonx.html": "https://www.ibm.com/watsonx",
"products_watsonx_data.html": "https://www.ibm.com/products/watsonx-data",
"products_watsonx_assistant.html": "https://www.ibm.com/products/watsonx-assistant",
"products_watsonx_code_assistant.html": "https://www.ibm.com/products/watsonx-code-assistant",
"products_watsonx_orchestrate.html": "https://www.ibm.com/products/watsonx-orchestrate",
"products_watsonx_governance.html": "https://www.ibm.com/products/watsonx-governance",
"granite_code_models_open_source.html": "https://research.ibm.com/blog/granite-code-models-open-source",
"red_hat_enterprise_linux_ai.html": "https://www.redhat.com/en/about/press-releases/red-hat-delivers-accessible-open-source-generative-ai-innovation-red-hat-enterprise-linux-ai",
"model_choice.html": "https://www.ibm.com/blog/announcement/enterprise-grade-model-choices/",
"democratizing.html": "https://www.ibm.com/blog/announcement/democratizing-large-language-model-development-with-instructlab-support-in-watsonx-ai/",
"ibm_consulting_expands_ai.html": "https://newsroom.ibm.com/Blog-IBM-Consulting-Expands-Capabilities-to-Help-Enterprises-Scale-AI",
"ibm_data_product_hub.html": "https://www.ibm.com/products/data-product-hub",
"ibm_price_performance_data.html": "https://www.ibm.com/blog/announcement/delivering-superior-price-performance-and-enhanced-data-management-for-ai-with-ibm-watsonx-data/",
"ibm_bi_adoption.html": "https://www.ibm.com/blog/a-new-era-in-bi-overcoming-low-adoption-to-make-smart-decisions-accessible-for-all/",
"watsonx_code_assistant_for_z.html": "https://www.ibm.com/blog/announcement/ibm-watsonx-code-assistant-for-z-accelerate-the-application-lifecycle-with-generative-ai-and-automation/",
"code_assistant_for_java.html": "https://www.ibm.com/blog/announcement/watsonx-code-assistant-java/",
"code_assistant_for_orchestrate.html": "https://www.ibm.com/blog/announcement/watsonx-orchestrate-ai-z-assistant/",
"accelerating_gen_ai.html": "https://newsroom.ibm.com/Blog-How-IBM-Cloud-is-Accelerating-Business-Outcomes-with-Gen-AI",
"watsonx_open_source.html": "https://newsroom.ibm.com/2024-05-21-IBM-Unveils-Next-Chapter-of-watsonx-with-Open-Source,-Product-Ecosystem-Innovations-to-Drive-Enterprise-AI-at-Scale",
"ibm_concert.html": "https://www.ibm.com/products/concert",
"ibm_consulting_advantage_news.html": "https://newsroom.ibm.com/2024-01-17-IBM-Introduces-IBM-Consulting-Advantage,-an-AI-Services-Platform-and-Library-of-Assistants-to-Empower-Consultants",
"ibm_consulting_advantage_info.html": "https://www.ibm.com/consulting/info/ibm-consulting-advantage"
}
COLLECTION_NAME = "askibm_think_2024"
Next, let's load our documents using the LangChain UnstructuredURLLoader for the list of URLs we have. We'll print a sample document at the end to see how it's been loaded.
documents = []
for url in list(URLS_DICTIONARY.values()):
loader = UnstructuredURLLoader(urls=[url])
data = loader.load()
documents += data
#show sample document
documents[0]
Output:
Document(page_content="View session catalog\n\nEvent information\n\nThink 2024 will be held in Boston, MA.\xa0 It will open with an exclusive, partner-only event on 20 May, filled with inspiration, networking, knowledge-sharing and business value for IBM’s global ecosystem of partners. Then, on 21–23 May, we will bring together senior business and technology leaders from across industries for two and a half days of exploration, discussion and innovation.\n\nThink 2024 will host senior business and technology leaders from across industries. Content will be geared toward C-level, line of business and senior IT leaders.\n\nThink 2024 programming will be held at the Boston Convention & Exhibition Center (BCEC), with some activities at the Omni Boston Hotel at the Seaport.\n\nAt IBM, we are committed to sustainability and environmentally responsible event planning. We are proud to partner with two distinguished venues, each known for their exemplary sustainable practices. Our event will take place at the Boston Convention & Exhibition Center and Omni Seaport in Boston. These venues share our dedication to reducing environmental impact, conserving resources and promoting eco-friendly practices. We are excited to collaborate with them to ensure that Think 2024 and our special events not only deliver an exceptional experience, but also align with our commitment to a greener, more sustainable future.\n\nRegistration\n\nYes, the Think 2024 event in Boston will be a fee-based event. Standard pricing is USD 1,899, effective 16 January through 23 May 2024.\n\nRequests for cancellation of a paid Think pass must be made in writing to\[email protected]. Requests received by 19 April 2024, 11:59 PM ET will receive a full refund minus an administrative fee of USD 150. After midnight ET on Saturday, 20 April 2024, no refunds will be issued. Non-attendance does not constitute cancellation. If you have extenuating circumstances, please reach out with an e-mail to\[email protected].\n\nSubstitutions for purchased passes will be allowed for attendees from the same company, and an administrative fee of USD 150 will be applied to the new registration.\n\nSubstitutions for complimentary pass holders are not permitted. Please contact your IBM account representative or the Think Guest Services Team at\[email protected].\n\nIndividuals must be over the age of 21 to participate in conference activities. Infants and young children, even under the supervision or care of an adult, may not attend conference activities at the Boston Convention and Exhibition Center and Omni Boston Hotel. This includes all activities, sessions, meal functions, meeting and exposition space, and the use of event transportation if applicable. There may be stated exceptions where individuals under the age of 21 are allowed with a purchased guest conference badge.\n\nIf you are having a problem logging into your IBMid, or if you need to create an IBMid, please contact the\xa0IBMid help desk.\n\nPlease contact Think Guest Services via between 9 AM and 6 PM ET.\n\nUS/Canada: +1-866-382-7150\n\nInternational: +1-650-416-1881\n\nEmail:\[email protected]\n\nHotel and Travel\n\nThe official Think hotel blocks are closed. We may still have rooms available for new requests. To inquire about a new hotel booking, please email [email protected] and provide your arrival and departure information. Think Guest services will respond within 24 hours. For hotel changes or cancellations, please contact the hotel directly after you receive your hotel confirmation email with confirmation number.\n\nCancellation or changes to hotel reservations are the sole responsibility of the attendee. Cancellations within 72 hours of the reservation check-in date will result in a late cancellation fee of 1 night’s room and tax. If you do not check in on your scheduled arrival date, the remainder of your reservation will be canceled.\n\nYour hotel loyalty information can be added at the time of booking or with the hotel directly.\n\nFor hotel changes or cancellations, please contact the hotel directly and provide your hotel confirmation number which is included with your hotel confirmation email.\n\nIf you need to arrive early or extend your stay outside of the event dates and dates you booked with Think Guest Services, please contract the hotel directly and provide them with you hotel confirmation number. Rates will be based on the hotel's availability and are not guaranteed at the conference rate.\n\nIf you did not request at the time of booking your reservation with Think Guest Services, please contact the hotel directly. Please note that all requests are based on hotel availability and are not guaranteed.\n\nFor IBM clients and Business Partners requesting a visa invitation letter, please contact the Think Customer Service team via email at\[email protected] and include the following information:\n\nName on passport\n\nCompany name\n\nAddress\n\nCity\n\nCountry\n\nPlease note that an invitation letter will be provided to registered attendees only. Letters cannot be provided for guests.\n\nAt the event\n\nYes, IBM is dedicated to providing a safe, respectful, comfortable, and harassment-free environment for all participants at IBM events.\n\nLearn more (link resides outside ibm.com)\xa0\xa0about IBM's Code of Conduct at events.\n\nNo, unfortunately conference passes are limited and reserved for invited and registered attendees.\n\nThe conference attire is business casual. Comfortable shoes are recommended. Weather conditions can vary, so please review the local weather prior to your departure for the conference.\n\nShuttle service will not be provided from conference hotels to the Boston Convention and Exhibition Center (BCEC). The recommended options for transportation would be through your choice of Uber/Lyft/Taxi or a short walk. Hotels are within a 3 mile radius to the BCEC.\n\nGround transportation is not provided from/to the airport. As attendees will arrive and depart at different times, the recommended options for transportation would be through your choice of Uber/Lyft/Taxi or a scheduled private pickup.\n\nNo, unfortunately attendance to the Wednesday evening “Blue Block Party” special event is limited and reserved for invited and registered attendees.\n\nTo self-park at the BCEC ($25 and $50 for oversized vehicles), from Summer Street, turn onto East Side Drive, drive past the valet area, and continue straight along the side of the building. At the end of the building, make a right and go down the ramp. At the bottom of the ramp, turn left and you will see the entrance to the South Parking lot in front of you. Accessible parking spaces are also located in this lot, closest to the building (the same parking rates apply).\n\nView nearby parking lots and garage information here\xa0(link resides outside ibm.com).\n\nDiversity and Inclusion\n\nIBM is committed to creating a respectful, friendly and inclusive experience for all participants. Diversity, equity and inclusion are core components of IBM's culture. IBM's conferences include elements such as accessibility, prayer/meditation rooms, and parent/mother rooms to help foster a deeper sense of inclusion and a welcoming experience for all participants.\n\nIf you have an accessibility-related need, please let us know in your registration profile or email Guest Services directly at [email protected], ideally at least two to three weeks prior to the conference. Once we receive your request, one of our ambassadors will contact you.\n\nWhen we know, you’ll know\n\nGet updates on speakers, sessions and essential conference information, delivered right to your inbox.\n\nSubscribe", metadata={'source': 'https://www.ibm.com/events/think/faq'})
Based on the sample document, it looks like there's a lot of white space and new line characters that we can get rid of. Let's clean that up and add some metadata to our documents, including an ID number and the source of the content.
doc_id = 0
for doc in documents:
doc.page_content = " ".join(doc.page_content.split()) # remove white space
doc.metadata["id"] = doc_id #make a document id and add it to the document metadata
print(doc.metadata)
doc_id += 1
Output:
{'source': 'https://www.ibm.com/events/think/faq', 'id': 0}
{'source': 'https://www.ibm.com/events/think/agenda', 'id': 1}
{'source': 'https://www.ibm.com/products/watsonx-ai', 'id': 2}
{'source': 'https://www.ibm.com/products/watsonx-ai/foundation-models', 'id': 3}
{'source': 'https://www.ibm.com/watsonx/pricing', 'id': 4}
{'source': 'https://www.ibm.com/watsonx', 'id': 5}
{'source': 'https://www.ibm.com/products/watsonx-data', 'id': 6}
{'source': 'https://www.ibm.com/products/watsonx-assistant', 'id': 7}
{'source': 'https://www.ibm.com/products/watsonx-code-assistant', 'id': 8}
{'source': 'https://www.ibm.com/products/watsonx-orchestrate', 'id': 9}
{'source': 'https://www.ibm.com/products/watsonx-governance', 'id': 10}
{'source': 'https://research.ibm.com/blog/granite-code-models-open-source', 'id': 11}
{'source': 'https://www.redhat.com/en/about/press-releases/red-hat-delivers-accessible-open-source-generative-ai-innovation-red-hat-enterprise-linux-ai', 'id': 12}
{'source': 'https://www.ibm.com/blog/announcement/enterprise-grade-model-choices/', 'id': 13}
{'source': 'https://www.ibm.com/blog/announcement/democratizing-large-language-model-development-with-instructlab-support-in-watsonx-ai/', 'id': 14}
{'source': 'https://newsroom.ibm.com/Blog-IBM-Consulting-Expands-Capabilities-to-Help-Enterprises-Scale-AI', 'id': 15}
{'source': 'https://www.ibm.com/products/data-product-hub', 'id': 16}
{'source': 'https://www.ibm.com/blog/announcement/delivering-superior-price-performance-and-enhanced-data-management-for-ai-with-ibm-watsonx-data/', 'id': 17}
{'source': 'https://www.ibm.com/blog/a-new-era-in-bi-overcoming-low-adoption-to-make-smart-decisions-accessible-for-all/', 'id': 18}
{'source': 'https://www.ibm.com/blog/announcement/ibm-watsonx-code-assistant-for-z-accelerate-the-application-lifecycle-with-generative-ai-and-automation/', 'id': 19}
{'source': 'https://www.ibm.com/blog/announcement/watsonx-code-assistant-java/', 'id': 20}
{'source': 'https://www.ibm.com/blog/announcement/watsonx-orchestrate-ai-z-assistant/', 'id': 21}
{'source': 'https://newsroom.ibm.com/Blog-How-IBM-Cloud-is-Accelerating-Business-Outcomes-with-Gen-AI', 'id': 22}
{'source': 'https://newsroom.ibm.com/2024-05-21-IBM-Unveils-Next-Chapter-of-watsonx-with-Open-Source,-Product-Ecosystem-Innovations-to-Drive-Enterprise-AI-at-Scale', 'id': 23}
{'source': 'https://www.ibm.com/products/concert', 'id': 24}
{'source': 'https://newsroom.ibm.com/2024-01-17-IBM-Introduces-IBM-Consulting-Advantage,-an-AI-Services-Platform-and-Library-of-Assistants-to-Empower-Consultants', 'id': 25}
{'source': 'https://www.ibm.com/consulting/info/ibm-consulting-advantage', 'id': 26}
Let's see how our sample document looks now after we cleaned it up:
documents[0]
Output:
Document(page_content="View session catalog Event information Think 2024 will be held in Boston, MA. It will open with an exclusive, partner-only event on 20 May, filled with inspiration, networking, knowledge-sharing and business value for IBM’s global ecosystem of partners. Then, on 21–23 May, we will bring together senior business and technology leaders from across industries for two and a half days of exploration, discussion and innovation. Think 2024 will host senior business and technology leaders from across industries. Content will be geared toward C-level, line of business and senior IT leaders. Think 2024 programming will be held at the Boston Convention & Exhibition Center (BCEC), with some activities at the Omni Boston Hotel at the Seaport. At IBM, we are committed to sustainability and environmentally responsible event planning. We are proud to partner with two distinguished venues, each known for their exemplary sustainable practices. Our event will take place at the Boston Convention & Exhibition Center and Omni Seaport in Boston. These venues share our dedication to reducing environmental impact, conserving resources and promoting eco-friendly practices. We are excited to collaborate with them to ensure that Think 2024 and our special events not only deliver an exceptional experience, but also align with our commitment to a greener, more sustainable future. Registration Yes, the Think 2024 event in Boston will be a fee-based event. Standard pricing is USD 1,899, effective 16 January through 23 May 2024. Requests for cancellation of a paid Think pass must be made in writing to [email protected]. Requests received by 19 April 2024, 11:59 PM ET will receive a full refund minus an administrative fee of USD 150. After midnight ET on Saturday, 20 April 2024, no refunds will be issued. Non-attendance does not constitute cancellation. If you have extenuating circumstances, please reach out with an e-mail to [email protected]. Substitutions for purchased passes will be allowed for attendees from the same company, and an administrative fee of USD 150 will be applied to the new registration. Substitutions for complimentary pass holders are not permitted. Please contact your IBM account representative or the Think Guest Services Team at [email protected]. Individuals must be over the age of 21 to participate in conference activities. Infants and young children, even under the supervision or care of an adult, may not attend conference activities at the Boston Convention and Exhibition Center and Omni Boston Hotel. This includes all activities, sessions, meal functions, meeting and exposition space, and the use of event transportation if applicable. There may be stated exceptions where individuals under the age of 21 are allowed with a purchased guest conference badge. If you are having a problem logging into your IBMid, or if you need to create an IBMid, please contact the IBMid help desk. Please contact Think Guest Services via between 9 AM and 6 PM ET. US/Canada: +1-866-382-7150 International: +1-650-416-1881 Email: [email protected] Hotel and Travel The official Think hotel blocks are closed. We may still have rooms available for new requests. To inquire about a new hotel booking, please email [email protected] and provide your arrival and departure information. Think Guest services will respond within 24 hours. For hotel changes or cancellations, please contact the hotel directly after you receive your hotel confirmation email with confirmation number. Cancellation or changes to hotel reservations are the sole responsibility of the attendee. Cancellations within 72 hours of the reservation check-in date will result in a late cancellation fee of 1 night’s room and tax. If you do not check in on your scheduled arrival date, the remainder of your reservation will be canceled. Your hotel loyalty information can be added at the time of booking or with the hotel directly. For hotel changes or cancellations, please contact the hotel directly and provide your hotel confirmation number which is included with your hotel confirmation email. If you need to arrive early or extend your stay outside of the event dates and dates you booked with Think Guest Services, please contract the hotel directly and provide them with you hotel confirmation number. Rates will be based on the hotel's availability and are not guaranteed at the conference rate. If you did not request at the time of booking your reservation with Think Guest Services, please contact the hotel directly. Please note that all requests are based on hotel availability and are not guaranteed. For IBM clients and Business Partners requesting a visa invitation letter, please contact the Think Customer Service team via email at [email protected] and include the following information: Name on passport Company name Address City Country Please note that an invitation letter will be provided to registered attendees only. Letters cannot be provided for guests. At the event Yes, IBM is dedicated to providing a safe, respectful, comfortable, and harassment-free environment for all participants at IBM events. Learn more (link resides outside ibm.com) about IBM's Code of Conduct at events. No, unfortunately conference passes are limited and reserved for invited and registered attendees. The conference attire is business casual. Comfortable shoes are recommended. Weather conditions can vary, so please review the local weather prior to your departure for the conference. Shuttle service will not be provided from conference hotels to the Boston Convention and Exhibition Center (BCEC). The recommended options for transportation would be through your choice of Uber/Lyft/Taxi or a short walk. Hotels are within a 3 mile radius to the BCEC. Ground transportation is not provided from/to the airport. As attendees will arrive and depart at different times, the recommended options for transportation would be through your choice of Uber/Lyft/Taxi or a scheduled private pickup. No, unfortunately attendance to the Wednesday evening “Blue Block Party” special event is limited and reserved for invited and registered attendees. To self-park at the BCEC ($25 and $50 for oversized vehicles), from Summer Street, turn onto East Side Drive, drive past the valet area, and continue straight along the side of the building. At the end of the building, make a right and go down the ramp. At the bottom of the ramp, turn left and you will see the entrance to the South Parking lot in front of you. Accessible parking spaces are also located in this lot, closest to the building (the same parking rates apply). View nearby parking lots and garage information here (link resides outside ibm.com). Diversity and Inclusion IBM is committed to creating a respectful, friendly and inclusive experience for all participants. Diversity, equity and inclusion are core components of IBM's culture. IBM's conferences include elements such as accessibility, prayer/meditation rooms, and parent/mother rooms to help foster a deeper sense of inclusion and a welcoming experience for all participants. If you have an accessibility-related need, please let us know in your registration profile or email Guest Services directly at [email protected], ideally at least two to three weeks prior to the conference. Once we receive your request, one of our ambassadors will contact you. When we know, you’ll know Get updates on speakers, sessions and essential conference information, delivered right to your inbox. Subscribe", metadata={'source': 'https://www.ibm.com/events/think/faq', 'id': 0})
We need to split up our text into smaller, more manageable pieces known as "chunks." LangChain's RecursiveCharacterTextSplitter takes a large text and splits it based on a specified chunk size using a predefined set of characters. In order, the default characters are:
- "\n\n" - two new line characters
- "\n" - one new line character
- " " - a space
- "" - an empty character
The process starts by attempting to split the text using the first character, "\n\n."" If the resulting chunks are still too large, it moves to the next character, "\n," and tries splitting again. This continues with each character in the set until the chunks are smaller than the specified chunk size. Since we already removed all the "\n\n" and "\n" characters when we cleaned up the text, the RecursiveCharacterTextSplitter will begin at the " " (space) character.
We settled on a chunk size of 512 after experimenting with a chunk size of 1000. When the chunks were that large, our model was getting too much context for question-answering. This led to confused responses by the LLM because it was receiving too much information, so we changed it to smaller chunks. Feel free to experiment with chunk size further!
text_splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=0)
docs = text_splitter.split_documents(documents)
Next, we chose an embedding model to be trained on our Think 2024 data set. The trained embedding model is used to generate embeddings for each data point in the dataset. For text data, popular open-source embedding models include Word2Vec, GloVe, FastText or pre-trained transformer-based models like BERT or RoBERTa. OpenAIembeddings can also be used by leveraging the OpenAI embeddings API endpoint, the langchain_openai package and getting an openai_api_key, however, there is a cost associated with this usage.
Unfortunately, because the embedding models are so large, vector embeddings often demand significant computational resources. We can greatly lower the costs linked to embedding vectors, while preserving performance and accuracy, by using WatsonxEmbeddings. We'll use the IBM embeddings model, Slate, an encoder-only (RoBERTa-based) model, which while not generative, is fast and effective for many NLP tasks.
embeddings = WatsonxEmbeddings(
model_id=EmbeddingTypes.IBM_SLATE_30M_ENG.value,
url=credentials["url"],
apikey=credentials["apikey"],
project_id=project_id,
)
Let's load our content into a local instance of a vector database, using Chroma.
vectorstore = Chroma.from_documents(documents=docs, embedding=embeddings)
Let's do a quick search of our vector database to test it out! Using similarity_search_with_score allows us to return the documents and the distance score of the query to them. The returned distance score is Euclidean distance. Therefore, a lower score is better.
prompt = "What is IBM concert?"
search = vectorstore.similarity_search_with_score(prompt)
search
Output:
[(Document(page_content='and solving problems before they happen. Concert will initially focus on helping application owners, SREs and IT leaders gain insights about, pre-empt and more quickly address issues around application risk and compliance management. Read this blog to learn more about IBM Concert. IBM expands ecosystem access to watsonx, adds third-party models IBM continues to foster a strong ecosystem of partners to offer clients choice and flexibility through bringing third-party models onto watsonx, enabling leading', metadata={'id': 23, 'source': 'https://newsroom.ibm.com/2024-05-21-IBM-Unveils-Next-Chapter-of-watsonx-with-Open-Source,-Product-Ecosystem-Innovations-to-Drive-Enterprise-AI-at-Scale'}),
0.3013177514076233),
(Document(page_content='can easily move to and operate across multi-cloud and hybrid cloud environments. Now, at THINK, IBM is continuing to advance its state-of-the-art in automation portfolio by previewing a new generative AI-powered tool called IBM Concert, which will be generally available in June 2024. IBM Concert will serve as the ‘nerve center’ of an enterprise’s technology and operations. Powered by AI from watsonx, IBM Concert will provide generative AI-driven insights across clients’ portfolios of applications to', metadata={'id': 23, 'source': 'https://newsroom.ibm.com/2024-05-21-IBM-Unveils-Next-Chapter-of-watsonx-with-Open-Source,-Product-Ecosystem-Innovations-to-Drive-Enterprise-AI-at-Scale'}),
0.3116500675678253),
(Document(page_content='IBM Concert Simplify and optimize your app management and technology operations with generative AI-driven insights Book a live demo Prioritize and act on the most significant risks to your business applications Explore risk management with IBM Concert Keep applications continuously compliant, even as they evolve Explore compliance management with IBM Concert What is IBM Concert? Get the overview IBM® Concert® puts you in control, so you can simplify and optimize your operations to focus on continuously', metadata={'id': 24, 'source': 'https://www.ibm.com/products/concert'}),
0.31172141432762146),
(Document(page_content='documentation for details. Context length is expressed in tokens. The IBM statements regarding its plans, directions and intent are subject to change or withdrawal without notice at its sole discretion. See Pricing for more details. Unless otherwise specified under Software pricing, all features, capabilities and potential updates refer exclusively to SaaS. IBM makes no representation that SaaS and software features and capabilities are the same.', metadata={'id': 3, 'source': 'https://www.ibm.com/products/watsonx-ai/foundation-models'}),
0.43464165925979614)]
Let's ask "Where is Think 2024?"
prompt = "Where is Think 2024?"
search = vectorstore.similarity_search_with_score(prompt)
search
Output:
[(Document(page_content='keynotes from Think 2024 and explore related content in the Think Hub. Watch on-demand Stay in the know. Catch up on all the action from Think 2024 and watch event replays. Watch on-demand', metadata={'id': 1, 'source': 'https://www.ibm.com/events/think/agenda'}),
0.20512934029102325),
(Document(page_content='View session catalog Event information Think 2024 will be held in Boston, MA. It will open with an exclusive, partner-only event on 20 May, filled with inspiration, networking, knowledge-sharing and business value for IBM’s global ecosystem of partners. Then, on 21–23 May, we will bring together senior business and technology leaders from across industries for two and a half days of exploration, discussion and innovation. Think 2024 will host senior business and technology leaders from across industries.', metadata={'id': 0, 'source': 'https://www.ibm.com/events/think/faq'}),
0.3093597888946533),
(Document(page_content='Think 2024 Think 2024 2024-05-20 Monday: IBM Partner Plus Day Programming for Business Partners 2024-05-21 Tuesday: Think programming kicks off Event registration begins. Scaling your business with AI and hybrid cloud. Join IBM and other industry experts for engaging sessions and demos in the Meeting Center. Unleash the transformative power of AI and automation. Chef-selected lunch items and connections with your peers. Full lunch menu in the Think mobile app. Scale productivity with watsonx assistants.', metadata={'id': 1, 'source': 'https://www.ibm.com/events/think/agenda'}),
0.32775259017944336),
(Document(page_content='Seaport in Boston. These venues share our dedication to reducing environmental impact, conserving resources and promoting eco-friendly practices. We are excited to collaborate with them to ensure that Think 2024 and our special events not only deliver an exceptional experience, but also align with our commitment to a greener, more sustainable future. Registration Yes, the Think 2024 event in Boston will be a fee-based event. Standard pricing is USD 1,899, effective 16 January through 23 May 2024. Requests', metadata={'id': 0, 'source': 'https://www.ibm.com/events/think/faq'}),
0.3494681119918823)]
Step 5. Set up a retriever
We'll set up our vector store as a retriever. The retrieved information from the vector store serves as additional context or knowledge that can be used by a generative model.
retriever = vectorstore.as_retriever()
Step 6. Generate a response with a generative model
Finally, we’ll generate a response. The generative model (like GPT-4 or IBM Granite) uses the retrieved information to produce a more accurate and contextually relevant response to our questions about Think 2024.
First, we'll establish which LLM we're going to use to generate the response. For this tutorial, we'll use IBM's Granite 13B Chat model.
model_id = "ibm/granite-3-8b-instruct"
The model parameters available can be found here. We experimented with various model parameters, including Temperature, Top P, and Top K. Here's some more information on model parameters and what they mean.
parameters = {
GenParams.DECODING_METHOD: 'greedy',
GenParams.TEMPERATURE: 2,
GenParams.TOP_P: 0,
GenParams.TOP_K: 100,
GenParams.MIN_NEW_TOKENS: 10,
GenParams.MAX_NEW_TOKENS: 512,
GenParams.REPETITION_PENALTY:1.2,
GenParams.STOP_SEQUENCES:['B)','\n'],
GenParams.RETURN_OPTIONS: {'input_tokens': True,'generated_tokens': True, 'token_logprobs': True, 'token_ranks': True, }
}
Next, we instantiate the LLM.
llm = WatsonxLLM(
model_id=model_id,
url=credentials.get("url"),
apikey=credentials.get("apikey"),
project_id=project_id,
params=parameters
)
We'll set up a prompttemplate to ask multiple questions. The "context" will be derived from our retriever (our vector database) with the relevant documents and the "question" will be derived from the user query.
template = """Generate a summary of the context that answers the question. Explain the answer in multiple steps if possible.
Answer style should match the context. Ideal Answer Length 2-3 sentences.\n\n{context}\nQuestion: {question}\nAnswer:
"""
prompt = ChatPromptTemplate.from_template(template)
Let's set up a helper function to format the docs accordingly:
def format_docs(docs):
return "\n\n".join([d.page_content for d in docs])
And now we can set up a chain with our context, our prompt and our LLM. The generative model processes the augmented context along with the user's question to produce a response.
chain = (
{"context": retriever | format_docs, "question": RunnablePassthrough()}
| prompt
| llm
| StrOutputParser()
)
And now we can ask multiple questions:
chain.invoke("Where is Think 2024?")
Output:
'\nAnswer: The location of Think 2024 is Boston, Massachusetts, USA.'
Let's ask about IBM Concert next.
chain.invoke("What is IBM Concert?")
Output:
'A) A new product announcement from IBM regarding their automation portfolio\n'
And finally, let's ask what IBM Think 2024 is.
chain.invoke("What is IBM Think 2024?")
Output:
"A: IBM's annual user conference that brings together senior business and technology leaders from across industries for exploration, discussion, and innovation around artificial intelligence, data and automation capabilities, and hybrid cloud solutions."
And that's it! Feel free to ask even more questions!
Summary and next steps
In this tutorial, you created a LangChain RAG system in Python with watsonx. You fetched 27 articles from https://www.ibm.com to create a vector store as context for an LLM to answer questions about the Think 2024 conference.
You can imagine a situation where we can create chatbots to field these questions.
We encourage you to check out the LangChain documentation page for more information and tutorials on RAG.
Try watsonx for free
Build an AI strategy for your business on one collaborative AI and data platform called IBM watsonx, which brings together new generative AI capabilities, powered by foundation models, and traditional machine learning into a powerful platform spanning the AI lifecycle. With watsonx.ai, you can train, validate, tune, and deploy models with ease and build AI applications in a fraction of the time with a fraction of the data.
Try watsonx.ai, the next-generation studio for AI builders.
Next steps
Explore more articles and tutorials about watsonx on IBM Developer.