Chroma Tutorial: Enhance Your Chatbot with Anthropic's Claude Model

Introduction

Anthropic's Claude Model Unveiled: Unlocking Unique Capabilities

Anthropic, a leading global AI company, introduces the revolutionary Claude Model. From Notion to Quora, major tech players are harnessing its versatility in conversations, text processing, and even coding. But what sets Claude apart? Dive into this distinctive tutorial exploring the remarkable features that make Claude stand out.

The Power of ChromaDB and Embeddings

Discover the pivotal role of embeddings in natural language processing and machine learning. Learn how these vector representations capture semantic meaning, enabling similarity-based text searches. Explore ChromaDB, an open-source embedding database, and its ability to store and query embeddings using document representations. Delve into this enlightening discussion on the dynamic world of embeddings and ChromaDB.

Prerequisites

Basic knowledge of Python
Access to Anthropic's Claude API
Access to OpenAI's API (for embedding function)
A Chroma database set up

Outline

Initializing the Project
Setting Up the Required Libraries
Write the Project Files
Testing the Basic Chatbot without Context Awareness
Making Use of Claude's Large Context Window
Testing the Chatbot with Context Awareness
Providing Contextual Information to Claude
Evaluation of the Enhanced Chatbot (Additional Context Provided)
Building a Knowledge Base for Claude
Testing Claude's Final Form (Enhanced by the Knowledge Base)
Discussion

Initializing the Project

Alright, let's roll up our sleeves and start coding! First things first, we need to initialize the project. We're going to name it chroma-claude this time. So, open up your favorite terminal, navigate to your preferred coding or project directory, and create a new directory for chroma-claude. Here's how you do it:

Next up, we'll create a new virtual environment for this project. If you're wondering why, well, a virtual environment helps us keep our project's dependencies separate from our global Python environment. This isolation is a lifesaver when you're juggling multiple projects with different dependencies.

But that's not the only advantage of a virtual environment! It also makes it easier for us to keep track of the specific libraries and versions we're using. We do this by 'freezing' them into a requirements.txt file, which is a handy record of all the dependencies your project needs to run.

Let's go ahead and set up that virtual environment!

To create a new virtual environment, we'll use the following command in our terminal:

python3 -m venv env

Once we've created the environment, we'll need to activate it. This step is slightly different depending on whether you're using Windows or a Unix-based system (like Linux or MacOS). Here's how to do it:

If you're on Windows, you'd use the following command in your terminal:
```
.
\env\Scripts\activate
```
If you're on Linux or MacOS, the command will look like this:
```
source env/bin/activate
```

After running the appropriate command, you should see the name of your environment (env, in this case) appearing in parentheses at the start of your terminal prompt. This means the environment is activated and ready to go! Here's what it should look like:

Great job! You've set up and activated your virtual environment.

Setting Up the Required Libraries

Next up, we need to install all the required libraries. If you've read my other tutorials about Chroma and OpenAI models, you'll find our installation process pretty familiar. This time, however, we'll also be installing the anthropic library.

Here's a quick rundown of the libraries we'll be using:

chromadb library: We'll use this to store and query the embeddings.
anthropic library: This is necessary to interact with Anthropic's Claude model.
halo library: This library provides cool loading indicators that show up while waiting for requests.

Let's start installing these libraries. I'll be using pip3 for installation. If you're more comfortable with pip, feel free to switch to that.

pip install chromadb anthropic halo

Write the Project Files

It's time to dive back into the coding part! Ensure we're still in our project directory. Open your favorite IDE or code editor and create a new file, let's name it main.py. This will be the sole Python file we'll need for this project.

Step 1: Import Necessary Libraries

Start by importing the necessary libraries. These include dotenv for loading environment variables, os for system-level operations, anthropic for interacting with the Claude model, pprint for pretty-printing, and halo for displaying a loading spinner.

Step 2: Load Environment Variables

Load the environment variables from your .env file. This is where you store sensitive data like your Anthropic API key.

Step 3: Define Response Generation Function

Define a function called generate_response() that takes in messages as input and returns a model-generated response. Inside this function, you first initialize a loading spinner that will run while the model is generating a response.

Step 4: Create an Anthropic Client and Send Completion Request

Next, create an Anthropic client using your API key and use it to send a completion request to the Claude model. The request includes your input messages, the model name, and the stop sequences to signal the end of a conversation turn.

Step 5: Stop the Spinner and Print the Request and Response

Once the response is received, stop the loading spinner. Print both the request and response to the console for debugging purposes. Then, return the response.

Step 6: Define Main Function

Define the main function. This is where you start a continuous loop to get user input and generate responses.

Step 7: Prepare the Prompt and Handle 'quit' Command

Inside the loop, you first prepare the prompt for the Claude model by concatenating the user's input text with the necessary conversation markers. You also handle the "quit" command: if the user types "quit", you break the loop to stop the chat.

Step 8: Generate Response and Print It

Next, you call the generate_response() function with the prepared messages to generate a response from the Claude model. Once the response is received, you print it to the console, with the prefix "Claude:" to denote the chatbot's response.

Step 9: Run the Main Function

Lastly, you include a conditional statement to ensure the main function runs only if the script is run directly (not imported as a module).

Basic Chatbot (No Context Awareness)

If you've followed the steps correctly, your complete main.py script should look like this:

...

This script will enable basic chatbot interaction with the Claude model, however, it does not maintain context between user inputs. This means that each input is treated as a separate conversation, and the model does not remember previous inputs.

.env

Remember the part when we load variables from the environment with load_dotenv() function? This is the file where the function takes the variables and their values. Let's create the file and write the variables and their values as follows:

CLAUDE_KEY=your_api_key_here

Please do note that the .env file contains sensitive information, and therefore you should keep the file in a safe place, never share these values, particularly the CLAUDE_KEY, which should be kept secret.

requirements.txt

Finally, while this part is optional, it's considered a best practice to record our dependencies list into a file called requirements.txt. This allows other developers to easily install the required libraries with one simple command.

To create the requirements.txt file, navigate to your project's virtual environment where all the necessary dependencies are installed, and run:

pip freeze > requirements.txt

Then, anyone who wants to run your project will be able to install all the required dependencies using this command:

pip install -r requirements.txt

This command is typically run by someone who wants to install or run your project. It's good practice to include it in your project's documentation, under a section about how to install or run your project.

Testing the Basic Chatbot without Context Awareness

Now, let's test our first iteration of the Claude chatbot! While this basic version should work well, it lacks 'context awareness' - a term used to describe a chatbot's ability to remember and refer back to prior parts of the conversation. This means that after each response, our bot will forget what the user has previously said.

To test the chatbot, run the main.py script with the python or python3 command in your terminal:

python main.py

If everything has been set up correctly, the terminal should display an input prompt, inviting you to enter a message. Try typing something and hitting enter. You will notice a loading indicator appear while the bot generates a response.

After a few seconds, the terminal will display the bot's response.

Now, try asking the bot a follow-up question based on its previous response. You will notice that Claude does not remember the context of the previous message.

Remember that if you encounter any issues while running the script, check the error message in the terminal and try to troubleshoot accordingly.

Making Use of Claude's Large Context Window

So, we're using a model with a really large context window. It would be a tragic waste to only use it for a conversation that it forgets instantly. Let's change our code a little. This time, our code should capture both the earlier chat, as well as the current user input.

This new block introduces a variable messages outside the main loop. This variable will hold the entire conversation history and will be updated continuously throughout the conversation.

Here, rather than just assigning the current user input to the messages variable, you're now appending it. This ensures that the messages variable maintains a record of all the user's inputs.

In this block, after generating a response from Claude, you append Claude's response to the messages variable. This ensures that both the user's inputs and Claude's responses are retained in the messages variable, giving Claude access to the full conversation history.

Finally, to accommodate the larger conversation histories that the messages variable might contain, you've increased max_tokens_to_sample from 300 to 1200. This instructs Claude to consider up to 1200 tokens when generating a response, which is especially important when dealing with longer conversations. However, do note that if a conversation exceeds the model's context window of 1200 tokens, the older parts of the conversation will be truncated or ignored, potentially leading to loss of context.

Now, let's take a look at our updated code:

...

Testing the Chatbot with Context Awareness

In this section, we'll put our context-aware chatbot to the test. Specifically, we'll ask Claude about the video game Dota2, and in particular about the latest patch, "New Frontier".

So far, so good. Now let's pose a second question: "What do you know about the latest patch, New Frontier?" Claude, despite being an advanced chatbot, confesses it doesn't have specific knowledge about the latest patch. This is due to the model's training cut-off, which ended in late 2021. In a more optimal scenario, Claude would still offer relevant, though perhaps outdated, information. In a less desirable scenario, Claude might generate what appears to be accurate but is in fact incorrect information a phenomenon known as "hallucination." It's crucial to be mindful of these limitations and exercise caution when prompting the model with queries about events or information postdating its training.

So, how can we navigate around this limitation and equip Claude with more recent information? Well, for the purposes of this tutorial, we will provide Claude with supplementary learning materials manually, by copying and pasting the data into text files, which we can then read in using Python's built-in functionality. While we're not technically making Claude "learn" in the conventional sense, we are providing fresh data that it can use to generate more up-to-date responses. Let's delve into this further in the next section.

Providing Contextual Information to Claude

In order to test Claude's full potential, we need to supply it with enough context-relevant data. For this tutorial, I have copied some text from wiki sites, like the one from IGN discussing the "New Frontiers" patch. This text is saved in a file named patch_notes.txt. Let's start by reading this file in our main function.

We read the text file and use its content to form the initial message for Claude. This message will supply Claude with context about the Dota2 patch, which it can refer to when responding to related queries. This method isn't technically making Claude "learn" new information, but it helps in generating accurate responses for this specific interaction.

We also add a token counter near the end of the main() function. This helps us monitor the total number of tokens used in the conversation, ensuring we do not exceed the model's maximum token limit. Here's how our updated code looks:

...

Evaluating the Enhanced Chatbot (Additional Context Provided)

Let's put our augmented chatbot to the test! First, we'll inquire about general information concerning Dota2's latest patch, "New Frontiers".

Impressively, Claude now enumerates all the major changes introduced by the patch with remarkable accuracy. However, it's noteworthy that the token count has already reached 5298. This reminds us that in-depth responses and extensive conversation history can rapidly consume tokens.

To further evaluate Claude's comprehension of the provided context, we'll ask it about the "Tormentors." This question can be seen as a litmus test of Claude's context awareness. To an uninformed listener, "Tormentors" might evoke images of cruel, menacing creatures, while in Dota2, they are actually benign, floating cubes.

Claude passed the test with flying colors! It's truly fascinating how appropriate prompts, coupled with additional context, can significantly enhance the accuracy of Claude's responses.

Building a Knowledge Base for Claude

The goal of building a knowledge base for Claude is to equip it with a vast array of data it can refer to when generating responses. This allows it to provide more accurate and context-aware responses, even when dealing with complex and diverse data, like the latest Dota2 patch notes.

To achieve this, we'll use the Chroma database as our knowledge base. This way, we can break the data into smaller pieces and store them in the database. This is more efficient than feeding the entire data to Claude every time we communicate with it. To break the data into smaller pieces, we'll use the tiktoken library.

First, let's install tiktoken and chromadb, and update our requirements.txt file:

pip install tiktoken chromadb

Next, we're going to introduce some changes to our script to enable Chroma database functionality:

First, we import all the required libraries:

from chromadb import Client
import tiktoken # <- get the tiktoken library

Now, let's create a function create_chunks to split our large text into smaller chunks. This function will return the successive chunks of text:

def create_chunks(text, max_length=512):
    return [text[i:i+max_length] for i in range(0, len(text), max_length)]

In the main function, we initialize the tokenizer, chunk the patch notes, and create a new Chroma collection. Each chunk of text is then added to the collection as a separate document:

def main():
    ...
    chunks = create_chunks(patch_notes)
    for chunk in chunks:
        chroma_collection.add_document(chunk)
    ...

During the conversation with the user, we query the Chroma collection for the top 20 documents relevant to the user input. The results from this query are then added to the context for Claude, improving its responses:

def main():
    ...
    relevant_docs = chroma_collection.query(user_input, top_k=20)
    context = "".join(relevant_docs) + previous_context
    response = generate_response(context)
    ...

Testing Claude's Final Form (Enhanced by the Knowledge Base)

We've finally arrived at the moment of truth! This tutorial may have been a bit longer than usual, but it's crucial for demonstrating Claude's large context window and the power of the Chroma database for enhancing Claude's knowledge base. So, let's put our enhanced Claude to the test!

In the first test, I'll ask Claude to list the three most significant game-changing updates in the latest Dota 2 patch.

Claude impressively identified the three most game-changing updates: the expanded map with 40% more terrain, the rework of the black king bar, and the new hero attribute type: Universal. Remember, we didn't feed Claude the entire patch notes at once. Instead, it's leveraging the power of its enhanced knowledge base to retrieve this relevant information.

For the next challenge, I'll ask Claude to explain what Tormentors are in the latest patch, as if I were about to enter a match completely blind.

Claude provides a general overview of Tormentors but misses some important details. For instance, it refers to "multiple Tormentors" when only two spawn at specific locations on the entire map. This highlights the need to continuously update the knowledge base with more accurate data to help Claude deliver more precise and helpful analysis.

These tests demonstrate the potential of AI models like Claude when combined with a rich, continuously updated knowledge base. They also show the continuous nature of AI development - as we feed our models with more and better data, they can deliver increasingly accurate and nuanced responses.

Conclusion: Unleashing the Power of Anthropic's Claude Model and Chroma Integration

In this comprehensive anthropic tutorial, we've explored the extraordinary capabilities of the Anthropic's Claude model and the Chroma database. The Claude model, with its expansive context window, has been instrumental in developing a chatbot that can read and understand extensive text data, providing detailed analysis on Dota 2 updates. The Chroma database has further amplified this capability, enabling the creation of a more efficient and scalable knowledge base, conserving token usage in the process.

While we've achieved notable success, this anthropic application tutorial has also highlighted potential areas for improvement. For instance, refining our model's understanding of specific game elements, such as the Tormentors in Dota 2, and continuously updating and expanding the knowledge base to ensure the chatbot's accuracy.

The potential for what we can build next with the Claude + Chroma combination is truly staggering. Whether it's a chatbot for in-depth game strategy analysis, a virtual assistant with expertise in a wide range of topics, or a powerful tool to help researchers make sense of vast amounts of data - the possibilities are limited only by our imagination, and yes, our disk space!