How to talk to your knowledge base?

Let's create a conversational agent that loads our own knowledge base and can chat about it. We'll do that in less than 30 lines of Python code.

We'll need a recent python and a couple of libraries:

pip install llama-index langchain PyPDF2

Initialise OpenAI key from the environment variable.

import openai
import os

openai.api_key = os.environ["OPENAI_API_KEY"]

Load folder with our knowledge base into a vector store (I have text files and PDFs there):

from llama_index import GPTVectorStoreIndex, ServiceContext, SimpleDirectoryReader

docs = SimpleDirectoryReader('/Users/rinat/brain').load_data()
ctx = ServiceContext.from_defaults(chunk_size_limit=512)
index = GPTVectorStoreIndex.from_documents(docs, service_context=ctx)

Teach agent how to use this vector store, by turning GPT Index into a Tool:

from langchain.agents import Tool
tools = [
    Tool(
        name = "GPT Index",
        func=lambda q: str(index.as_query_engine().query(q)),
        description="useful for when you want to answer questions about Rinat. The input to this tool should be a complete english sentence.",
        return_direct=True
    ),
]

Create a chat agent. Give it memory, connect to LLM and give it a Tool:

from langchain.chains.conversation.memory import ConversationBufferMemory
from langchain import OpenAI
from langchain.agents import initialize_agent

agent_type = "conversational-react-description"
memory = ConversationBufferMemory(memory_key="chat_history")
llm = OpenAI(temperature=0)
agent_chain = initialize_agent(tools, llm, agent=agent_type, memory =memory, verbose=True)

Let's ask this agent a question:

agent_chain.run(input="What does Rinat say about event sourcing?")

This will launch an agent chain:

> Entering new AgentExecutor chain...
Thought: Do I need to use a tool? Yes
Action: GPT Index
Action Input: What does Rinat say about event sourcing?
Observation: Rinat Abdullin states that event sourcing can be used to represent the entire state of an Aggregate as a sequence of Events that have occurred since it was created. He also states that event sourcing guarantees that the reason for each change to an Aggregate instance will not be lost, and that it can be invaluable for the business. He further states that the append-only nature of Event Streams performs outstandingly well and supports an array of data replication options. Finally, he states that the Event-centric approach to Aggregate design can allow developers to focus more of their attention on behaviors expressed by the Ubiquitous Language.

> Finished chain.

Output will be:

Rinat Abdullin states that event sourcing can be used to represent the entire state of an Aggregate as a sequence of Events that have occurred since it was created. He also states that event sourcing guarantees that the reason for each change to an Aggregate instance will not be lost, and that it can be invaluable for the business. He further states that the append-only nature of Event Streams performs outstandingly well and supports an array of data replication options. Finally, he states that the Event-centric approach to Aggregate design can allow developers to focus more of their attention on behaviors expressed by the Ubiquitous Language.

Let's check how well agent uses memory and LLM capabilities:

agent_chain.run(input="Answer the last question like I'm 5")

Note that agent doesn't use tools. It relies on data from the memory to provide an answer:

> Entering new AgentExecutor chain...
AI: Event sourcing is like a diary for a computer. It keeps track of all the changes that have been made to a computer, like when something was added or removed. This way, if something ever needs to be changed back, it can be done easily.

> Finished chain.

And the output is:

Event sourcing is like a diary for a computer. It keeps track of all the changes that have been made to a computer, like when something was added or removed. This way, if something ever needs to be changed back, it can be done easily.

Notes

You can give agents more tools. Here are a few examples: Bing Search, Python REPL, Wolfram Alpha or Zapier
GPTVectorStoreIndex is one of the simpler ways to index information. Depending on your knowledge base, you might need to craft a more capable index. For example, see How to segment texts for embeddings?
Obviously, there is no persistence in this sample. You'll need to add that separately.

Published: May 09, 2023.

Next post in Ship with ChatGPT story: Benchmarking LLM for business workloads

🤗 Check out my newsletter! It is about building products with ChatGPT and LLMs: latest news, technical insights and my journey. Check out it out