- Introduction
- Prompt Engineering
- Working with Prompts Across Models
- Building a Q/A Bot with ChatGPT
- Summary
This chapter is from the book
This chapter is from the book
This chapter is from the book
Building a Q/A Bot with ChatGPT
Let’s build a very simple Q/A bot using ChatGPT and the semantic retrieval system we built in Chapter 2. Recall that one of our API endpoints is used to retrieve documents from the BoolQ dataset given a natural query.
Here’s what we need to do to get off the ground:
Design a system prompt for ChatGPT.
Search for context in our knowledge with every new user message.
Inject any context we find from our database directly into ChatGPT’s system prompt.
Let ChatGPT do its job and answer the question.
Figure 3.12 outlines these high-level steps.
)
FIGURE 3.12 A 10,000-foot view of our chatbot, which uses ChatGPT to provide a conversational interface in front of our semantic search API.
To dig into this process a bit deeper, Figure 3.13 shows how this will work at the prompt level, step by step.
)
FIGURE 3.13 Starting from the top left and reading left to right, these four states represent how our bot is architected. Every time a user says something that surfaces a confident document from our knowledge base, that document is inserted directly into the system prompt, where we tell ChatGPT to use only documents from our knowledge base.
Let’s wrap all of this logic into a Python class, which will have a skeleton like that shown in Listing 3.1.
Listing 3.1 A ChatGPT Q/A bot
# Define a system prompt that gives the bot context throughout the
conversation and will be amended with content from our knowledge base.
SYSTEM_PROMPT = '''You are a helpful Q/A bot that can only reference material
from a knowledge base.
All context was pulled from a knowledge base.
If a user asks anything that is not "from the knowledge base," say that you cannot
answer.
'''
# Define the ChatbotGPT class
class ChatbotGPT():
# Define the constructor method for the class
def __init__(self, system_prompt, threshold=.8):
# Initialize the conversation list with the system prompt as the first turn
# Set a threshold for the similarity score between the user's input and the
knowledge base
pass
# Define a method to display the conversation in a readable format
def display_conversation(self):
# Iterate through each turn in the conversation
# Get the role and content of the turn
# Print out the role and content in a readable format
pass
# Define a method to handle the user's input
def user_turn(self, message):
# Add the user's input as a turn in the conversation
# Get the best matching result from the knowledge base using Pinecone
# Check if the confidence score between the user's input and the document
meets the threshold
# Add the context from the knowledge base to the system prompt if we meet the
threshold
# Generate a response from the ChatGPT model using OpenAI's API
# Add the GPT-3.5 response as a turn in the conversation
# Return the assistant's response
pass
A full implementation of this code using GPT-4 can be found in the book’s code repository. Figure 3.14 presents a sample conversation we can have with it.
)
FIGURE 3.14 Asking our bot about information from the BoolQ dataset yields cohesive and conversational answers. Asking about Barack Obama’s age (which is information not present in the knowledge base) causes the AI to politely decline to answer, even though that is general knowledge it would try to use otherwise.
As a part of testing, I decided to try something out of the box and built a new namespace in the same vector database (thank you, Pinecone). I then chunked documents out of a PDF of a Star Wars–themed card game I like. I wanted to use the chatbot to ask basic questions about the game and let ChatGPT retrieve portions of the manual to answer my questions. Figure 3.15 was the result.
)
FIGURE 3.15 The same architecture and system prompt against a new knowledge base of a card game manual. Now I can ask questions in the manual but my questions from BoolQ are no longer in scope.
Not bad at all, if I do say so myself.