Retrieval-Augmented Generation (RAG) using verba
Extracting information from conversations between a Customer and an Agent, using Retrieval-Augmented Generation (RAG) with the verba library
The Basics
Large Language Models (LLMs) are trained on billions of parameters. This gives them a vast database of available information that they can use to answer questions, generate text, etc, but because this information is general, it is not likely that any given LLM will have targeted information about a specific topic of interest. It will try its best, but more than likely it will end up returning out-of-date or flat out incorrect information.
Retrieval-Augmented Generation (RAG) is the ability to prime a LLM with targeted data that covers a specific topic. It is analogous to a student in college. The student learned a variety of information during their high school career, but in order to pass a specific college class, they must study a specific subset of information that will be used to pass midterms and finals. RAG works the same way - it allows LLMs to build on their existing knowledge by referencing user-specified data, greatly enhancing their accuracy.
Introducing the verba library
The verba library is a Python package I wrote for working with LLMs and performing NLP tasks. It is designed to offer an alternative to the overly-abstracted methods found in commercialized packages (such as langchain). Included within is a module called Ragby (Retrieval-Augmented Generation By Yourself) which will be the focus of this exercise.
In order to use Ragby, you must first have the Ollama dependencies installed on your machine. The Ollama framework and Ollama Python library must both be installed. Once installed, an Ollama LLM needs to be downloaded for use. Any LLM should work, but here I will be using llama3.1, which can be downloaded using this terminal command:
$ ollama pull llama3.1
After that, the verba library can be installed using pip:
$ pip install verba
Setting up the RAG Pipeline
In this exercise we will be working with a transcription of a conversation between a Customer and an Agent. The transcript can be found here. Because this is a sample template conversation, I have filled out the blank placeholder values. As an example, I changed the (customer name) value to Mr. Phillips.
Note - I have included the full conversation text at the bottom of this article
AGENT: Welcome, Mr. Phillips, how are you today?
CUSTOMER: Welcome, I’m fine.
AGENT: With you Jim from the complaints and suggestions department, how can I help you today, Mr. Phillips?
CUSTOMER: I have a big problem with the product I bought from your site this week
...
The Ragby module requires that you store your data in its own directory, in the current working path. By default this directory is named data (this name can be modified on initialization using the 'data_dir=' arg). For this exercise, the coversation data file will be stored as "agent_customer_conversation.txt", so the full path will be "data/agent_customer_conversation.txt".
Once you have placed your data in the data dir, you can load and initialize the Ragby module. Initializing the Ragby module automatically creates the chunks and embeddings directories for you, used later to store the chunks and embeddings objects.
from verba.OllamaHelper import Ragby
ragby = Ragby(
chat_model = "llama3.1", # the chat model to be used
embedding_model = "llama3.1" # the model used to create embeddings
)
--- Creating 'chunks' folder
--- Creating 'embeddings' folder
Next, we will create the chunks and embeddings objects using the make_n_chunks() and make_embeddings() methods, respectively.
Note that for this exercise, we want to combine four lines of conversations as one chunk using the n = 4 arg.
chunks_obj, chunks_path = ragby.make_n_chunks("agent_customer_conversation.txt", n = 4)
--- Created chunks of length: 13
--- Each chunk contains 4 sub-chunks
--- Chunks stored in 'chunks/agent_customer_conversation.json'
embeddings_path = ragby.make_embeddings(chunks_obj, "agent_customer_conversation.txt")
--- Created embeddings of length: 13
--- Embeddings stored in 'embeddings/agent_customer_conversation.json'
Finally, to round out our setup, we define the System Prompt to be used in the chat method.
SYSTEM_PROMPT = "You are a helpful assistant who answers questions based on a CONVERSATION between an AGENT and a CUSTOMER. Answer only using the CONVERSATION provided, being as concise as possible. If you are unsure, just say that you don't know.\n\nCONVERSATION:\n"
With that out of the way, we can now move onto using the pipeline.
Utilizing the RAG Pipeline
Once the RAG Pipeline has been set up, we can now utilize it to retrieve information from our conversation data.
As an example, we can retrieve specific information from our data, such as the name of the Customer in the conversation:
QUESTION = "What is the named of the CUSTOMER? Provide no other information. If you do not know, answer with 'NA'."
CHAT = ragby.chat(
user_prompt = QUESTION,
system_prompt = SYSTEM_PROMPT,
text_chunks_path = chunks_path,
text_embeddings_path = embeddings_path
)
print(CHAT)
Phillips
As another example, here we ask for the time of the shipment:
QUESTION = "What is the time of the shipment?"
CHAT = ragby.chat(
user_prompt = QUESTION,
system_prompt = SYSTEM_PROMPT,
text_chunks_path = chunks_path,
text_embeddings_path = embeddings_path
)
print(CHAT)
The product was supposed to be delivered at 6 p.m., but the agent was two hours late. The exact time of the shipment is not specified in the conversation, only that it was on Wednesday at 6:00 p.m. and also stated as being 2 hours behind schedule.
When looking at the actual transcript, we can see that this is objectively correct.
Interestingly, the model picks up on the fact that the shipment was late:
...
CUSTOMER: ok I am with you now
AGENT: All right, the shipment was 6:00 p.m. on Wednesday, is it right?
CUSTOMER: The product was supposed to be delivered to me at 6 p.m., but I was surprised that the agent was two hours late!!!
AGENT: Ok Mr. Phillips, can you tell me the number of the representative on the bill?
...
Along with specific questions as seen above, we can also retrieve a summary of the conversation:
QUESTION = "Provide a bullet-point summary of the CONVERSATION"
CHAT = ragby.chat(
user_prompt = QUESTION,
system_prompt = SYSTEM_PROMPT,
text_chunks_path = chunks_path,
text_embeddings_path = embeddings_path
)
print(CHAT)
Here is a bullet-point summary:
* Customer contacts Agent with complaints about shipping representative
* Agent apologizes for being late and promises to help with complaint
* Agent informs customer that a representative from the department of inquiries will contact them to discuss product usage
* Agent acknowledges customer's thanks and company name (Doug's Fishing Imporium Limited)
* Agent asks customer to review delivery information on bill
* Customer shares problem with shipping representative: delayed, untidy shipment, and unacceptable attitude towards customer
Summary
In this article we have applied a RAG Pipeline to a transcript of a conversation between a Customer and an Agent, using the verba library.
Thanks for reading!
AGENT: Welcome, Mr. Phillips, how are you today?
CUSTOMER: Welcome, I’m fine.
AGENT: With you Jim from the complaints and suggestions department, how can I help you today, Mr. Phillips?
CUSTOMER: I have a big problem with the product I bought from your site this week
AGENT: Moments with me Mr. Phillips, I will review your request now
CUSTOMER: Ok
AGENT: Your request number 1234567 was on Wednesday correct Mr. Phillips?
CUSTOMER: Yup
AGENT: Ok, Mr. Phillips can you explain to me what is the problem exactly?
CUSTOMER: I didn’t like the product and I didn’t get it as the same picture on the company website, you are liars!
AGENT: I apologize very much to you Mr. Phillips but on our website, we put pictures of real products only, and the proof is the positive comments and evaluations from all our customers in Saudi Arabia
CUSTOMER: I don’t believe it; I believe my eyes who see the product
AGENT: Sorry, Mr. Phillips I’m not accusing you of lying at all, and I will solve this problem immediately, can you please explain to me in detail how the product is different from the picture?
CUSTOMER: In the picture, the size of the product is bigger than what I received
AGENT: Ok, Mr. Phillips if you opened our website, you would find the sizes written in detail in the product description, if you can look at it now while you are with me on the line?
CUSTOMER: Ok
AGENT: Take your time, Mr. Phillips
CUSTOMER: it is Written but the product in the picture is very big than the size that is written in the description?
AGENT: This is because of the angle of photography Mr. Phillips because we in the company must show all the details of the product to the customer, so the pictures are a little close to the product to show all the details to the customers, and we provide the product with a specific and clear description of materials and sizes and advantages and everything so that the customer knows the product that he will buy, are you still with me Mr. Phillips?
CUSTOMER: yes
AGENT: Are there any other differences than the size Mr. Phillips?
CUSTOMER: I don’t know how to use this product; I mean the way I used it and it didn’t work
AGENT: There is no problem Mr. Phillips I will send you an accurate description of the method of use and will follow up with you one of the customer service representatives in the department of inquiries to clarify all the details and information about the product
CUSTOMER: Pretty good!
AGENT: Do you have any other complaints, Mr. Phillips?
CUSTOMER: I wanted to complain to the shipping representative who got me this product
AGENT: Moments with me, Mr. Phillips now I will determine who is responsible for your shipping operation
CUSTOMER: OK I am waiting for you
AGENT: I apologize for being late, moments Mr. Phillips
CUSTOMER: Ok
AGENT: are you still with me Mr. Phillips ?
CUSTOMER: Go ahead I am with you
AGENT: Sorry again for being late Mr. Phillips, now we will review with you some data about the shipment, ok?
CUSTOMER: Ok
AGENT: If you can check the delivery information on the bill!
CUSTOMER: One minute
AGENT: Ok
CUSTOMER: ok I am with you now
AGENT: All right, the shipment was 6:00 p.m. on Wednesday, is it right?
CUSTOMER: The product was supposed to be delivered to me at 6 p.m., but I was surprised that the agent was two hours late!!!
AGENT: Ok Mr. Phillips, can you tell me the number of the representative on the bill?
CUSTOMER: Moments… Agent number is (95520)
AGENT: OK Mr. Phillips, can you tell me now about the whole problem, go-ahead
CUSTOMER: The representative was two hours behind schedule, and when I received the shipment was untidy and once humiliated, and unacceptably spoke to me!
AGENT: I apologize to you very much Mr. Phillips, and the necessary work will be done, and strict measures will be taken with the representative immediately, and an apology from our company to you we will offer you a 50% discount on any product you want from our site and shipping the product is free
CUSTOMER: nice thank you Jim
AGENT: I’m always at your service, Mr. Phillips, do you have any complaints or other inquiries?
CUSTOMER: no
AGENT: Ok, Mr. Phillips a customer service representative will contact you from the department of inquiries about the method of using the product as soon as possible, I wish you a happy day
CUSTOMER: thanks a lot
AGENT: I am pleased to serve you, thank you for contacting the customer service of the company Doug's Fishing Imporium Limited