Customer Service Discord Bot Using Local Models with Agentic RAG#

You can also check this cookbook in colab here

To run this, press β€œRuntime” and press β€œRun all” on a free Tesla T4 Google Colab instance!

This notebook demonstrates how to build a customer service Discord bot powered by Retrieval Augmented Generation (RAG) using local models. It leverages the following technologies:

  • CAMEL: An open-source toolkit for building and deploying large language model (LLM) applications.

  • Firecrawl: A tool for web scraping and creating a local knowledge base.

  • Qdrant: A vector database for efficient knowledge retrieval.

  • Ollama: A local model deployment for running the LLM without external dependencies.

By following this notebook, you can build your own custom customer service bot that uses local models and a custom knowledge base.

aa97151a1f354ed39fa8165685c26551 c70514dcdd6541c19a504298078ec6c3

Join our Discord if you need help + ⭐ Star us on Github ⭐

12345.png

Installation and Setup#

First, install the CAMEL package with all its dependencies

[ ]:
!pip install "camel-ai[all]==0.2.16"
!pip install starlette
!pip install nest_asyncio

Next, prepare the knowledge base with Firecrawl. Firecrawl is a versatile web scraping and crawling tool designed to extract data efficiently from websites, which has been integrated with CAMEL. For more information, you can check out our Firecrawl cookbook: https://colab.research.google.com/drive/1lOmM3VmgR1hLwDKdeLGFve_75RFW0R9I?usp=sharing#scrollTo=1Nj0Oqnoy6oJ

Let’s set up your Firecrawl! You may skip this part if you already have your knowledge file.

In order to run everything locally, we can use self-hosted firecrawl.

For more details, please check out firecrawl documentation: https://docs.firecrawl.dev/contributing/guide

[ ]:
from getpass import getpass

firecrawl_api_url = getpass('Enter your API url: ')
Enter your API url: Β·Β·Β·Β·Β·Β·Β·Β·Β·Β·

Local setup#

Please make a copy of this notebook (important), or run this notebook locally.

If you choose to make a copy of this notebook and stay in Google colab, connect the copied notebook to your local runtime by follow the following steps:

  1. Install notebook locally by running the following command in your terminal:

pip install notebook
jupyter notebook --NotebookApp.allow_origin='https://colab.research.google.com' \
               --port=8888 \
               --no-browser

You will see something like this in your terminal:

To access the server, open this file in a browser:
      <some_path>
  Or copy and paste one of these URLs:
      <url1>
      <url2>
  1. Copy any of the url, and click on β€˜connect to a local runtime’ button in Google Colab, and paste the copied url into Backend Url.

  2. Click on β€˜connect’

Basic Agent and local model Setup#

  1. Download Ollama for a local model at: https://ollama.com/download

  2. After setting up Ollama, pull the Llama3 model by typing the following command into the terminal:

ollama pull qwq

3. cd into a desired directory
```bash
cd <target_drectory_path>

4. Create a `ModelFile` similar the one below in your project directory. (Optional)
```bash
FROM qwq

# Set parameters
PARAMETER temperature 0.8
PARAMETER stop Result

# Sets a custom system message to specify the behavior of the chat assistant
# Leaving it blank for now.

SYSTEM """ """
  1. Create a script to get the base model (llama3) and create a custom model using the ModelFile above. Save this as a .sh file: (Optional)

#!/bin/zsh

# variables
model_name="qwq"
custom_model_name="camel-qwq"

#get the base model
ollama pull $model_name

#create the model file
ollama create $custom_model_name -f ./ModelFile

Now you have the local model deployed!

[2]:
from camel.models import ModelFactory
from camel.types import ModelPlatformType

ollama_model = ModelFactory.create(
    model_platform=ModelPlatformType.OLLAMA,
    model_type="qwq",
    url="http://localhost:11434/v1", #optional
    model_config_dict={"temperature": 0.4},
)

2024-12-29 11:15:47,983 - camel - INFO - Camel library logging has been configured.
[6]:
from camel.agents import ChatAgent
from camel.logger import disable_logging

disable_logging()
chat_agent = ChatAgent(
    system_message="You're a helpful assistant",
    message_window_size=10,
    model=ollama_model,
    token_limit=8192, #change base on your input size
)

Knowledge Crawling and Storage#

Use Firecrawl to crawl a website and store the content in a markdown file:

[ ]:
import os
from camel.loaders import Firecrawl
from camel.messages import BaseMessage

os.makedirs('local_data', exist_ok=True)

firecrawl = Firecrawl(api_url=firecrawl_api_url, api_key="_")

crawl_response = firecrawl.crawl(
    url="https://docs.camel-ai.org/"
)

with open('local_data/camel.md', 'w') as file:
     file.write(crawl_response["data"][0]["markdown"])

Insert the external knowledge to Agent

[8]:
with open('local_data/camel.md', 'r') as file:
  knowledge = file.read()

knowledge_message = BaseMessage.make_user_message(
    role_name="User", content=f"Based on the following knowledge: {knowledge}"
)
chat_agent.update_memory(knowledge_message, "user")

Basic Chatbot Setup#

[ ]:
print("Start chatting! Type 'exit' to end the conversation.")
while True:
    user_input = input("User: ")

    if user_input.lower() == "exit":
        print("Ending conversation.")
        break

    assistant_response = chat_agent.step(user_input)
    print(f"Assistant: {assistant_response.msgs[0].content}")
Start chatting! Type 'exit' to end the conversation.
User: what is camel?
2024-12-28 14:57:51,584 - httpx - INFO - HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
Assistant: CAMEL is a multi-agent framework that allows you to build and use large language model (LLM)-based agents for real-world task solving. It was introduced as one of the earliest LLM-based multi-agent frameworks in research, and it provides a generic platform for creating various types of agents, tasks, prompts, models, and simulated environments.

The primary goal of CAMEL is to facilitate large-scale studies on agent behaviors, capabilities, and potential risks by providing a comprehensive framework for building and interacting with LLM-based agents. It supports different modules such as models, messages, memory, tools, prompts, tasks, loaders, storages, societies, embeddings, retrievers, and workforce, each serving specific purposes in the agent ecosystem.

CAMEL offers a range of cookbooks and tutorials to help users get started with creating their first agents and agent societies, using tools, implementing memory and retrieval mechanisms, generating tasks, and more. Additionally, it provides API references and indices for developers looking to delve deeper into its functionalities.

If you're interested in contributing to CAMEL, whether through research, coding, or simply engaging with the community, there are various ways to get involved, including joining their Discord, WeChat group, or Slack channel.
User: exit
Ending conversation.

Basic Discord Bot Integration#

To build a discord bot, a discord bot token is necessary.

If you don’t have a bot token, you can obtain one by following these steps:

  1. Go to the Discord Developer Portal:https://discord.com/developers/applications

  2. Log in with your Discord account, or create an account if you don’t have one

  3. Click on β€˜New Application’ to create a new bot.

  4. Give your application a name and click β€˜Create’.

  5. Navigate to the β€˜Bot’ tab on the left sidebar and click β€˜Add Bot’.

  6. Once the bot is created, you will find a β€˜Token’ section. Click β€˜Reset Token’ to generate a new token.

  7. Copy the generated token securely.

To invite the bot:

  1. Navigate to the β€˜OAuth2’ tab, then to β€˜URL Generator’.

  2. Under β€˜Scopes’, select β€˜bot’.

  3. Under β€˜Bot Permissions’, select the permissions your bot will need (e.g., β€˜Send Messages’, β€˜Read Messages’ for our bot use)

  4. Copy the generated URL and paste it into your browser to invite the bot to your server.

To grant the bot permissions:

  1. Navigate to the β€˜Bot’ tab

  2. Under β€˜Privileged Gateway Intents’, check β€˜Server Members Intent’ and β€˜Message Content Intent’.

For more details, you can also check the official Discord bot documentation: https://discord.com/developers/docs/intro

[9]:
import os
from getpass import getpass

discord_bot_token = getpass('Enter your Discord bot token: ')
os.environ["DISCORD_BOT_TOKEN"] = discord_bot_token
Enter your Discord bot token: Β·Β·Β·Β·Β·Β·Β·Β·Β·Β·

This code cell sets up a simple Discord bot using the DiscordApp class from the camel.bots library. The bot listens for messages in any channel it has access to and provides a response based on the input message.

[ ]:
from camel.bots import DiscordApp
import nest_asyncio
import discord

nest_asyncio.apply()
discord_bot = DiscordApp(token=discord_bot_token)

@discord_bot.client.event
async def on_message(message: discord.Message):
    if message.author == discord_bot.client.user:
        return

    if message.type != discord.MessageType.default:
        return

    if message.author.bot:
        return
    user_input = message.content
    chat_agent.reset()
    chat_agent.update_memory(knowledge_message, "user")
    assistant_response = chat_agent.step(user_input)

    response_content = assistant_response.msgs[0].content

    if len(response_content) > 2000: # discord message length limit
        for chunk in [response_content[i:i+2000] for i in range(0, len(response_content), 2000)]:
            await message.channel.send(chunk)
    else:
        await message.channel.send(response_content)

discord_bot.run()