Customer Service Discord Bot Using SambaNova with Agentic RAG#
You can also check this cookbook in colab here
⭐ Star the Repo
If you find CAMEL useful or interesting, please consider giving it a star on our CAMEL GitHub Repo! Your stars help others find this project and motivate us to continue improving it.
Installation and Setup#
First, install the CAMEL package with all its dependencies
[ ]:
!pip install "camel-ai[all]==0.2.14"
!pip install starlette
!pip install nest_asyncio
Next, set up your API keys for Firecrawl and SambaNova
If you don’t have a FireCrawl API key, you can obtain one by following these steps:
Visit the FireCrawl API Key page https://www.firecrawl.dev/app/api-keys
Log in or sign up for a FireCrawl account.
Navigate to the ‘API Keys’ section.
Click on ‘Create API Key’ button to generate a new API key.
For more details, you can also check the Firecrawl documentation: https://docs.firecrawl.dev/api-reference/introduction
[2]:
import os
from getpass import getpass
firecrawl_api_key = getpass('Enter your API key: ')
os.environ["FIRECRAWL_API_KEY"] = firecrawl_api_key
Enter your API key: ··········
If you don’t have a SambaNova Cloud API key, you can obtain one by following these steps:
Visit the SambaNova Cloud page https://cloud.sambanova.ai/apis
Log in or sign up for a SambaNova account.
Navigate to the ‘API Keys’ section.
Click on ‘Create API Key’ button to generate a new API key.
For more details, you can also check the SambaNova documentation: https://community.sambanova.ai/c/docs/
[3]:
import os
from getpass import getpass
samba_api_key = getpass('Enter your API key: ')
os.environ["SAMBA_API_KEY"] = samba_api_key
Enter your API key: ··········
Knowledge Crawling and Storage#
Use Firecrawl to crawl a website and get markdown content as external knowledge:
[5]:
import os
from camel.loaders import Firecrawl
firecrawl = Firecrawl()
knowledge = firecrawl.crawl(
url="https://sambanova.ai/blog/qwen-2.5-32b-coder-available-on-sambanova-cloud"
)["data"][0]["markdown"]
Store the content in a markdown file:
[6]:
os.makedirs('local_data', exist_ok=True)
with open('local_data/sambanova_announcement.md', 'w') as file:
file.write(knowledge)
Basic Agent Setup#
Qwen is large language model developed by Alibaba. It is trained on a massive dataset of text and code and can generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.
Use Qwen models with SambaNova Cloud to set up CAMEL agent:
[16]:
from camel.configs import SambaCloudAPIConfig
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType
from camel.agents import ChatAgent
from camel.messages import BaseMessage
#### Set up Agent using Qwen2.5-Coder-32B-Instruct #####
qwen_model = ModelFactory.create(
model_platform=ModelPlatformType.SAMBA,
model_type="Qwen2.5-Coder-32B-Instruct",
model_config_dict=SambaCloudAPIConfig(max_tokens=4000).as_dict(),
)
# ##### Set up Agent using Qwen2.5-72B-Instruct #####
# qwen_model = ModelFactory.create(
# model_platform=ModelPlatformType.SAMBA,
# model_type="Qwen2.5-72B-Instruct",
# model_config_dict=SambaCloudAPIConfig(max_tokens=4000).as_dict(),
# )
chat_agent = ChatAgent(
system_message="You're a helpful assistant",
message_window_size=20,
model=qwen_model
)
Insert the external knowledge to Agent
[17]:
knowledge_message = BaseMessage.make_user_message(
role_name="User", content=f"Based on the following knowledge: {knowledge}"
)
chat_agent.update_memory(knowledge_message, "user")
Basic Chatbot Setup#
Let’s set up the basic Chatbot with CAMEL Agent and ask some questions!
Example question you can ask:
How SambaNova Cloud supports Qwen 2.5 Coder and how fast it is?
[18]:
print("Start chatting! Type 'exit' to end the conversation.")
while True:
user_input = input("User: ")
if user_input.lower() == "exit":
print("Ending conversation.")
break
assistant_response = chat_agent.step(user_input)
print(f"Assistant: {assistant_response.msgs[0].content}")
Start chatting! Type 'exit' to end the conversation.
User: How SambaNova Cloud supports Qwen 2.5 Coder and how fast it is?
Assistant: SambaNova Cloud supports Qwen 2.5 Coder by providing a platform that leverages the efficiency and performance of SambaNova's RDU (Reconfigurable Dataflow Unit) chips. These chips are designed to handle the most demanding AI workloads, including generative AI models like Qwen 2.5 Coder, with superior performance compared to traditional GPUs.
### Key Points:
1. **Performance**:
- **5X Faster than GPUs**: The Qwen 2.5 Coder model runs over 5X faster on SambaNova Cloud compared to leading GPU providers. This speedup is due to the optimized architecture of the RDU chips, which are specifically designed for AI workloads.
- **Efficient Inference**: The RDU chips provide fast inference speeds, which are crucial for real-time applications and seamless user experiences.
2. **Integration**:
- **Ease of Use**: SambaNova Cloud makes it easy for developers to integrate the Qwen 2.5 Coder model into their applications. You can get started by obtaining an API key and integrating it into your development environment.
- **Partnerships**: SambaNova Cloud has partnered with tools like Continue.dev to simplify the integration process and provide a better developer experience. This partnership allows developers to easily integrate and utilize the Qwen 2.5 Coder model in their IDEs.
3. **Use Cases**:
- **Coding Agents**: The Qwen 2.5 Coder model is particularly useful for developing coding agents that can write, edit, and optimize code autonomously. The high performance of SambaNova Cloud ensures that these agents can operate efficiently and provide quick responses.
- **Real-Time Applications**: The fast inference speeds provided by SambaNova Cloud make it ideal for real-time applications where quick response times are critical.
### Getting Started:
1. **Visit SambaNova Cloud**: Go to [SambaNova Cloud](https://cloud.sambanova.ai/) to sign up and obtain your API key.
2. **Integrate the Model**: Use the API key to integrate the Qwen 2.5 Coder model into your development environment.
3. **Explore Further**: Check out resources like [Continue’s blog post](https://blog.continue.dev/p/d0796190-6827-42d1-8d81-bd9ca31e1828/) for more information on how to integrate and use the model effectively.
By leveraging SambaNova Cloud, developers can harness the power of the Qwen 2.5 Coder model with the speed and efficiency needed for modern AI applications.
User: exit
Ending conversation.
Basic Discord Bot Integration#
To build a discord bot, a discord bot token is necessary.
If you don’t have a bot token, you can obtain one by following these steps:
Go to the Discord Developer Portal:https://discord.com/developers/applications
Log in with your Discord account, or create an account if you don’t have one
Click on ‘New Application’ to create a new bot.
Give your application a name and click ‘Create’.
Navigate to the ‘Bot’ tab on the left sidebar and click ‘Add Bot’.
Once the bot is created, you will find a ‘Token’ section. Click ‘Reset Token’ to generate a new token.
Copy the generated token securely.
To invite the bot:
Navigate to the ‘OAuth2’ tab, then to ‘URL Generator’.
Under ‘Scopes’, select ‘bot’.
Under ‘Bot Permissions’, select the permissions your bot will need (e.g., ‘Send Messages’, ‘Read Messages’ for our bot use)
Copy the generated URL and paste it into your browser to invite the bot to your server.
To grant the bot permissions:
Navigate to the ‘Bot’ tab
Under ‘Privileged Gateway Intents’, check ‘Server Members Intent’ and ‘Message Content Intent’.
For more details, you can also check the official Discord bot documentation: https://discord.com/developers/docs/intro
[24]:
import os
from getpass import getpass
discord_bot_token = getpass('Enter your Discord bot token: ')
os.environ["DISCORD_BOT_TOKEN"] = discord_bot_token
Enter your Discord bot token: ··········
This code cell sets up a simple Discord bot using the DiscordApp class from the camel.bots library. The bot listens for messages in any channel it has access to and provides a response based on the input message.
[21]:
from camel.bots import DiscordApp
import nest_asyncio
import discord
nest_asyncio.apply()
discord_bot = DiscordApp(token=discord_bot_token)
@discord_bot.client.event
async def on_message(message: discord.Message):
if message.author == discord_bot.client.user:
return
if message.type != discord.MessageType.default:
return
if message.author.bot:
return
user_input = message.content
chat_agent.reset()
chat_agent.update_memory(knowledge_message, "user")
assistant_response = chat_agent.step(user_input)
response_content = assistant_response.msgs[0].content
if len(response_content) > 2000: # discord message length limit
for chunk in [response_content[i:i+2000] for i in range(0, len(response_content), 2000)]:
await message.channel.send(chunk)
else:
await message.channel.send(response_content)
discord_bot.run()
2024-12-10 16:16:51 INFO discord.client logging in using static token
2024-12-10 16:16:51 INFO discord.client logging in using static token
INFO:discord.client:logging in using static token
2024-12-10 16:16:52 INFO discord.gateway Shard ID None has connected to Gateway (Session ID: 5b6a3257cd69b0e64ea74f15e0abc8ca).
2024-12-10 16:16:52 INFO discord.gateway Shard ID None has connected to Gateway (Session ID: 5b6a3257cd69b0e64ea74f15e0abc8ca).
INFO:discord.gateway:Shard ID None has connected to Gateway (Session ID: 5b6a3257cd69b0e64ea74f15e0abc8ca).
Integrating Qdrant for More Files to build a more powerful Discord bot#
Qdrant is a vector similarity search engine and vector database. It is designed to perform fast and efficient similarity searches on large datasets of vectors. This enables the chatbot to access and utilize external information to provide more comprehensive and accurate responses. By storing knowledge as vectors, Qdrant enables efficient semantic search, allowing the chatbot to find relevant information based on the meaning of the user’s query.
In this section, we will add more data source, including camel’s example code regarding how to use SambaNova Cloud, then ask more complex questions.
Set up an embedding model and retriever for Qdrant: You can use Tesla T4 Google Colab instance for running open-source embedding models with RAG functionality for bots, feel free switch to other embedding models supported by CAMEL.
[ ]:
from camel.embeddings import SentenceTransformerEncoder # CAMEL also support other embedding models
from camel.types import EmbeddingModelType
sentence_encoder = SentenceTransformerEncoder(model_name='intfloat/e5-large-v2')
Set up the AutoRetriever for retrieving relevant information from a storage system.
[30]:
from camel.retrievers import AutoRetriever
from camel.types import StorageType
assistant_sys_msg = """You are a helpful assistant to answer question,
I will give you the Original Query and Retrieved Context,
answer the Original Query based on the Retrieved Context,
if you can't answer the question just say I don't know.
Just give the answer to me directly, no more other words needed.
"""
auto_retriever = AutoRetriever(
vector_storage_local_path="local_data2/",
storage_type=StorageType.QDRANT,
embedding_model=sentence_encoder
)
chat_agent_with_rag = ChatAgent(system_message=assistant_sys_msg, model=qwen_model)
Use Auto RAG to retrieve first and then answer the user’s query using CAMEL ChatAgent based on the retrieved info:
[31]:
from camel.bots import DiscordApp
import nest_asyncio
import discord
nest_asyncio.apply()
discord_q_bot = DiscordApp(token=discord_bot_token)
@discord_q_bot.client.event # triggers when a message is sent in the channel
async def on_message(message: discord.Message):
if message.author == discord_q_bot.client.user:
return
if message.type != discord.MessageType.default:
return
if message.author.bot:
return
user_input = message.content
query_and_retrieved_info = auto_retriever.run_vector_retriever(
query=user_input,
contents=[
"local_data/sambanova_announcement.md", # SambaNova's anncouncement
"https://github.com/camel-ai/camel/blob/master/examples/models/samba_model_example.py", # CAMEL's example code for SambaNova Usage
],
top_k=3,
return_detailed_info=False,
similarity_threshold=0.5
)
user_msg = str(query_and_retrieved_info)
assistant_response = chat_agent_with_rag.step(user_msg)
response_content = assistant_response.msgs[0].content
if len(response_content) > 2000: # discord message length limit
for chunk in [response_content[i:i+2000] for i in range(0, len(response_content), 2000)]:
await message.channel.send(chunk)
else:
await message.channel.send(response_content)
discord_q_bot.run()
2024-12-10 16:35:53 INFO discord.client logging in using static token
2024-12-10 16:35:53 INFO discord.client logging in using static token
2024-12-10 16:35:53 INFO discord.client logging in using static token
2024-12-10 16:35:53 INFO discord.client logging in using static token
2024-12-10 16:35:53 INFO discord.client logging in using static token
2024-12-10 16:35:53 INFO discord.client logging in using static token
INFO:discord.client:logging in using static token
2024-12-10 16:35:54 INFO discord.gateway Shard ID None has connected to Gateway (Session ID: c87148a016d58631bf1aeec331917521).
2024-12-10 16:35:54 INFO discord.gateway Shard ID None has connected to Gateway (Session ID: c87148a016d58631bf1aeec331917521).
2024-12-10 16:35:54 INFO discord.gateway Shard ID None has connected to Gateway (Session ID: c87148a016d58631bf1aeec331917521).
2024-12-10 16:35:54 INFO discord.gateway Shard ID None has connected to Gateway (Session ID: c87148a016d58631bf1aeec331917521).
2024-12-10 16:35:54 INFO discord.gateway Shard ID None has connected to Gateway (Session ID: c87148a016d58631bf1aeec331917521).
2024-12-10 16:35:54 INFO discord.gateway Shard ID None has connected to Gateway (Session ID: c87148a016d58631bf1aeec331917521).
INFO:discord.gateway:Shard ID None has connected to Gateway (Session ID: c87148a016d58631bf1aeec331917521).
Start from the same query as before:
Since we also added CAMEL’s example code to the RAG Bot, let’s ask some code related question:
Ask the bot to guide you through setting up Qwen2.5-Coder-32B-Instruct. CAMEL’s bot, equipped with memory capabilities, can assist effectively by leveraging its ability to recall related information from previous interactions!
That’s everything: Got questions about 🐫 CAMEL-AI? Join us on Discord! Whether you want to share feedback, explore the latest in multi-agent systems, get support, or connect with others on exciting projects, we’d love to have you in the community! 🤝
Check out some of our other work:
🐫 Creating Your First CAMEL Agent free Colab
Graph RAG Cookbook free Colab
🧑⚖️ Create A Hackathon Judge Committee with Workforce free Colab
🔥 3 ways to ingest data from websites with Firecrawl & CAMEL free Colab
🦥 Agentic SFT Data Generation with CAMEL and Mistral Models, Fine-Tuned with Unsloth free Colab
Thanks from everyone at 🐫 CAMEL-AI
