Track CAMEL Agents with AgentOps#
You can also check this cookbook in colab here
Goal: Track and analysis the running of CAMEL Single Agent and Multiple Agents including LLMs and Tools usage
📦 Installation#
First, install the CAMEL and AgentOps package with all its dependencies:
[ ]:
%pip install camel-ai[all]==0.2.9
%pip install agentops==0.3.10
🔑 Setting Up API Keys#
[2]:
import agentops
import os
from getpass import getpass
[3]:
# Prompt for the OpenAI API key securely
openai_api_key = getpass('Enter your API key: ')
os.environ["OPENAI_API_KEY"] = openai_api_key
Enter your API key: ··········
You can go to here to get free API Key from AgentOps
[4]:
# Prompt for the AgentOps API key securely
agentops_api_key = getpass('Enter your API key: ')
os.environ["AGENTOPS_API_KEY"] = agentops_api_key
Enter your API key: ··········
🤖 Run CAMEL Single Agent with Tool by using AgentOps to track the whole process!#
Import required modules from CAMEL
[5]:
from camel.agents import ChatAgent
from camel.configs import ChatGPTConfig
from camel.messages import BaseMessage
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType
Initilize AgentOps, you need to import toolkits after init of agentops so that the tool useage would be tracked.
[6]:
AGENTOPS_API_KEY = os.getenv("AGENTOPS_API_KEY")
agentops.init(AGENTOPS_API_KEY, default_tags=["CAMEL X AgentOps Single Agent with Tool Example"])
from camel.toolkits import SearchToolkit
🖇 AgentOps: WARNING: agentops is out of date. Please update with the command: 'pip install --upgrade agentops'
🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=db687abe-4e5c-43d5-8eb2-21bf403256e7
Set one Agent with Search Tools
[7]:
# Define system message
sys_msg = BaseMessage.make_assistant_message(
role_name='Tools calling opertor', content='You are a helpful assistant'
)
# Set model config
tools = [*SearchToolkit().get_tools()]
model = ModelFactory.create(
model_platform=ModelPlatformType.OPENAI,
model_type=ModelType.GPT_4O_MINI,
)
# Set agent
camel_agent = ChatAgent(
system_message=sys_msg,
model=model,
tools=tools,
)
Set user query and run the agent
[8]:
# Define a user message
usr_msg = 'What is CAMEL-AI.org?'
# Get response information
response = camel_agent.step(usr_msg)
print(response)
agentops.end_session("Success")
msgs=[BaseMessage(role_name='Tools calling opertor', role_type=<RoleType.ASSISTANT: 'assistant'>, meta_dict={}, content='CAMEL-AI.org is the official website for CAMEL, which is described as the first large language model (LLM) multi-agent framework. It is an open-source community dedicated to exploring the scaling laws of agents. The framework allows users to customize agents using modular components for specific tasks, facilitating research and development in the field of artificial intelligence.\n\nHere are some key points about CAMEL-AI:\n\n1. **Multi-Agent Framework**: CAMEL provides a structure for developing and utilizing multiple agents that can work together.\n2. **Open Source**: The project is open-source, encouraging collaboration and contributions from the community.\n3. **Customization**: Users can customize agents to perform specific tasks, enhancing their functionality and adaptability.\n4. **Research Focus**: The initiative aims to study agents on a large scale to gain insights into their behaviors, capabilities, and potential risks.\n\nFor more information, you can visit their official website: [CAMEL-AI.org](https://www.camel-ai.org/).', video_bytes=None, image_list=None, image_detail='auto', video_detail='low')] terminated=False info={'id': 'chatcmpl-ALp4yBEZ70iWRyC2nU7Ns5Y4cqS5N', 'usage': {'completion_tokens': 204, 'prompt_tokens': 993, 'total_tokens': 1197, 'completion_tokens_details': {'audio_tokens': None, 'reasoning_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': None, 'cached_tokens': 0}}, 'termination_reasons': ['stop'], 'num_tokens': 679, 'tool_calls': [FunctionCallingRecord(func_name='search_google', args={'query': 'CAMEL-AI.org'}, result=[{'error': 'google search failed.'}]), FunctionCallingRecord(func_name='search_duckduckgo', args={'query': 'CAMEL-AI.org'}, result=[{'result_id': 1, 'title': 'Camel-ai', 'description': 'CAMEL-AI.org is the 1st LLM multi-agent framework and an open-source community dedicated to finding the scaling law of agents. Get Started. 1 Customize agents. Customizable agents are the fundamental entities of the CAMEL architecture. CAMEL empowers you to customize agents using our modular components for specific tasks.', 'url': 'https://www.camel-ai.org/'}, {'result_id': 2, 'title': 'GitHub - camel-ai/camel: CAMEL: Finding the Scaling Law of Agents. A ...', 'description': '🐫 CAMEL: Finding the Scaling Law of Agents. A multi-agent framework. https://www.camel-ai.org - camel-ai/camel', 'url': 'https://github.com/camel-ai/camel'}, {'result_id': 3, 'title': 'CAMEL-AI - CRAB: Cross-environment Agent Benchmark for Multimodal ...', 'description': 'Please open the X app on my phone, search for CAMEL-AI.org, check the latest post, summarize them, and then send the summary to Tianqi Xu on Slack from my PC. Settings: OpenAI GPT-4o + Single Agent Demo videos are edited for a better viewing experience. In actual execution, there are tens of seconds of waiting time between each step.', 'url': 'https://crab.camel-ai.org/'}, {'result_id': 4, 'title': "Welcome to CAMEL's documentation! — CAMEL 0.2.2 documentation", 'description': "Welcome to CAMEL's documentation! #. We design practical artificial general intelligence. CAMEL emerges as the earliest LLM-based multi-agent framework, and is now a generic framework to build and use LLM-based agents for real-world task solving. We believe that studying these agents on a large scale offers valuable insights into their ...", 'url': 'https://docs.camel-ai.org/'}, {'result_id': 5, 'title': 'Team - CAMEL-AI', 'description': 'Camel-AI Team. We are finding the scaling law of agent. 🐫 CAMEL is an open-source library designed for the study of autonomous and communicative agents. We believe that studying these agents on a large scale offers valuable insights into their behaviors, capabilities, and potential risks. To facilitate research in this field, we implement ...', 'url': 'https://www.camel-ai.org/about'}])], 'external_tool_request': None}
🖇 AgentOps: This run's cost $0.000411
🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=db687abe-4e5c-43d5-8eb2-21bf403256e7
🎉 Go to the AgentOps link shown above, you will be able to see the detailed record for this running like below.#
🤖🤖 Run CAMEL Multi-agent with Tool by using AgentOps to track the whole process!#
Import required modules
[9]:
from typing import List
from colorama import Fore
from camel.agents.chat_agent import FunctionCallingRecord
from camel.configs import ChatGPTConfig
from camel.models import ModelFactory
from camel.societies import RolePlaying
from camel.types import ModelPlatformType, ModelType
from camel.utils import print_text_animated
import agentops
Initilize AgentOps, you need to import toolkits after init of agentops so that the tool useage would be tracked.
[10]:
agentops.start_session(tags=["CAMEL X AgentOps Multi-agent with Tools"])
from camel.toolkits import (
SearchToolkit,
MathToolkit,
)
🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=6c46dfa1-662b-40e1-8343-b40934c5659c
Set your task prompt
[11]:
task_prompt = (
"Assume now is 2024 in the Gregorian calendar, "
"estimate the current age of University of Oxford "
"and then add 10 more years to this age, "
"and get the current weather of the city where "
"the University is located."
)
Set tools for the assistant agent, we wish the agent would be able to do mathmatic calculation and search information from websites
[12]:
tools = [
*MathToolkit().get_tools(),
*SearchToolkit().get_tools(),
]
Set up Role Playing session
[13]:
role_play_session = RolePlaying(
assistant_role_name="Searcher",
user_role_name="Professor",
assistant_agent_kwargs=dict(
model=ModelFactory.create(
model_platform=ModelPlatformType.OPENAI,
model_type=ModelType.GPT_4O_MINI,
),
tools=tools,
),
user_agent_kwargs=dict(
model=ModelFactory.create(
model_platform=ModelPlatformType.OPENAI,
model_type=ModelType.GPT_4O_MINI,
),
),
task_prompt=task_prompt,
with_task_specify=False,
)
Print the system message and task prompt
[14]:
print(
Fore.GREEN
+ f"AI Assistant sys message:\n{role_play_session.assistant_sys_msg}\n"
)
print(Fore.BLUE + f"AI User sys message:\n{role_play_session.user_sys_msg}\n")
print(Fore.YELLOW + f"Original task prompt:\n{task_prompt}\n")
print(
Fore.CYAN
+ "Specified task prompt:"
+ f"\n{role_play_session.specified_task_prompt}\n"
)
print(Fore.RED + f"Final task prompt:\n{role_play_session.task_prompt}\n")
AI Assistant sys message:
BaseMessage(role_name='Searcher', role_type=<RoleType.ASSISTANT: 'assistant'>, meta_dict={'task': 'Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.', 'assistant_role': 'Searcher', 'user_role': 'Professor'}, content='===== RULES OF ASSISTANT =====\nNever forget you are a Searcher and I am a Professor. Never flip roles! Never instruct me!\nWe share a common interest in collaborating to successfully complete a task.\nYou must help me to complete the task.\nHere is the task: Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.. Never forget our task!\nI must instruct you based on your expertise and my needs to complete the task.\n\nI must give you one instruction at a time.\nYou must write a specific solution that appropriately solves the requested instruction and explain your solutions.\nYou must decline my instruction honestly if you cannot perform the instruction due to physical, moral, legal reasons or your capability and explain the reasons.\nUnless I say the task is completed, you should always start with:\n\nSolution: <YOUR_SOLUTION>\n\n<YOUR_SOLUTION> should be very specific, include detailed explanations and provide preferable detailed implementations and examples and lists for task-solving.\nAlways end <YOUR_SOLUTION> with: Next request.', video_bytes=None, image_list=None, image_detail='auto', video_detail='low')
AI User sys message:
BaseMessage(role_name='Professor', role_type=<RoleType.USER: 'user'>, meta_dict={'task': 'Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.', 'assistant_role': 'Searcher', 'user_role': 'Professor'}, content='===== RULES OF USER =====\nNever forget you are a Professor and I am a Searcher. Never flip roles! You will always instruct me.\nWe share a common interest in collaborating to successfully complete a task.\nI must help you to complete the task.\nHere is the task: Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.. Never forget our task!\nYou must instruct me based on my expertise and your needs to solve the task ONLY in the following two ways:\n\n1. Instruct with a necessary input:\nInstruction: <YOUR_INSTRUCTION>\nInput: <YOUR_INPUT>\n\n2. Instruct without any input:\nInstruction: <YOUR_INSTRUCTION>\nInput: None\n\nThe "Instruction" describes a task or question. The paired "Input" provides further context or information for the requested "Instruction".\n\nYou must give me one instruction at a time.\nI must write a response that appropriately solves the requested instruction.\nI must decline your instruction honestly if I cannot perform the instruction due to physical, moral, legal reasons or my capability and explain the reasons.\nYou should instruct me not ask me questions.\nNow you must start to instruct me using the two ways described above.\nDo not add anything else other than your instruction and the optional corresponding input!\nKeep giving me instructions and necessary inputs until you think the task is completed.\nWhen the task is completed, you must only reply with a single word <CAMEL_TASK_DONE>.\nNever say <CAMEL_TASK_DONE> unless my responses have solved your task.', video_bytes=None, image_list=None, image_detail='auto', video_detail='low')
Original task prompt:
Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.
Specified task prompt:
None
Final task prompt:
Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.
Set terminate rule and print the chat message
[15]:
n = 0
input_msg = role_play_session.init_chat()
while n < 50:
n += 1
assistant_response, user_response = role_play_session.step(input_msg)
if assistant_response.terminated:
print(
Fore.GREEN
+ (
"AI Assistant terminated. Reason: "
f"{assistant_response.info['termination_reasons']}."
)
)
break
if user_response.terminated:
print(
Fore.GREEN
+ (
"AI User terminated. "
f"Reason: {user_response.info['termination_reasons']}."
)
)
break
# Print output from the user
print_text_animated(
Fore.BLUE + f"AI User:\n\n{user_response.msg.content}\n"
)
# Print output from the assistant, including any function
# execution information
print_text_animated(Fore.GREEN + "AI Assistant:")
tool_calls: List[FunctionCallingRecord] = assistant_response.info[
'tool_calls'
]
for func_record in tool_calls:
print_text_animated(f"{func_record}")
print_text_animated(f"{assistant_response.msg.content}\n")
if "CAMEL_TASK_DONE" in user_response.msg.content:
break
input_msg = assistant_response.msg
AI User:
Instruction: Estimate the founding year of the University of Oxford.
Input: None
AI Assistant:
Solution: The University of Oxford is believed to have been founded in the 12th century, with the earliest evidence of teaching at Oxford dating back to around 1096. However, it is often cited that the university was officially established in 1209 when a group of scholars fled from the University of Cambridge due to disputes with the townspeople.
For our estimation, we can consider the founding year to be approximately 1209.
Next request.
AI User:
Instruction: Calculate the current age of the University of Oxford as of 2024 based on the estimated founding year of 1209.
Input: None
AI Assistant:
Solution: To calculate the current age of the University of Oxford as of 2024, we subtract the estimated founding year (1209) from the current year (2024).
Calculation:
2024 - 1209 = 815
Therefore, the current age of the University of Oxford in 2024 is approximately 815 years.
Next request.
AI User:
Instruction: Add 10 years to the current age of the University of Oxford.
Input: None
AI Assistant:
Solution: To add 10 years to the current age of the University of Oxford, which we calculated to be 815 years in 2024, we simply perform the addition:
Calculation:
815 + 10 = 825
Thus, the age of the University of Oxford, after adding 10 years, will be approximately 825 years.
Next request.
AI User:
Instruction: Find the current weather in Oxford, England.
Input: None
AI Assistant:
Function Execution: search_google
Args: {'query': 'current weather in Oxford, England'}
Result: [{'error': 'google search failed.'}]
Solution: I attempted to retrieve the current weather in Oxford, England, but the search for this information was unsuccessful due to a failure in the Google search.
As an alternative, I can try to search for the current weather using a different method or source. Would you like me to proceed with that?
Next request.
AI User:
Instruction: Please use an alternative method or source to find the current weather in Oxford, England.
Input: None
AI Assistant:
Function Execution: search_duckduckgo
Args: {'query': 'current weather in Oxford, England'}
Result: [{'result_id': 1, 'title': 'Oxford (Oxfordshire) weather - Met Office', 'description': 'Oxford 7 day weather forecast including weather warnings, temperature, rain, wind, visibility, humidity and UV ... Use my current location . Recent places. ... A change in the broad weather pattern over the UK is expected towards mid-November and beyond, as high pressure initially over the nearby continent tends to become more focussed to the ...', 'url': 'https://www.metoffice.gov.uk/weather/forecast/gcpn7mp10'}, {'result_id': 2, 'title': 'Oxford, Oxfordshire, United Kingdom Weather Forecast | AccuWeather', 'description': 'Oxford, Oxfordshire, United Kingdom Weather Forecast, with current conditions, wind, air quality, and what to expect for the next 3 days.', 'url': 'https://www.accuweather.com/en/gb/oxford/ox1-3/weather-forecast/330217'}, {'result_id': 3, 'title': 'Oxford - BBC Weather', 'description': '14-day weather forecast for Oxford.', 'url': 'https://www.bbc.com/weather/2640729'}, {'result_id': 4, 'title': 'Oxford, Oxfordshire, United Kingdom Current Weather | AccuWeather', 'description': 'Current weather in Oxford, Oxfordshire, United Kingdom. Check current conditions in Oxford, Oxfordshire, United Kingdom with radar, hourly, and more.', 'url': 'https://www.accuweather.com/en/gb/oxford/ox1-3/current-weather/330217'}, {'result_id': 5, 'title': '10-day weather forecast for Oxford, England - The Weather Channel', 'description': 'Be prepared with the most accurate 10-day forecast for Oxford, England with highs, lows, chance of precipitation from The Weather Channel and Weather.com', 'url': 'https://weather.com/en-GB/weather/tenday/l/Oxford+England?canonicalCityId=0c16ff8887c1d01b06a4230a12f6f1f3e20275c65db637043a26a8cb26337644'}]
Solution: I found several sources that provide the current weather in Oxford, England. Here are some options:
1. **Met Office**: [Oxford Weather Forecast](https://www.metoffice.gov.uk/weather/forecast/gcpn7mp10) - This site provides a detailed 7-day weather forecast including temperature, rain, wind, and more.
2. **AccuWeather**: [Current Weather in Oxford](https://www.accuweather.com/en/gb/oxford/ox1-3/current-weather/330217) - This page offers current conditions, wind, air quality, and a short-term forecast.
3. **BBC Weather**: [Oxford Weather](https://www.bbc.com/weather/2640729) - This site provides a 14-day weather forecast for Oxford.
4. **The Weather Channel**: [10-day Weather Forecast for Oxford](https://weather.com/en-GB/weather/tenday/l/Oxford+England?canonicalCityId=0c16ff8887c1d01b06a4230a12f6f1f3e20275c65db637043a26a8cb26337644) - This page includes a comprehensive 10-day forecast.
You can visit any of these links to get the most current weather information for Oxford, England.
Next request.
AI User:
Instruction: Confirm the current weather in Oxford, England, using one of the provided sources.
Input: None
AI Assistant:
Function Execution: search_wiki
Args: {'entity': 'Oxford, England'}
Result: Oxford ( ) is a cathedral city and non-metropolitan district in Oxfordshire, England, of which it is the county town. Founded in the 8th century, it was granted city status in 1542. The city is located at the confluence of the rivers Thames (locally known as the Isis) and Cherwell. It had a population of 163,257 in 2022. It is 56 miles (90 km) north-west of London, 64 miles (103 km) south-east of Birmingham and 61 miles (98 km) north-east of Bristol.
Solution: I attempted to confirm the current weather in Oxford, England, using the Wikipedia source, but it does not provide real-time weather information. Instead, I recommend checking one of the previously mentioned sources for accurate and up-to-date weather conditions.
For example, you can visit the [AccuWeather page for Oxford](https://www.accuweather.com/en/gb/oxford/ox1-3/current-weather/330217) to find the current weather details.
Next request.
AI User:
Instruction: Summarize the findings regarding the age of the University of Oxford and the current weather in Oxford, England.
Input: None
AI Assistant:
Solution:
1. **Age of the University of Oxford**:
- The University of Oxford is estimated to have been founded around the year 1209.
- As of 2024, this makes the university approximately 815 years old.
- After adding 10 years, the age of the University of Oxford will be approximately 825 years.
2. **Current Weather in Oxford, England**:
- I attempted to retrieve the current weather information but faced challenges with the search.
- However, I provided several reliable sources where the current weather can be checked, including:
- [Met Office](https://www.metoffice.gov.uk/weather/forecast/gcpn7mp10)
- [AccuWeather](https://www.accuweather.com/en/gb/oxford/ox1-3/current-weather/330217)
- [BBC Weather](https://www.bbc.com/weather/2640729)
- [The Weather Channel](https://weather.com/en-GB/weather/tenday/l/Oxford+England?canonicalCityId=0c16ff8887c1d01b06a4230a12f6f1f3e20275c65db637043a26a8cb26337644)
These sources can provide the most accurate and current weather conditions for Oxford, England.
Next request.
AI User:
<CAMEL_TASK_DONE>
AI Assistant:
Solution: The task is now marked as completed. If you have any further inquiries or need assistance with another task in the future, feel free to reach out.
Next request.
End the AgentOps session
[16]:
agentops.end_session("Success")
🖇 AgentOps: This run's cost $0.004198
🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=6c46dfa1-662b-40e1-8343-b40934c5659c
🎉 Go to the AgentOps link shown above, you will be able to see the detailed record for the multi-agent running like below.#
