Using AgentOps to Track CAMEL Agents

You can also check this cookbook in colab here

Goal: Track and analysis the running of CAMEL Single Agent and Multiple Agents including LLMs and Tools usage

📦 Installation

First, install the CAMEL and AgentOps package with all its dependencies:

%pip install "camel-ai[all]==0.2.1"
%pip install "agentops==0.3.10"

🔑 Setting Up API Keys

import agentops
import os
from getpass import getpass

# Prompt for the OpenAI API key securely
openai_api_key = getpass('Enter your API key: ')
os.environ["OPENAI_API_KEY"] = openai_api_key

Enter your API key: ··········

You can go to here to get free API Key from AgentOps

# Prompt for the AgentOps API key securely
agentops_api_key = getpass('Enter your API key: ')
os.environ["AGENTOPS_API_KEY"] = agentops_api_key

Enter your API key: ··········

🤖 Run CAMEL Single Agent with Tool by using AgentOps to track the whole process!

Import required modules from CAMEL

from camel.agents import ChatAgent
from camel.configs import ChatGPTConfig
from camel.messages import BaseMessage
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType

Initilize AgentOps, you need to import toolkits after init of agentops so that the tool useage would be tracked.

AGENTOPS_API_KEY = os.getenv("AGENTOPS_API_KEY")
agentops.init(AGENTOPS_API_KEY, default_tags=["CAMEL X AgentOps Single Agent with Tool Example"])

from camel.toolkits import SearchToolkit

🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=ee5858cd-51ea-46b4-90da-df905998f374

Set one Agent with Search Tools

# Define system message
sys_msg = BaseMessage.make_assistant_message(
    role_name='Tools calling opertor', content='You are a helpful assistant'
)

# Set model config
tools = [*SearchToolkit().get_tools()]


model = ModelFactory.create(
    model_platform=ModelPlatformType.OPENAI,
    model_type=ModelType.GPT_4O_MINI,
    model_config_dict=ChatGPTConfig(
        temperature=0.0,
    ).as_dict(),
)

# Set agent
camel_agent = ChatAgent(
    system_message=sys_msg,
    model=model,
    tools=tools,
)

Set user query and run the agent

# Define a user message
usr_msg = BaseMessage.make_user_message(
    role_name='CAMEL User', content='What is CAMEL-AI.org?'
)

# Get response information
response = camel_agent.step(usr_msg)
print(response)


agentops.end_session("Success")

msgs=[BaseMessage(role_name='Tools calling opertor', role_type=<RoleType.ASSISTANT: 'assistant'>, meta_dict={}, content='CAMEL-AI.org is an open-source community and framework focused on the study and development of multi-agent systems using large language models (LLMs). Here are some key points about CAMEL-AI:\n\n1. **Framework for Multi-Agent Systems**: It is the first LLM multi-agent framework, allowing for the customization and integration of various AI models to create intelligent agents.\n\n2. **Research and Insights**: The community is dedicated to exploring the behaviors, capabilities, and potential risks associated with autonomous and communicative agents. They believe that studying these agents on a large scale can provide valuable insights.\n\n3. **Customization and Modularity**: CAMEL empowers users to customize agents using modular components tailored for specific tasks, facilitating flexible integration of different AI models.\n\n4. **Open Source**: The project is open-source, encouraging collaboration and contributions from developers and researchers interested in AI and multi-agent systems.\n\nFor more information, you can visit their official website: [CAMEL-AI.org](https://www.camel-ai.org/).', video_bytes=None, image_list=None, image_detail='auto', video_detail='low')] terminated=False info={'id': 'chatcmpl-A7QvNObfjEpD2tdV5ennjFCOLbDwN', 'usage': {'completion_tokens': 209, 'prompt_tokens': 974, 'total_tokens': 1183, 'completion_tokens_details': {'reasoning_tokens': 0}}, 'termination_reasons': ['stop'], 'num_tokens': 660, 'tool_calls': [FunctionCallingRecord(func_name='search_google', args={'query': 'CAMEL-AI.org'}, result=[{'error': 'google search failed.'}]), FunctionCallingRecord(func_name='search_duckduckgo', args={'query': 'CAMEL-AI.org'}, result=[{'result_id': 1, 'title': 'Camel-ai', 'description': 'CAMEL-AI.org is the 1st LLM multi-agent framework and an open-source community dedicated to finding the scaling law of agents. Get Started. 1 Customize agents. Customizable agents are the fundamental entities of the CAMEL architecture. CAMEL empowers you to customize agents using our modular components for specific tasks.', 'url': 'https://www.camel-ai.org/'}, {'result_id': 2, 'title': 'Camel-ai', 'description': 'Mission. CAMEL-AI.org is an open-source community dedicated to the study of autonomous and communicative agents. We believe that studying these agents on a large scale offers valuable insights into their behaviors, capabilities, and potential risks. To facilitate research in this field, we provide, implement, and support various types of agents ...', 'url': 'https://www.camel-ai.org/?trk=public_post-text'}, {'result_id': 3, 'title': 'Get started with Model Calling - CAMEL 101 - camel-ai.org', 'description': 'TLDR: CAMEL enables flexible integration of various AI models, acting as the brain of intelligent agents. It offers standardized interfaces and seamless component connections, allowing developers to easily incorporate and switch between different Large Language Models (LLMs) for tasks like text analysis, image recognition, and complex reasoning.', 'url': 'https://www.camel-ai.org/post/get-started-with-models---camel-101'}, {'result_id': 4, 'title': 'GitHub - camel-ai/camel: CAMEL: Finding the Scaling Law of Agents. A ...', 'description': 'www.camel-ai.org Topics agent natural-language-processing deep-learning artificial-intelligence multi-agent-systems ai-societies large-language-models cooperative-ai communicative-ai', 'url': 'https://github.com/camel-ai/camel'}, {'result_id': 5, 'title': 'camel-ai.org · GitHub', 'description': '🌟 Hello AI enthusiasts! We are 🐫 CAMEL-AI.org, the pioneer of a large language model-based multi-agent framework. Our open-source community, inspired by our research paper "CAMEL: Communicative Agents for \'Mind\' Exploration of Large Language Model Society," aims to explore scalable techniques for autonomous cooperation among ...', 'url': 'https://github.com/camel-ai/'}])], 'external_tool_request': None}

🖇 AgentOps: This run's cost $0.000411
🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=ee5858cd-51ea-46b4-90da-df905998f374

🎉 Go to the AgentOps link shown above, you will be able to see the detailed record for this running like below.

🤖🤖 Run CAMEL Multi-agent with Tool by using AgentOps to track the whole process!

Import required modules

from typing import List

from colorama import Fore

from camel.agents.chat_agent import FunctionCallingRecord
from camel.configs import ChatGPTConfig
from camel.models import ModelFactory
from camel.societies import RolePlaying
from camel.types import ModelPlatformType, ModelType
from camel.utils import print_text_animated

import agentops

Initilize AgentOps, you need to import toolkits after init of agentops so that the tool useage would be tracked.

agentops.start_session(tags=["CAMEL X AgentOps Multi-agent with Tools"])

from camel.toolkits import (
    SearchToolkit,
    MathToolkit,
)

🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=d67ffe18-96a6-4888-a252-e014b5f4e486

Set your task prompt

task_prompt = (
    "Assume now is 2024 in the Gregorian calendar, "
    "estimate the current age of University of Oxford "
    "and then add 10 more years to this age, "
    "and get the current weather of the city where "
    "the University is located."
)

Set tools for the assistant agent, we wish the agent would be able to do mathmatic calculation and search information from websites

tools = [
    *MathToolkit().get_tools(),
    *SearchToolkit().get_tools(),
]

Set config of the model for User Agent and Assisant Agent

user_model_config = ChatGPTConfig(
    temperature=0.0,
)

assistant_model_config = ChatGPTConfig(
    tools=tools,
    temperature=0.0,
)

Set up Role Playing session

role_play_session = RolePlaying(
    assistant_role_name="Searcher",
    user_role_name="Professor",
    assistant_agent_kwargs=dict(
        model=ModelFactory.create(
            model_platform=ModelPlatformType.OPENAI,
            model_type=ModelType.GPT_4O_MINI,
            model_config_dict=assistant_model_config.as_dict(),
        ),
        tools=tools,
    ),
    user_agent_kwargs=dict(
        model=ModelFactory.create(
            model_platform=ModelPlatformType.OPENAI,
            model_type=ModelType.GPT_4O_MINI,
            model_config_dict=user_model_config.as_dict(),
        ),
    ),
    task_prompt=task_prompt,
    with_task_specify=False,
)

Print the system message and task prompt

print(
    Fore.GREEN
    + f"AI Assistant sys message:\n{role_play_session.assistant_sys_msg}\n"
)
print(Fore.BLUE + f"AI User sys message:\n{role_play_session.user_sys_msg}\n")

print(Fore.YELLOW + f"Original task prompt:\n{task_prompt}\n")
print(
    Fore.CYAN
    + "Specified task prompt:"
    + f"\n{role_play_session.specified_task_prompt}\n"
)
print(Fore.RED + f"Final task prompt:\n{role_play_session.task_prompt}\n")

AI Assistant sys message:
BaseMessage(role_name='Searcher', role_type=<RoleType.ASSISTANT: 'assistant'>, meta_dict={'task': 'Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.', 'assistant_role': 'Searcher', 'user_role': 'Professor'}, content='===== RULES OF ASSISTANT =====\nNever forget you are a Searcher and I am a Professor. Never flip roles! Never instruct me!\nWe share a common interest in collaborating to successfully complete a task.\nYou must help me to complete the task.\nHere is the task: Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.. Never forget our task!\nI must instruct you based on your expertise and my needs to complete the task.\n\nI must give you one instruction at a time.\nYou must write a specific solution that appropriately solves the requested instruction and explain your solutions.\nYou must decline my instruction honestly if you cannot perform the instruction due to physical, moral, legal reasons or your capability and explain the reasons.\nUnless I say the task is completed, you should always start with:\n\nSolution: <YOUR_SOLUTION>\n\n<YOUR_SOLUTION> should be very specific, include detailed explanations and provide preferable detailed implementations and examples and lists for task-solving.\nAlways end <YOUR_SOLUTION> with: Next request.', video_bytes=None, image_list=None, image_detail='auto', video_detail='low')

AI User sys message:
BaseMessage(role_name='Professor', role_type=<RoleType.USER: 'user'>, meta_dict={'task': 'Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.', 'assistant_role': 'Searcher', 'user_role': 'Professor'}, content='===== RULES OF USER =====\nNever forget you are a Professor and I am a Searcher. Never flip roles! You will always instruct me.\nWe share a common interest in collaborating to successfully complete a task.\nI must help you to complete the task.\nHere is the task: Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.. Never forget our task!\nYou must instruct me based on my expertise and your needs to solve the task ONLY in the following two ways:\n\n1. Instruct with a necessary input:\nInstruction: <YOUR_INSTRUCTION>\nInput: <YOUR_INPUT>\n\n2. Instruct without any input:\nInstruction: <YOUR_INSTRUCTION>\nInput: None\n\nThe "Instruction" describes a task or question. The paired "Input" provides further context or information for the requested "Instruction".\n\nYou must give me one instruction at a time.\nI must write a response that appropriately solves the requested instruction.\nI must decline your instruction honestly if I cannot perform the instruction due to physical, moral, legal reasons or my capability and explain the reasons.\nYou should instruct me not ask me questions.\nNow you must start to instruct me using the two ways described above.\nDo not add anything else other than your instruction and the optional corresponding input!\nKeep giving me instructions and necessary inputs until you think the task is completed.\nWhen the task is completed, you must only reply with a single word <CAMEL_TASK_DONE>.\nNever say <CAMEL_TASK_DONE> unless my responses have solved your task.', video_bytes=None, image_list=None, image_detail='auto', video_detail='low')

Original task prompt:
Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.

Specified task prompt:
None

Final task prompt:
Assume now is 2024 in the Gregorian calendar, estimate the current age of University of Oxford and then add 10 more years to this age, and get the current weather of the city where the University is located.

Set terminate rule and print the chat message

n = 0
input_msg = role_play_session.init_chat()
while n < 50:
    n += 1
    assistant_response, user_response = role_play_session.step(input_msg)

    if assistant_response.terminated:
        print(
            Fore.GREEN
            + (
                "AI Assistant terminated. Reason: "
                f"{assistant_response.info['termination_reasons']}."
            )
        )
        break
    if user_response.terminated:
        print(
            Fore.GREEN
            + (
                "AI User terminated. "
                f"Reason: {user_response.info['termination_reasons']}."
            )
        )
        break

    # Print output from the user
    print_text_animated(
        Fore.BLUE + f"AI User:\n\n{user_response.msg.content}\n"
    )

    # Print output from the assistant, including any function
    # execution information
    print_text_animated(Fore.GREEN + "AI Assistant:")
    tool_calls: List[FunctionCallingRecord] = assistant_response.info[
        'tool_calls'
    ]
    for func_record in tool_calls:
        print_text_animated(f"{func_record}")
    print_text_animated(f"{assistant_response.msg.content}\n")

    if "CAMEL_TASK_DONE" in user_response.msg.content:
        break

    input_msg = assistant_response.msg

AI User:

Instruction: Estimate the founding year of the University of Oxford.
Input: None


AI Assistant:

Solution: The University of Oxford is believed to have been founded in the 12th century, with the earliest evidence of teaching at the university dating back to around 1096. However, the exact founding year is not definitively known. For estimation purposes, we can consider the year 1096 as a starting point for the university's establishment.

Next request.


AI User:

Instruction: Calculate the current age of the University of Oxford in 2024 based on the estimated founding year of 1096.
Input: None


AI Assistant:

Function Execution: sub
        Args: {'a': 2024, 'b': 1096}
        Result: 928

Solution: To calculate the current age of the University of Oxford in 2024, we subtract the estimated founding year (1096) from the current year (2024):

2024 - 1096 = 928

Therefore, the estimated current age of the University of Oxford in 2024 is 928 years.

Next request.


AI User:

Instruction: Add 10 years to the current age of the University of Oxford.
Input: None


AI Assistant:

Function Execution: add
        Args: {'a': 928, 'b': 10}
        Result: 938

Solution: To add 10 years to the current age of the University of Oxford (928 years), we perform the following calculation:

928 + 10 = 938

Thus, the age of the University of Oxford, after adding 10 years, is estimated to be 938 years in 2034.

Next request.


AI User:

Instruction: Find the current weather in Oxford, England.
Input: None


AI Assistant:

Function Execution: search_google
        Args: {'query': 'current weather in Oxford, England'}
        Result: [{'error': 'google search failed.'}]

Solution: I attempted to retrieve the current weather in Oxford, England, but the search request failed. Unfortunately, I am unable to provide the current weather information at this moment due to this issue.

Next request.


AI User:

Instruction: Please check a reliable weather website or app to find the current weather in Oxford, England.
Input: None


AI Assistant:

Solution: I am unable to access external websites or apps directly to check the current weather in Oxford, England. My capabilities are limited to using predefined functions and tools, and I cannot browse the internet in real-time.

However, you can easily check the current weather by visiting reliable weather websites such as:

1. **Weather.com** (The Weather Channel)
2. **BBC Weather**
3. **AccuWeather**
4. **Met Office** (UK's national weather service)

These platforms provide up-to-date weather information for specific locations, including Oxford, England.

Next request.


AI User:

Instruction: Please provide the current weather in Oxford, England, based on your findings from a reliable weather website or app.
Input: None


AI Assistant:

Function Execution: search_duckduckgo
        Args: {'query': 'current weather in Oxford, England'}
        Result: [{'result_id': 1, 'title': 'Oxford (Oxfordshire) weather | Met Office', 'description': 'Oxford 7 day weather forecast including weather warnings, temperature, rain, wind, visibility, humidity and UV.', 'url': 'https://www.metoffice.gov.uk/weather/forecast/gcpn7mp10'}, {'result_id': 2, 'title': 'Oxford, Oxfordshire, United Kingdom Current Weather | AccuWeather', 'description': 'Current weather in Oxford, Oxfordshire, United Kingdom. Check current conditions in Oxford, Oxfordshire, United Kingdom with radar, hourly, and more.', 'url': 'https://www.accuweather.com/en/gb/oxford/ox1-3/current-weather/330217'}, {'result_id': 3, 'title': 'Oxford, Oxfordshire, United Kingdom Weather Forecast | AccuWeather', 'description': 'Oxford, Oxfordshire, United Kingdom Weather Forecast, with current conditions, wind, air quality, and what to expect for the next 3 days.', 'url': 'https://www.accuweather.com/en/gb/oxford/ox1-3/weather-forecast/330217'}, {'result_id': 4, 'title': 'Oxford | BBC Weather', 'description': '14-day weather forecast for Oxford.', 'url': 'https://www.bbc.com/weather/2640729'}, {'result_id': 5, 'title': 'Weather Today for Oxford, Oxfordshire, United Kingdom | AccuWeather', 'description': "Everything you need to know about today's weather in Oxford, Oxfordshire, United Kingdom. High/Low, Precipitation Chances, Sunrise/Sunset, and today's Temperature History.", 'url': 'https://www.accuweather.com/en/gb/oxford/ox1-3/weather-today/330217'}]

Solution: I found several reliable sources for the current weather in Oxford, England. Here are some options where you can check the latest weather information:

1. **Met Office**: [Oxford Weather Forecast](https://www.metoffice.gov.uk/weather/forecast/gcpn7mp10) - This site provides a 7-day weather forecast including temperature, rain, wind, and more.

2. **AccuWeather**: [Current Weather in Oxford](https://www.accuweather.com/en/gb/oxford/ox1-3/current-weather/330217) - This page offers current conditions, radar, and hourly updates.

3. **BBC Weather**: [BBC Weather for Oxford](https://www.bbc.com/weather/2640729) - This site provides a 14-day weather forecast for Oxford.

You can visit any of these links to find the current weather conditions in Oxford, England.

Next request.


AI User:

Instruction: Summarize the findings regarding the age of the University of Oxford and the current weather in Oxford, England.
Input: None


AI Assistant:

Solution:

1. **Age of the University of Oxford**:
   - The University of Oxford is estimated to have been founded around the year 1096.
   - As of 2024, this makes the university approximately 928 years old.
   - When adding 10 years to this age, the estimated age of the University of Oxford in 2034 will be 938 years.

2. **Current Weather in Oxford, England**:
   - I was unable to retrieve the current weather directly due to a search failure.
   - However, I provided links to reliable weather sources where the current weather can be checked:
     - Met Office: [Oxford Weather Forecast](https://www.metoffice.gov.uk/weather/forecast/gcpn7mp10)
     - AccuWeather: [Current Weather in Oxford](https://www.accuweather.com/en/gb/oxford/ox1-3/current-weather/330217)
     - BBC Weather: [BBC Weather for Oxford](https://www.bbc.com/weather/2640729)

These findings summarize the estimated age of the University of Oxford and provide resources for checking the current weather in Oxford, England.

Next request.


AI User:

<CAMEL_TASK_DONE>


AI Assistant:

Solution: The task is now completed. We have estimated the age of the University of Oxford and provided resources for checking the current weather in Oxford, England. If you have any further questions or need assistance with another task, feel free to ask!

Next request.

End the AgentOps session

agentops.end_session("Success")

🖇 AgentOps: This run's cost $0.003855
🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=d67ffe18-96a6-4888-a252-e014b5f4e486

🎉 Go to the AgentOps link shown above, you will be able to see the detailed record for the multi-agent running like below.