Video Analysis#

You can also check this cookbook in colab here

f84569878cfd4ea1921fa3c3376389ea

⭐ Star us on Github, join our Discord or follow our X

This notebook demonstrates how to set up and leverage CAMEL’s ability to do video analysis.

In this notebook, you’ll explore:

CAMEL: A powerful multi-agent framework that enables Retrieval-Augmented Generation and multi-agent role-playing scenarios, allowing for sophisticated AI-driven tasks.
Video Analysis: How to use CAMEL to read and generate descriptions of uploaded videos.

📦 Installation#

[ ]:

%pip install "camel-ai[all]==0.2.16"

🔑 Setting Up API Keys#

[2]:

import os
from getpass import getpass

# Prompt for the API key securely
openai_api_key = getpass('Enter your API key: ')
os.environ["OPENAI_API_KEY"] = openai_api_key

Enter your API key: ··········

Set up an agent for video analysis task#

[3]:

from camel.agents import ChatAgent
from camel.configs.openai_config import ChatGPTConfig
from camel.messages import BaseMessage
from camel.prompts.prompt_templates import PromptTemplateGenerator
from camel.types import ModelType, ModelPlatformType
from camel.types.enums import RoleType, TaskType
from camel.models import ModelFactory

# Define system message
sys_msg_prompt = PromptTemplateGenerator().get_prompt_from_key(
    TaskType.VIDEO_DESCRIPTION, RoleType.ASSISTANT
)
sys_msg = BaseMessage.make_assistant_message(
    role_name="Assistant",
    content=sys_msg_prompt,
)

# Set model
model=ModelFactory.create(
    model_platform=ModelPlatformType.OPENAI,
    model_type=ModelType.GPT_4O,
    model_config_dict=ChatGPTConfig().as_dict(),
)

# Set agent
camel_agent = ChatAgent(
    sys_msg,
    model=model
)

Providing video and set user message#

[ ]:

# Provide your video path
video_cctv = "/content/CCTV.mov"
with open(video_cctv, "rb") as video_cctv:
    video_bytes_cctv = video_cctv.read()

# Set user message
user_msg_cctv = BaseMessage.make_user_message(
    role_name="User",
    content="These are frames from a video that I want to upload. Generate a"
    "compelling description that I can upload along with the video.",
    video_bytes=video_bytes_cctv,
)

# Get response information
response_cctv = camel_agent.step(user_msg_cctv)
print(response_cctv.msgs[0].content)

"Step into the fascinating world of scientific discovery with our latest video! Watch as a dedicated researcher meticulously works in a high-tech laboratory, surrounded by intricate equipment and cutting-edge technology. This behind-the-scenes footage offers a glimpse into the meticulous process of experimentation and innovation. Join us on this journey of exploration and witness the passion and precision that drive scientific breakthroughs. Don't miss out on this captivating look at the heart of scientific research!"

[ ]:

# Provide your video path
video_help = "/content/help.mov"
with open(video_help, "rb") as video_help:
    video_bytes_help = video_help.read()

# Set user message
user_msg_help = BaseMessage.make_user_message(
    role_name="User",
    content="These are frames from a video that I want to upload. Generate a"
    "compelling description that I can upload along with the video.",
    video_bytes=video_bytes_help,
)

# Get response information
response_help = camel_agent.step(user_msg_help)
print(response_help.msgs[0].content)

"Embark on a breathtaking journey through lush, green landscapes and rugged mountain trails. This video captures the serene beauty of nature, with winding paths leading you through picturesque scenery under a vibrant blue sky. Perfect for nature lovers and adventure seekers alike, this visual escape will transport you to a tranquil world far from the hustle and bustle of everyday life. Join us as we explore the untouched beauty of this stunning trail."

[ ]:

# Provide your video path
video_content_mode = "/content/content mod.mov"
with open(video_content_mode, "rb") as video_content_mode:
    video_bytes_content_mode = video_content_mode.read()

# Set user message
user_msg_content_mode = BaseMessage.make_user_message(
    role_name="User",
    content="These are frames from a video that I want to upload. Generate a"
    "compelling description that I can upload along with the video.",
    video_bytes=video_bytes_content_mode,
)

# Get response information
response_content_mode = camel_agent.step(user_msg_content_mode)
print(response_content_mode.msgs[0].content)

"Join us for a candid glimpse into a moment of relaxation and reflection in the kitchen. Watch as our protagonist unwinds after a long day, savoring a drink and enjoying the simple pleasures of life. This video captures the essence of taking a break and finding comfort in familiar surroundings. Don't miss this relatable and heartwarming scene!"

🌟 Highlights#

This notebook has guided you through setting up an agent and analying videos using CAMEL.

Now, you know how to generate description for uploaded videos.

That’s everything: Got questions about 🐫 CAMEL-AI? Join us on Discord! Whether you want to share feedback, explore the latest in multi-agent systems, get support, or connect with others on exciting projects, we’d love to have you in the community! 🤝

Check out some of our other work:

🐫 Creating Your First CAMEL Agent free Colab
Graph RAG Cookbook free Colab
🧑‍⚖️ Create A Hackathon Judge Committee with Workforce free Colab
🔥 3 ways to ingest data from websites with Firecrawl & CAMEL free Colab
🦥 Agentic SFT Data Generation with CAMEL and Mistral Models, Fine-Tuned with Unsloth free Colab

Thanks from everyone at 🐫 CAMEL-AI

cfdb7559e62948b682608206cb530967