Role-Playing Scraper for Report & Knowledge Graph Generation

Role-Playing Scraper for Report & Knowledge Graph Generation#

You can also check this cookbook in colab here

This notebook demonstrates how to set up and leverage CAMEL’s Retrieval-Augmented Generation (RAG) combined with Firecrawl for efficient web scraping, multi-agent role-playing tasks, and knowledge graph construction. We will walk through an example of conducting a comprehensive study of the Turkish shooter in the 2024 Paris Olympics by using Mistral’s models.

In this notebook, you’ll explore:

  • CAMEL: A powerful multi-agent framework that enables Retrieval-Augmented Generation and multi-agent role-playing scenarios, allowing for sophisticated AI-driven tasks.

  • Mistral: Utilized for its state-of-the-art language models, which enable tool-calling capabilities to execute external functions, while its powerful embeddings are employed for semantic search and content retrieval.

  • Firecrawl: A robust web scraping tool that simplifies extracting and cleaning content from various web pages.

  • AgentOps: Track and analysis the running of CAMEL Agents.

  • Qdrant: An efficient vector storage system used with CAMEL’s AutoRetriever to store and retrieve relevant information based on vector similarities.

  • Neo4j: A leading graph database management system used for constructing and storing knowledge graphs, enabling complex relationships between entities to be mapped and queried efficiently.

  • DuckDuckGo Search: Utilized within the SearchToolkit to gather relevant URLs and information from the web, serving as the primary search engine for retrieving initial content.

  • Unstructured IO: Used for content chunking, facilitating the management of unstructured data for more efficient processing.

This setup not only demonstrates a practical application but also serves as a flexible framework that can be adapted for various scenarios requiring advanced web information retrieval, AI collaboration, and multi-source data aggregation.

⭐ Star the Repo

If you find CAMEL useful or interesting, please consider giving it a star on our CAMEL GitHub Repo! Your stars help others find this project and motivate us to continue improving it.