camel.storages.vectordb_storages package

Submodules

camel.storages.vectordb_storages.base module

class camel.storages.vectordb_storages.base.BaseVectorStorage

Bases: ABC

An abstract base class for vector storage systems.

abstract add(records: List[VectorRecord], **kwargs: Any) → None

Saves a list of vector records to the storage.

Parameters:

• records (List[VectorRecord]) – List of vector records to be saved.

• **kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the saving process.

abstract clear() → None

Remove all vectors from the storage.

abstract property client: Any

Provides access to the underlying vector database client.

abstract delete(ids: List[str], **kwargs: Any) → None

Deletes a list of vectors identified by their IDs from the storage.

Parameters:

• ids (List[str]) – List of unique identifiers for the vectors to be deleted.

• **kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the deletion process.

get_payloads_by_vector(vector: List[float], top_k: int) → List[Dict[str, Any]]

Returns payloads of top k vector records that closest to the given vector.

This function is a wrapper of BaseVectorStorage.query.

Parameters:

• vector (List[float]) – The search vector.

• top_k (int) – The number of top similer vectors.

Returns:

A list of vector payloads retrieved

from the storage based on similarity to the query vector.

Return type:

List[List[Dict[str, Any]]]

abstract load() → None

Load the collection hosted on cloud service.

abstract query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult]

Searches for similar vectors in the storage based on the provided query.

Parameters:

• query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.

• **kwargs (Any) – Additional keyword arguments.

Returns:

A list of vectors retrieved from the

storage based on similarity to the query vector.

Return type:

List[VectorDBQueryResult]

abstract status() → VectorDBStatus

Returns status of the vector database.

Returns:

The vector database status.

Return type:

VectorDBStatus

class camel.storages.vectordb_storages.base.VectorDBQuery(query_vector: List[float], top_k: int)

Bases: BaseModel

Represents a query to a vector database.

query_vector

The numerical representation of the query vector.

Type:

List[float]

top_k

The number of top similar vectors to retrieve from the database. (default: 1)

Type:

int, optional

model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, FieldInfo]] = {'query_vector': FieldInfo(annotation=List[float], required=True), 'top_k': FieldInfo(annotation=int, required=False, default=1)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

query_vector: List[float]

The numerical representation of the query vector.

top_k: int

The number of top similar vectors to retrieve from the database.

class camel.storages.vectordb_storages.base.VectorDBQueryResult(*, record: VectorRecord, similarity: float)

Bases: BaseModel

Encapsulates the result of a query against a vector database.

record

The target vector record.

Type:

VectorRecord

similarity

The similarity score between the query vector and the record.

Type:

float

classmethod create(similarity: float, vector: List[float], id: str, payload: Dict[str, Any] | None = None) → VectorDBQueryResult

A class method to construct a VectorDBQueryResult instance.

model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, FieldInfo]] = {'record': FieldInfo(annotation=VectorRecord, required=True), 'similarity': FieldInfo(annotation=float, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

record: VectorRecord

similarity: float

class camel.storages.vectordb_storages.base.VectorDBStatus(*, vector_dim: int, vector_count: int)

Bases: BaseModel

Vector database status.

vector_dim

The dimention of stored vectors.

Type:

int

vector_count

The number of stored vectors.

Type:

int

model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, FieldInfo]] = {'vector_count': FieldInfo(annotation=int, required=True), 'vector_dim': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

vector_count: int

vector_dim: int

class camel.storages.vectordb_storages.base.VectorRecord(*, vector: List[float], id: str = None, payload: Dict[str, Any] | None = None)

Bases: BaseModel

Encapsulates information about a vector’s unique identifier and its payload, which is primarily used as a data transfer object when saving to vector storage.

vector

The numerical representation of the vector.

Type:

List[float]

id

A unique identifier for the vector. If not provided, an random uuid will be assigned.

Type:

str, optional

payload

Any additional metadata or information related to the vector. (default: None)

Type:

Optional[Dict[str, Any]], optional

id: str

model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, FieldInfo]] = {'id': FieldInfo(annotation=str, required=False, default_factory=<lambda>), 'payload': FieldInfo(annotation=Union[Dict[str, Any], NoneType], required=False, default=None), 'vector': FieldInfo(annotation=List[float], required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

payload: Dict[str, Any] | None

vector: List[float]

camel.storages.vectordb_storages.milvus module

class camel.storages.vectordb_storages.milvus.MilvusStorage(vector_dim: int, url_and_api_key: Tuple[str, str], collection_name: str | None = None, **kwargs: Any)

Bases: BaseVectorStorage

An implementation of the BaseVectorStorage for interacting with Milvus, a cloud-native vector search engine.

The detailed information about Milvus is available at: Milvus

Parameters:

• vector_dim (int) – The dimenstion of storing vectors.

• url_and_api_key (Tuple[str, str]) – Tuple containing the URL and API key for connecting to a remote Milvus instance. URL maps to Milvus uri concept, typically “endpoint:port”. API key maps to Milvus token concept, for self-hosted it’s “username:pwd”, for Zilliz Cloud (fully-managed Milvus) it’s API Key.

• collection_name (Optional[str], optional) – Name for the collection in the Milvus. If not provided, set it to the current time with iso format. (default: None)

• **kwargs (Any) – Additional keyword arguments for initializing MilvusClient.

Raises:

ImportError – If pymilvus package is not installed.

add(records: List[VectorRecord], **kwargs) → None

Adds a list of vectors to the specified collection.

Parameters:

• records (List[VectorRecord]) – List of vectors to be added.

• **kwargs (Any) – Additional keyword arguments pass to insert.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None

Removes all vectors from the Milvus collection. This method deletes the existing collection and then recreates it with the same schema to effectively remove all stored vectors.

property client: Any

Provides direct access to the Milvus client. This property allows for direct interactions with the Milvus client for operations that are not covered by the MilvusStorage class.

Returns:

The Milvus client instance.

Return type:

Any

delete(ids: List[str], **kwargs: Any) → None

Deletes a list of vectors identified by their IDs from the storage. If unsure of ids you can first query the collection to grab the corresponding data.

Parameters:

• ids (List[str]) – List of unique identifiers for the vectors to be deleted.

• **kwargs (Any) – Additional keyword arguments passed to delete.

Raises:

RuntimeError – If there is an error during the deletion process.

load() → None

Load the collection hosted on cloud service.

query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult]

Searches for similar vectors in the storage based on the provided query.

Parameters:

• query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.

• **kwargs (Any) – Additional keyword arguments passed to search.

Returns:

A list of vectors retrieved from the

storage based on similarity to the query vector.

Return type:

List[VectorDBQueryResult]

status() → VectorDBStatus

Retrieves the current status of the Milvus collection. This method provides information about the collection, including its vector dimensionality and the total number of vectors stored.

Returns:

An object containing information about the

collection’s status.

Return type:

VectorDBStatus

camel.storages.vectordb_storages.qdrant module

class camel.storages.vectordb_storages.qdrant.QdrantStorage(vector_dim: int, collection_name: str | None = None, url_and_api_key: Tuple[str, str] | None = None, path: str | None = None, distance: VectorDistance = VectorDistance.COSINE, delete_collection_on_del: bool = False, **kwargs: Any)

Bases: BaseVectorStorage

An implementation of the BaseVectorStorage for interacting with Qdrant, a vector search engine.

The detailed information about Qdrant is available at: Qdrant

Parameters:

• vector_dim (int) – The dimenstion of storing vectors.

• collection_name (Optional[str], optional) – Name for the collection in the Qdrant. If not provided, set it to the current time with iso format. (default: None)

• url_and_api_key (Optional[Tuple[str, str]], optional) – Tuple containing the URL and API key for connecting to a remote Qdrant instance. (default: None)

• path (Optional[str], optional) – Path to a directory for initializing a local Qdrant client. (default: None)

• distance (VectorDistance, optional) – The distance metric for vector comparison (default: VectorDistance.COSINE)

• delete_collection_on_del (bool, optional) – Flag to determine if the collection should be deleted upon object destruction. (default: False)

• **kwargs (Any) – Additional keyword arguments for initializing QdrantClient.

Notes

• If url_and_api_key is provided, it takes priority and the client will attempt to connect to the remote Qdrant instance using the URL endpoint.

• If url_and_api_key is not provided and path is given, the client will use the local path to initialize Qdrant.

• If neither url_and_api_key nor path is provided, the client will be initialized with an in-memory storage (“:memory:”).

add(records: List[VectorRecord], **kwargs) → None

Adds a list of vectors to the specified collection.

Parameters:

• vectors (List[VectorRecord]) – List of vectors to be added.

• **kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None

Remove all vectors from the storage.

property client: Any

Provides access to the underlying vector database client.

delete(ids: List[str], **kwargs: Any) → None

Deletes a list of vectors identified by their IDs from the storage.

Parameters:

• ids (List[str]) – List of unique identifiers for the vectors to be deleted.

• **kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the deletion process.

load() → None

Load the collection hosted on cloud service.

query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult]

Searches for similar vectors in the storage based on the provided query.

Parameters:

• query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.

• **kwargs (Any) – Additional keyword arguments.

Returns:

A list of vectors retrieved from the

storage based on similarity to the query vector.

Return type:

List[VectorDBQueryResult]

status() → VectorDBStatus

Returns status of the vector database.

Returns:

The vector database status.

Return type:

VectorDBStatus

Module contents

class camel.storages.vectordb_storages.BaseVectorStorage

Bases: ABC

An abstract base class for vector storage systems.

abstract add(records: List[VectorRecord], **kwargs: Any) → None

Saves a list of vector records to the storage.

Parameters:

• records (List[VectorRecord]) – List of vector records to be saved.

• **kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the saving process.

abstract clear() → None

Remove all vectors from the storage.

abstract property client: Any

Provides access to the underlying vector database client.

abstract delete(ids: List[str], **kwargs: Any) → None

Deletes a list of vectors identified by their IDs from the storage.

Parameters:

• ids (List[str]) – List of unique identifiers for the vectors to be deleted.

• **kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the deletion process.

get_payloads_by_vector(vector: List[float], top_k: int) → List[Dict[str, Any]]

Returns payloads of top k vector records that closest to the given vector.

This function is a wrapper of BaseVectorStorage.query.

Parameters:

• vector (List[float]) – The search vector.

• top_k (int) – The number of top similer vectors.

Returns:

A list of vector payloads retrieved

from the storage based on similarity to the query vector.

Return type:

List[List[Dict[str, Any]]]

abstract load() → None

Load the collection hosted on cloud service.

abstract query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult]

Searches for similar vectors in the storage based on the provided query.

Parameters:

• query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.

• **kwargs (Any) – Additional keyword arguments.

Returns:

A list of vectors retrieved from the

storage based on similarity to the query vector.

Return type:

List[VectorDBQueryResult]

abstract status() → VectorDBStatus

Returns status of the vector database.

Returns:

The vector database status.

Return type:

VectorDBStatus

class camel.storages.vectordb_storages.MilvusStorage(vector_dim: int, url_and_api_key: Tuple[str, str], collection_name: str | None = None, **kwargs: Any)

Bases: BaseVectorStorage

An implementation of the BaseVectorStorage for interacting with Milvus, a cloud-native vector search engine.

The detailed information about Milvus is available at: Milvus

Parameters:

• vector_dim (int) – The dimenstion of storing vectors.

• url_and_api_key (Tuple[str, str]) – Tuple containing the URL and API key for connecting to a remote Milvus instance. URL maps to Milvus uri concept, typically “endpoint:port”. API key maps to Milvus token concept, for self-hosted it’s “username:pwd”, for Zilliz Cloud (fully-managed Milvus) it’s API Key.

• collection_name (Optional[str], optional) – Name for the collection in the Milvus. If not provided, set it to the current time with iso format. (default: None)

• **kwargs (Any) – Additional keyword arguments for initializing MilvusClient.

Raises:

ImportError – If pymilvus package is not installed.

add(records: List[VectorRecord], **kwargs) → None

Adds a list of vectors to the specified collection.

Parameters:

• records (List[VectorRecord]) – List of vectors to be added.

• **kwargs (Any) – Additional keyword arguments pass to insert.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None

Removes all vectors from the Milvus collection. This method deletes the existing collection and then recreates it with the same schema to effectively remove all stored vectors.

property client: Any

Provides direct access to the Milvus client. This property allows for direct interactions with the Milvus client for operations that are not covered by the MilvusStorage class.

Returns:

The Milvus client instance.

Return type:

Any

delete(ids: List[str], **kwargs: Any) → None

Deletes a list of vectors identified by their IDs from the storage. If unsure of ids you can first query the collection to grab the corresponding data.

Parameters:

• ids (List[str]) – List of unique identifiers for the vectors to be deleted.

• **kwargs (Any) – Additional keyword arguments passed to delete.

Raises:

RuntimeError – If there is an error during the deletion process.

load() → None

Load the collection hosted on cloud service.

query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult]

Searches for similar vectors in the storage based on the provided query.

Parameters:

• query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.

• **kwargs (Any) – Additional keyword arguments passed to search.

Returns:

A list of vectors retrieved from the

storage based on similarity to the query vector.

Return type:

List[VectorDBQueryResult]

status() → VectorDBStatus

Retrieves the current status of the Milvus collection. This method provides information about the collection, including its vector dimensionality and the total number of vectors stored.

Returns:

An object containing information about the

collection’s status.

Return type:

VectorDBStatus

class camel.storages.vectordb_storages.QdrantStorage(vector_dim: int, collection_name: str | None = None, url_and_api_key: Tuple[str, str] | None = None, path: str | None = None, distance: VectorDistance = VectorDistance.COSINE, delete_collection_on_del: bool = False, **kwargs: Any)

Bases: BaseVectorStorage

An implementation of the BaseVectorStorage for interacting with Qdrant, a vector search engine.

The detailed information about Qdrant is available at: Qdrant

Parameters:

• vector_dim (int) – The dimenstion of storing vectors.

• collection_name (Optional[str], optional) – Name for the collection in the Qdrant. If not provided, set it to the current time with iso format. (default: None)

• url_and_api_key (Optional[Tuple[str, str]], optional) – Tuple containing the URL and API key for connecting to a remote Qdrant instance. (default: None)

• path (Optional[str], optional) – Path to a directory for initializing a local Qdrant client. (default: None)

• distance (VectorDistance, optional) – The distance metric for vector comparison (default: VectorDistance.COSINE)

• delete_collection_on_del (bool, optional) – Flag to determine if the collection should be deleted upon object destruction. (default: False)

• **kwargs (Any) – Additional keyword arguments for initializing QdrantClient.

Notes

• If url_and_api_key is provided, it takes priority and the client will attempt to connect to the remote Qdrant instance using the URL endpoint.

• If url_and_api_key is not provided and path is given, the client will use the local path to initialize Qdrant.

• If neither url_and_api_key nor path is provided, the client will be initialized with an in-memory storage (“:memory:”).

add(records: List[VectorRecord], **kwargs) → None

Adds a list of vectors to the specified collection.

Parameters:

• vectors (List[VectorRecord]) – List of vectors to be added.

• **kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None

Remove all vectors from the storage.

property client: Any

Provides access to the underlying vector database client.

delete(ids: List[str], **kwargs: Any) → None

Deletes a list of vectors identified by their IDs from the storage.

Parameters:

• ids (List[str]) – List of unique identifiers for the vectors to be deleted.

• **kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the deletion process.

load() → None

Load the collection hosted on cloud service.

query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult]

Searches for similar vectors in the storage based on the provided query.

Parameters:

• query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.

• **kwargs (Any) – Additional keyword arguments.

Returns:

A list of vectors retrieved from the

storage based on similarity to the query vector.

Return type:

List[VectorDBQueryResult]

status() → VectorDBStatus

Returns status of the vector database.

Returns:

The vector database status.

Return type:

VectorDBStatus

class camel.storages.vectordb_storages.VectorDBQuery(query_vector: List[float], top_k: int)

Bases: BaseModel

Represents a query to a vector database.

query_vector

The numerical representation of the query vector.

Type:

List[float]

top_k

The number of top similar vectors to retrieve from the database. (default: 1)

Type:

int, optional

model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, FieldInfo]] = {'query_vector': FieldInfo(annotation=List[float], required=True), 'top_k': FieldInfo(annotation=int, required=False, default=1)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

query_vector: List[float]

The numerical representation of the query vector.

top_k: int

The number of top similar vectors to retrieve from the database.

class camel.storages.vectordb_storages.VectorDBQueryResult(*, record: VectorRecord, similarity: float)

Bases: BaseModel

Encapsulates the result of a query against a vector database.

record

The target vector record.

Type:

VectorRecord

similarity

The similarity score between the query vector and the record.

Type:

float

classmethod create(similarity: float, vector: List[float], id: str, payload: Dict[str, Any] | None = None) → VectorDBQueryResult

A class method to construct a VectorDBQueryResult instance.

model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, FieldInfo]] = {'record': FieldInfo(annotation=VectorRecord, required=True), 'similarity': FieldInfo(annotation=float, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

record: VectorRecord

similarity: float

class camel.storages.vectordb_storages.VectorDBStatus(*, vector_dim: int, vector_count: int)

Bases: BaseModel

Vector database status.

vector_dim

The dimention of stored vectors.

Type:

int

vector_count

The number of stored vectors.

Type:

int

model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, FieldInfo]] = {'vector_count': FieldInfo(annotation=int, required=True), 'vector_dim': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

vector_count: int

vector_dim: int

class camel.storages.vectordb_storages.VectorRecord(*, vector: List[float], id: str = None, payload: Dict[str, Any] | None = None)

Bases: BaseModel

Encapsulates information about a vector’s unique identifier and its payload, which is primarily used as a data transfer object when saving to vector storage.

vector

The numerical representation of the vector.

Type:

List[float]

id

A unique identifier for the vector. If not provided, an random uuid will be assigned.

Type:

str, optional

payload

Any additional metadata or information related to the vector. (default: None)

Type:

Optional[Dict[str, Any]], optional

id: str

model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, FieldInfo]] = {'id': FieldInfo(annotation=str, required=False, default_factory=<lambda>), 'payload': FieldInfo(annotation=Union[Dict[str, Any], NoneType], required=False, default=None), 'vector': FieldInfo(annotation=List[float], required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

payload: Dict[str, Any] | None

vector: List[float]