camel.storages.object_storages package

camel.storages.object_storages package#

Submodules#

camel.storages.object_storages.amazon_s3 module#

class camel.storages.object_storages.amazon_s3.AmazonS3Storage(bucket_name: str, create_if_not_exists: bool = True, access_key_id: str | None = None, secret_access_key: str | None = None, anonymous: bool = False)[source]#

Bases: BaseObjectStorage

A class to connect with AWS S3 object storage to put and get objects from one S3 bucket. The class will first try to use the credentials passed as arguments, if not provided, it will look for the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. If none of these are provided, it will try to use the local credentials (will be created if logged in with AWS CLI).

Parameters:

  • bucket_name (str) – The name of the S3 bucket.

  • create_if_not_exists (bool, optional) – Whether to create the bucket if it does not exist. Defaults to True.

  • access_key_id (Optional[str], optional) – The AWS access key ID. Defaults to None.

  • secret_access_key (Optional[str], optional) – The AWS secret access key. Defaults to None.

  • anonymous (bool, optional) – Whether to use anonymous access. Defaults to False.

References

https://aws.amazon.com/pm/serv-s3/

https://aws.amazon.com/cli/

static canonicalize_path(file_path: PurePath) → Tuple[str, str][source]#

Canonicalize file path for Amazon S3.

Parameters:

file_path (PurePath) – The path to be canonicalized.

Returns:

The canonicalized file key and file name.

Return type:

Tuple[str, str]

camel.storages.object_storages.azure_blob module#

class camel.storages.object_storages.azure_blob.AzureBlobStorage(storage_account_name: str, container_name: str, create_if_not_exists: bool = True, access_key: str | None = None)[source]#

Bases: BaseObjectStorage

A class to connect to Azure Blob Storage. It will connect to one container in the storage account.

Parameters:

  • storage_account_name (str) – The name of the storage account.

  • container_name (str) – The name of the container.

  • access_key (Optional[str], optional) – The access key of the storage account. Defaults to None.

References

https://azure.microsoft.com/en-us/products/storage/blobs

static canonicalize_path(file_path: PurePath) → Tuple[str, str][source]#

Canonicalize file path for Azure Blob Storage.

Parameters:

file_path (PurePath) – The path to be canonicalized.

Returns:

The canonicalized file key and file name.

Return type:

Tuple[str, str]

camel.storages.object_storages.base module#

class camel.storages.object_storages.base.BaseObjectStorage[source]#

Bases: ABC

abstract static canonicalize_path(file_path: PurePath) → Tuple[str, str][source]#
download_file(local_file_path: Path, remote_file_path: PurePath) → None[source]#

Download a file from the object storage to the local system.

Parameters:

  • local_file_path (Path) – The path to the local file to be saved.

  • remote_file_path (PurePath) – The path to the object in storage.

get_file(file_path: PurePath) → File[source]#

Get a file from the object storage.

Parameters:

file_path (PurePath) – The path to the object in the storage.

Returns:

The file object get from the storage.

Return type:

File

object_exists(file_path: PurePath) → bool[source]#

Check if the object exists in the storage.

Parameters:

file_path (PurePath) – The path to the object in the storage.

Returns:

True if the object exists, False otherwise.

Return type:

bool

put_file(file_path: PurePath, file: File) → None[source]#

Put a file to the object storage.

Parameters:

  • file_path (PurePath) – The path to the object in the storage.

  • file (File) – The file to be put.

upload_file(local_file_path: Path, remote_file_path: PurePath) → None[source]#

Upload a local file to the object storage.

Parameters:

  • local_file_path (Path) – The path to the local file to be uploaded.

  • remote_file_path (PurePath) – The path to the object in storage.

camel.storages.object_storages.google_cloud module#

class camel.storages.object_storages.google_cloud.GoogleCloudStorage(bucket_name: str, create_if_not_exists: bool = True, anonymous: bool = False)[source]#

Bases: BaseObjectStorage

A class to connect to Google Cloud Storage. It will connect to one bucket in the storage account.

Note that Google Cloud Storage does not support api key authentication. Therefore, before using this class, you need to log in with gcloud command line tool and save the credentials first.

Parameters:

  • bucket_name (str) – The name of the bucket.

  • create_if_not_exists (bool, optional) – Whether to create the bucket if it does not exist. Defaults to True.

  • anonymous (bool, optional) – Whether to use anonymous access. Defaults to False.

References

https://cloud.google.com/storage

https://cloud.google.com/docs/authentication/api-keys

static canonicalize_path(file_path: PurePath) → Tuple[str, str][source]#

Canonicalize the path for Google Cloud Storage.

Parameters:

file_path (PurePath) – The path to be canonicalized.

Returns:

The canonicalized file key and file name.

Return type:

Tuple[str, str]

Module contents#

class camel.storages.object_storages.AmazonS3Storage(bucket_name: str, create_if_not_exists: bool = True, access_key_id: str | None = None, secret_access_key: str | None = None, anonymous: bool = False)[source]#

Bases: BaseObjectStorage

A class to connect with AWS S3 object storage to put and get objects from one S3 bucket. The class will first try to use the credentials passed as arguments, if not provided, it will look for the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. If none of these are provided, it will try to use the local credentials (will be created if logged in with AWS CLI).

Parameters:

  • bucket_name (str) – The name of the S3 bucket.

  • create_if_not_exists (bool, optional) – Whether to create the bucket if it does not exist. Defaults to True.

  • access_key_id (Optional[str], optional) – The AWS access key ID. Defaults to None.

  • secret_access_key (Optional[str], optional) – The AWS secret access key. Defaults to None.

  • anonymous (bool, optional) – Whether to use anonymous access. Defaults to False.

References

https://aws.amazon.com/pm/serv-s3/

https://aws.amazon.com/cli/

static canonicalize_path(file_path: PurePath) → Tuple[str, str][source]#

Canonicalize file path for Amazon S3.

Parameters:

file_path (PurePath) – The path to be canonicalized.

Returns:

The canonicalized file key and file name.

Return type:

Tuple[str, str]

class camel.storages.object_storages.AzureBlobStorage(storage_account_name: str, container_name: str, create_if_not_exists: bool = True, access_key: str | None = None)[source]#

Bases: BaseObjectStorage

A class to connect to Azure Blob Storage. It will connect to one container in the storage account.

Parameters:

  • storage_account_name (str) – The name of the storage account.

  • container_name (str) – The name of the container.

  • access_key (Optional[str], optional) – The access key of the storage account. Defaults to None.

References

https://azure.microsoft.com/en-us/products/storage/blobs

static canonicalize_path(file_path: PurePath) → Tuple[str, str][source]#

Canonicalize file path for Azure Blob Storage.

Parameters:

file_path (PurePath) – The path to be canonicalized.

Returns:

The canonicalized file key and file name.

Return type:

Tuple[str, str]

class camel.storages.object_storages.GoogleCloudStorage(bucket_name: str, create_if_not_exists: bool = True, anonymous: bool = False)[source]#

Bases: BaseObjectStorage

A class to connect to Google Cloud Storage. It will connect to one bucket in the storage account.

Note that Google Cloud Storage does not support api key authentication. Therefore, before using this class, you need to log in with gcloud command line tool and save the credentials first.

Parameters:

  • bucket_name (str) – The name of the bucket.

  • create_if_not_exists (bool, optional) – Whether to create the bucket if it does not exist. Defaults to True.

  • anonymous (bool, optional) – Whether to use anonymous access. Defaults to False.

References

https://cloud.google.com/storage

https://cloud.google.com/docs/authentication/api-keys

static canonicalize_path(file_path: PurePath) → Tuple[str, str][source]#

Canonicalize the path for Google Cloud Storage.

Parameters:

file_path (PurePath) – The path to be canonicalized.

Returns:

The canonicalized file key and file name.

Return type:

Tuple[str, str]