​
BaseLoader

class BaseLoader(ABC):

Abstract base class for all data loaders in CAMEL.

​
_load_single

def _load_single(self, source: Union[str, Path]):

Load data from a single source.

Parameters:

  • source (Union[str, Path]): The data source to load from.

Returns:

Dict[str, Any]: A dictionary containing the loaded data. It is recommended that the dictionary includes a “content” key with the primary data and optional metadata keys.

​
load

def load(self, source: Union[str, Path, List[Union[str, Path]]]):

Load data from one or multiple sources.

Parameters:

  • source (Union[str, Path, List[Union[str, Path]]]): The data source (s) to load from. Can be: - A single path/URL (str or Path) - A list of paths/URLs

Returns:

Dict[str, List[Dict[str, Any]]]: A dictionary with a single key “contents” containing a list of loaded data. If a single source is provided, the list will contain a single item.

​
supported_formats

def supported_formats(self):

Returns:

set[str]: A set of strings representing the supported formats/ sources.