​
BaseLoader

class BaseLoader(ABC):
Abstract base class for all data loaders in CAMEL.

​
_load_single

def _load_single(self, source: Union[str, Path]):
Load data from a single source. Parameters:
  • source (Union[str, Path]): The data source to load from.
Returns: Dict[str, Any]: A dictionary containing the loaded data. It is recommended that the dictionary includes a “content” key with the primary data and optional metadata keys.

​
load

def load(self, source: Union[str, Path, List[Union[str, Path]]]):
Load data from one or multiple sources. Parameters:
  • source (Union[str, Path, List[Union[str, Path]]]): The data source (s) to load from. Can be: - A single path/URL (str or Path) - A list of paths/URLs
Returns: Dict[str, List[Dict[str, Any]]]: A dictionary with a single key “contents” containing a list of loaded data. If a single source is provided, the list will contain a single item.

​
supported_formats

def supported_formats(self):
Returns: set[str]: A set of strings representing the supported formats/ sources.