camel.models.reward package

Submodules

camel.models.reward.base_reward_model module

class camel.models.reward.base_reward_model.BaseRewardModel(model_type: ~<unknown>.ModelType | str, api_key: str | None = None, url: str | None = None)

Bases: ABC

Abstract base class for reward models. Reward models are used to evaluate messages and return scores based on different criteria.

Subclasses should implement the ‘evaluate’ and ‘get_scores_types’ methods.

abstract evaluate(messages: List[Dict[str, str]]) → Dict[str, float]

Evaluate the messages and return scores based on different criteria.

Parameters:

messages (List[Dict[str, str]]) – A list of messages where each message is a dictionary with ‘role’ and ‘content’.

Returns:

A dictionary mapping score types to their values.

Return type:

Dict[str, float]

abstract get_scores_types() → List[str]

Get the list of score types that the reward model can return.

Returns:

A list of score types that the reward model can return.

Return type:

List[str]

camel.models.reward.evaluator module

class camel.models.reward.evaluator.Evaluator(reward_model: BaseRewardModel)

Bases: object

Evaluator class to evaluate messages using a reward model and filter data based on the scores.

Parameters:

reward_model (BaseRewardModel) – A reward model to evaluate messages.

evaluate(messages: List[Dict[str, str]]) → Dict[str, float]

Evaluate the messages using the reward model.

Parameters:

messages (List[Dict[str, str]]) – A list of messages where each message is a dictionary with ‘role’ and ‘content’.

Returns:

A dictionary mapping score types to their values.

Return type:

Dict[str, float]

filter_data(messages: List[Dict[str, str]], thresholds: Dict[str, float]) → bool

Filter messages based on the scores.

Parameters:

• messages (List[Dict[str, str]]) – A list of messages where each message is a dictionary with ‘role’ and ‘content’.

• thresholds (Dict[str, float]) – A dictionary mapping score types to their values.

Returns:

A boolean indicating whether the messages pass the filter.

Return type:

bool

camel.models.reward.nemotron_model module

class camel.models.reward.nemotron_model.NemotronRewardModel(model_type: ~<unknown>.ModelType | str, api_key: str | None = None, url: str | None = None)

Bases: BaseRewardModel

Reward model based on the Nemotron model with OpenAI compatibility.

Parameters:

• model_type (Union[ModelType, str]) – Model for which a backend is created.

• api_key (Optional[str], optional) – The API key for authenticating with the model service. (default: None)

• url (Optional[str], optional) – The url to the model service.

Note

The Nemotron model does not support model config.

evaluate(messages: List[Dict[str, str]]) → Dict[str, float]

Evaluate the messages using the Nemotron model.

Parameters:

messages (List[Dict[str, str]]) – A list of messages where each message is a dictionary format.

Returns:

A dictionary mapping score types to their

values.

Return type:

Dict[str, float]

get_scores_types() → List[str]

Get the list of score types that the reward model can return.

Returns:

A list of score types that the reward model can return.

Return type:

List[str]

camel.models.reward.skywork_model module

class camel.models.reward.skywork_model.SkyworkRewardModel(model_type: ~<unknown>.ModelType | str, api_key: str | None = None, url: str | None = None, device_map: str | None = 'auto', attn_implementation: str | None = 'flash_attention_2', offload_folder: str | None = 'offload')

Bases: BaseRewardModel

Reward model based on the transformers, it will download the model from huggingface.

Parameters:

• model_type (Union[ModelType, str]) – Model for which a backend is created.

• api_key (Optional[str], optional) – Not used. (default: None)

• url (Optional[str], optional) – Not used. (default: None)

• device_map (Optional[str], optional) – choose the device map. (default: auto)

• attn_implementation (Optional[str], optional) – choose the attention implementation. (default: flash_attention_2)

• offload_folder (Optional[str], optional) – choose the offload folder. (default: offload)

evaluate(messages: List[Dict[str, str]]) → Dict[str, float]

Evaluate the messages using the Skywork model.

Parameters:

messages (List[Dict[str, str]]) – A list of messages.

Returns:

A ChatCompletion object with the scores.

Return type:

ChatCompletion

get_scores_types() → List[str]

get the scores types

Returns:

list of scores types

Return type:

List[str]

Module contents

class camel.models.reward.BaseRewardModel(model_type: ~<unknown>.ModelType | str, api_key: str | None = None, url: str | None = None)

Bases: ABC

Abstract base class for reward models. Reward models are used to evaluate messages and return scores based on different criteria.

Subclasses should implement the ‘evaluate’ and ‘get_scores_types’ methods.

abstract evaluate(messages: List[Dict[str, str]]) → Dict[str, float]

Evaluate the messages and return scores based on different criteria.

Parameters:

messages (List[Dict[str, str]]) – A list of messages where each message is a dictionary with ‘role’ and ‘content’.

Returns:

A dictionary mapping score types to their values.

Return type:

Dict[str, float]

abstract get_scores_types() → List[str]

Get the list of score types that the reward model can return.

Returns:

A list of score types that the reward model can return.

Return type:

List[str]

class camel.models.reward.Evaluator(reward_model: BaseRewardModel)

Bases: object

Evaluator class to evaluate messages using a reward model and filter data based on the scores.

Parameters:

reward_model (BaseRewardModel) – A reward model to evaluate messages.

evaluate(messages: List[Dict[str, str]]) → Dict[str, float]

Evaluate the messages using the reward model.

Parameters:

messages (List[Dict[str, str]]) – A list of messages where each message is a dictionary with ‘role’ and ‘content’.

Returns:

A dictionary mapping score types to their values.

Return type:

Dict[str, float]

filter_data(messages: List[Dict[str, str]], thresholds: Dict[str, float]) → bool

Filter messages based on the scores.

Parameters:

• messages (List[Dict[str, str]]) – A list of messages where each message is a dictionary with ‘role’ and ‘content’.

• thresholds (Dict[str, float]) – A dictionary mapping score types to their values.

Returns:

A boolean indicating whether the messages pass the filter.

Return type:

bool

class camel.models.reward.NemotronRewardModel(model_type: ~<unknown>.ModelType | str, api_key: str | None = None, url: str | None = None)

Bases: BaseRewardModel

Reward model based on the Nemotron model with OpenAI compatibility.

Parameters:

• model_type (Union[ModelType, str]) – Model for which a backend is created.

• api_key (Optional[str], optional) – The API key for authenticating with the model service. (default: None)

• url (Optional[str], optional) – The url to the model service.

Note

The Nemotron model does not support model config.

evaluate(messages: List[Dict[str, str]]) → Dict[str, float]

Evaluate the messages using the Nemotron model.

Parameters:

messages (List[Dict[str, str]]) – A list of messages where each message is a dictionary format.

Returns:

A dictionary mapping score types to their

values.

Return type:

Dict[str, float]

get_scores_types() → List[str]

Get the list of score types that the reward model can return.

Returns:

A list of score types that the reward model can return.

Return type:

List[str]

class camel.models.reward.SkyworkRewardModel(model_type: ~<unknown>.ModelType | str, api_key: str | None = None, url: str | None = None, device_map: str | None = 'auto', attn_implementation: str | None = 'flash_attention_2', offload_folder: str | None = 'offload')

Bases: BaseRewardModel

Reward model based on the transformers, it will download the model from huggingface.

Parameters:

• model_type (Union[ModelType, str]) – Model for which a backend is created.

• api_key (Optional[str], optional) – Not used. (default: None)

• url (Optional[str], optional) – Not used. (default: None)

• device_map (Optional[str], optional) – choose the device map. (default: auto)

• attn_implementation (Optional[str], optional) – choose the attention implementation. (default: flash_attention_2)

• offload_folder (Optional[str], optional) – choose the offload folder. (default: offload)

evaluate(messages: List[Dict[str, str]]) → Dict[str, float]

Evaluate the messages using the Skywork model.

Parameters:

messages (List[Dict[str, str]]) – A list of messages.

Returns:

A ChatCompletion object with the scores.

Return type:

ChatCompletion

get_scores_types() → List[str]

get the scores types

Returns:

list of scores types

Return type:

List[str]