Camel.utils.token counting
get_model_encoding
Get model encoding from tiktoken.
Parameters:
- value_for_tiktoken: Model value for tiktoken.
Returns:
tiktoken.Encoding: Model encoding.
BaseTokenCounter
Base class for token counters of different kinds of models.
count_tokens_from_messages
Count number of tokens in the provided message list.
Parameters:
- messages (List[OpenAIMessage]): Message list with the chat history in OpenAI API format.
Returns:
int: Number of tokens in the messages.
encode
Encode text into token IDs.
Parameters:
- text (str): The text to encode.
Returns:
List[int]: List of token IDs.
decode
Decode token IDs back to text.
Parameters:
- token_ids (List[int]): List of token IDs to decode.
Returns:
str: Decoded text.
OpenAITokenCounter
init
Constructor for the token counter for OpenAI models.
Parameters:
- model (UnifiedModelType): Model type for which tokens will be counted.
count_tokens_from_messages
Count number of tokens in the provided message list with the help of package tiktoken.
Parameters:
- messages (List[OpenAIMessage]): Message list with the chat history in OpenAI API format.
Returns:
int: Number of tokens in the messages.
_count_tokens_from_image
Count image tokens for OpenAI vision model. An :obj:"auto"
resolution model will be treated as :obj:"high"
. All images with
:obj:"low"
detail cost 85 tokens each. Images with :obj:"high"
detail
are first scaled to fit within a 2048 x 2048 square, maintaining their
aspect ratio. Then, they are scaled such that the shortest side of the
image is 768px long. Finally, we count how many 512px squares the image
consists of. Each of those squares costs 170 tokens. Another 85 tokens are
always added to the final total. For more details please refer to OpenAI
vision docs
Parameters:
- image (PIL.Image.Image): Image to count number of tokens.
- detail (OpenAIVisionDetailType): Image detail type to count number of tokens.
Returns:
int: Number of tokens for the image given a detail type.
encode
Encode text into token IDs.
Parameters:
- text (str): The text to encode.
Returns:
List[int]: List of token IDs.
decode
Decode token IDs back to text.
Parameters:
- token_ids (List[int]): List of token IDs to decode.
Returns:
str: Decoded text.
AnthropicTokenCounter
init
Constructor for the token counter for Anthropic models.
Parameters:
- model (str): The name of the Anthropic model being used.
count_tokens_from_messages
Count number of tokens in the provided message list using loaded tokenizer specific for this type of model.
Parameters:
- messages (List[OpenAIMessage]): Message list with the chat history in OpenAI API format.
Returns:
int: Number of tokens in the messages.
encode
Encode text into token IDs.
Parameters:
- text (str): The text to encode.
Returns:
List[int]: List of token IDs.
decode
Decode token IDs back to text.
Parameters:
- token_ids (List[int]): List of token IDs to decode.
Returns:
str: Decoded text.
LiteLLMTokenCounter
init
Constructor for the token counter for LiteLLM models.
Parameters:
- model_type (UnifiedModelType): Model type for which tokens will be counted.
token_counter
completion_cost
count_tokens_from_messages
Count number of tokens in the provided message list using the tokenizer specific to this type of model.
Parameters:
- messages (List[OpenAIMessage]): Message list with the chat history in LiteLLM API format.
Returns:
int: Number of tokens in the messages.
calculate_cost_from_response
Calculate the cost of the given completion response.
Parameters:
- response (dict): The completion response from LiteLLM.
Returns:
float: The cost of the completion call in USD.
encode
Encode text into token IDs.
Parameters:
- text (str): The text to encode.
Returns:
List[int]: List of token IDs.
decode
Decode token IDs back to text.
Parameters:
- token_ids (List[int]): List of token IDs to decode.
Returns:
str: Decoded text.
MistralTokenCounter
init
Constructor for the token counter for Mistral models.
Parameters:
- model_type (ModelType): Model type for which tokens will be counted.
count_tokens_from_messages
Count number of tokens in the provided message list using loaded tokenizer specific for this type of model.
Parameters:
- messages (List[OpenAIMessage]): Message list with the chat history in OpenAI API format.
Returns:
int: Total number of tokens in the messages.
_convert_response_from_openai_to_mistral
Convert an OpenAI message to a Mistral ChatCompletionRequest.
Parameters:
- openai_msg (OpenAIMessage): An individual message with OpenAI format.
Returns:
ChatCompletionRequest: The converted message in Mistral’s request format.
encode
Encode text into token IDs.
Parameters:
- text (str): The text to encode.
Returns:
List[int]: List of token IDs.
decode
Decode token IDs back to text.
Parameters:
- token_ids (List[int]): List of token IDs to decode.
Returns:
str: Decoded text.