Camel.models.samba model
SambaModel
SambaNova service interface.
Parameters:
- model_type (Union[ModelType, str]): Model for which a SambaNova backend is created. Supported models via SambaNova Cloud:
https://community.sambanova.ai/t/supported-models/193
. Supported models via SambaVerse API is listed inhttps://sambaverse.sambanova.ai/models
. - model_config_dict (Optional[Dict[str, Any]], optional): A dictionary that will be fed into:obj:
openai.ChatCompletion.create()
. If :obj:None
, :obj:SambaCloudAPIConfig().as_dict()
will be used. (default: :obj:None
) - api_key (Optional[str], optional): The API key for authenticating with the SambaNova service. (default: :obj:
None
) - url (Optional[str], optional): The url to the SambaNova service. Current support SambaVerse API: :obj:
"https://sambaverse.sambanova.ai/api/predict"
and SambaNova Cloud: :obj:"https://api.sambanova.ai/v1"
(default: :obj:https://api. sambanova.ai/v1
) - token_counter (Optional[BaseTokenCounter], optional): Token counter to use for the model. If not provided, :obj:
OpenAITokenCounter( ModelType.GPT_4O_MINI)
will be used. - timeout (Optional[float], optional): The timeout value in seconds for API calls. If not provided, will fall back to the MODEL_TIMEOUT environment variable or default to 180 seconds. (default: :obj:
None
)
init
token_counter
Returns:
BaseTokenCounter: The token counter following the model’s tokenization style.
check_model_config
_run
Runs SambaNova’s service.
Parameters:
- messages (List[OpenAIMessage]): Message list with the chat history in OpenAI API format.
Returns:
Union[ChatCompletion, Stream[ChatCompletionChunk]]:
ChatCompletion
in the non-stream mode, or
Stream[ChatCompletionChunk]
in the stream mode.
_run_streaming
Handles streaming inference with SambaNova’s API.
Parameters:
- messages (List[OpenAIMessage]): A list of messages representing the chat history in OpenAI API format.
Returns:
Stream[ChatCompletionChunk]: A generator yielding
ChatCompletionChunk
objects as they are received from the
API.
_run_non_streaming
Handles non-streaming inference with SambaNova’s API.
Parameters:
- messages (List[OpenAIMessage]): A list of messages representing the message in OpenAI API format.
Returns:
ChatCompletion: A ChatCompletion
object containing the complete
response from the API.
_sambaverse_to_openai_response
Converts SambaVerse API response into an OpenAI-compatible response.
Parameters:
- samba_response (Dict[str, Any]): A dictionary representing responses from the SambaVerse API.
Returns:
ChatCompletion: A ChatCompletion
object constructed from the
aggregated response data.
stream
Returns:
bool: Whether the model is in stream mode.