Skip to main content

​
AihubMixConfig

class AihubMixConfig(BaseConfig):
Defines the parameters for generating chat completions using the AihubMix API. Parameters:
  • temperature (float, optional): Sampling temperature to use, between :obj:0 and :obj:2. Higher values make the output more random, while lower values make it more focused and deterministic. (default: :obj:0.8)
  • max_tokens (int, optional): The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length. (default: :obj:1024)
  • top_p (float, optional): An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So :obj:0.1 means only the tokens comprising the top 10% probability mass are considered. (default: :obj:1)
  • frequency_penalty (float, optional): Number between :obj:-2.0 and :obj:2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim. (default: :obj:0)
  • presence_penalty (float, optional): Number between :obj:-2.0 and :obj:2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics. (default: :obj:0)
  • stream (bool, optional): If True, partial message deltas will be sent as data-only server-sent events as they become available. (default: :obj:False)
  • web_search_options (dict, optional): Search model’s web search options, only supported by specific search models. (default: :obj:None)
  • tools (list[FunctionTool], optional): A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.
  • tool_choice (Union[dict[str, str], str], optional): Controls which (if any) tool is called by the model. :obj:"none" means the model will not call any tool and instead generates a message. :obj:"auto" means the model can pick between generating a message or calling one or more tools. :obj:"required" means the model must call one or more tools. Specifying a particular tool via {"type": "function", "function": {"name": "my_function"}} forces the model to call that tool. :obj:"none" is the default when no tools are present. :obj:"auto" is the default if tools are present.
  • parallel_tool_calls (bool, optional): A parameter specifying whether the model should call tools in parallel or not. (default: :obj:None)
  • extra_headers: Optional[Dict[str, str]]: Extra headers to use for the model. (default: :obj:None)