> ## Documentation Index
> Fetch the complete documentation index at: https://docs.camel-ai.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Camel.benchmarks.apibank

<a id="camel.benchmarks.apibank" />

<a id="camel.benchmarks.apibank.process_messages" />

## process\_messages

```python theme={"system"}
def process_messages(chat_history: List[Dict[str, Any]], prompt: str):
```

Processes chat history into a structured format for further use.

**Parameters:**

* **chat\_history** (List\[Dict\[str, Any]): A list of dictionaries representing the chat history.
* **prompt** (str): A prompt to be set as the system message.

**Returns:**

List\[Dict\[str, str]]: A list of dictionaries representing
the processed messages, where each dictionary has:

* 'role': The role of the message ('system', 'user', or 'assistant').
* 'content': The content of the message, including formatted
  API responses when applicable.

<a id="camel.benchmarks.apibank.APIBankBenchmark" />

## APIBankBenchmark

```python theme={"system"}
class APIBankBenchmark(BaseBenchmark):
```

API-Bank Benchmark adapted from `API-Bank:
A Comprehensive Benchmark for Tool-Augmented LLMs`
`<https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/api-bank>`.

**Parameters:**

* **save\_to** (str): The file to save the results.
* **processes** (int, optional): The number of processes to use. (default: :obj:`1`)

<a id="camel.benchmarks.apibank.APIBankBenchmark.__init__" />

### **init**

```python theme={"system"}
def __init__(self, save_to: str, processes: int = 1):
```

Initialize the APIBank benchmark.

**Parameters:**

* **save\_to** (str): The file to save the results.
* **processes** (int, optional): The number of processes to use for parallel processing. (default: :obj:`1`)

<a id="camel.benchmarks.apibank.APIBankBenchmark.download" />

### download

```python theme={"system"}
def download(self):
```

Download APIBank dataset and code from Github.

<a id="camel.benchmarks.apibank.APIBankBenchmark.load" />

### load

```python theme={"system"}
def load(self, level: str, force_download: bool = False):
```

Load the APIBank Benchmark dataset.

**Parameters:**

* **level** (str): Level to run benchmark on.
* **force\_download** (bool, optional): Whether to force download the data.

<a id="camel.benchmarks.apibank.APIBankBenchmark.run" />

### run

```python theme={"system"}
def run(
    self,
    agent: ChatAgent,
    level: Literal['level-1', 'level-2'],
    api_test_enabled = True,
    randomize: bool = False,
    subset: Optional[int] = None
):
```

Run the benchmark.

**Parameters:**

* **agent** (ChatAgent): The agent to run the benchmark.
* **level** (`Literal['level-1', 'level-2']`): The level to run the benchmark on.
* **randomize** (bool, optional): Whether to randomize the data.
* **api\_test\_enabled** (bool): Whether to test API calling (`True`) or response (`False`) (default: :obj:`False`)
* **subset** (Optional\[int], optional): The subset of data to run. (default: :obj:`None`)

**Returns:**

Dict\[str, Any]: The results of the benchmark.

<a id="camel.benchmarks.apibank.agent_call" />

## agent\_call

```python theme={"system"}
def agent_call(messages: List[Dict], agent: ChatAgent):
```

Add messages to agent memory and get response.

<a id="camel.benchmarks.apibank.calculate_rouge_l_score" />

## calculate\_rouge\_l\_score

```python theme={"system"}
def calculate_rouge_l_score(reference, hypothesis):
```

Calculate rouge l score between hypothesis and reference.

<a id="camel.benchmarks.apibank.get_api_call" />

## get\_api\_call

```python theme={"system"}
def get_api_call(model_output):
```

Parse api call from model output.

<a id="camel.benchmarks.apibank.APIBankSample" />

## APIBankSample

```python theme={"system"}
class APIBankSample:
```

APIBank sample used to load the datasets.

<a id="camel.benchmarks.apibank.APIBankSample.__init__" />

### **init**

```python theme={"system"}
def __init__(
    self,
    chat_history,
    apis,
    ground_truth
):
```

<a id="camel.benchmarks.apibank.APIBankSample.__repr__" />

### **repr**

```python theme={"system"}
def __repr__(self):
```

<a id="camel.benchmarks.apibank.APIBankSample.from_chat_history" />

### from\_chat\_history

```python theme={"system"}
def from_chat_history(cls, chat_history):
```

<a id="camel.benchmarks.apibank.Evaluator" />

## Evaluator

```python theme={"system"}
class Evaluator:
```

Evaluator for APIBank benchmark.

<a id="camel.benchmarks.apibank.Evaluator.__init__" />

### **init**

```python theme={"system"}
def __init__(self, samples: List[APIBankSample]):
```

<a id="camel.benchmarks.apibank.Evaluator.get_all_sample_ids" />

### get\_all\_sample\_ids

```python theme={"system"}
def get_all_sample_ids(self):
```

<a id="camel.benchmarks.apibank.Evaluator.get_api_description" />

### get\_api\_description

```python theme={"system"}
def get_api_description(self, api_name):
```

<a id="camel.benchmarks.apibank.Evaluator.get_model_input" />

### get\_model\_input

```python theme={"system"}
def get_model_input(self, sample_id: int):
```

<a id="camel.benchmarks.apibank.Evaluator.evaluate" />

### evaluate

```python theme={"system"}
def evaluate(self, sample_id, model_output):
```