Camel.datagen.evol instruct.evol instruct
EvolInstructPipeline
Pipeline for evolving prompts using the Evol-Instruct methodology.
Supports custom templates defining evolution strategies and methods. The pipeline leverages language models to iteratively refine prompts through specified evolution strategies.
Parameters:
- templates (Type[EvolInstructTemplates]): Template class containing evolution strategy and method definitions. Must provide
EVOL_METHODS
andSTRATEGY
attributes. (default: :obj:EvolInstructTemplates
) - agent (Optional[ChatAgent]): Chat agent instance for LLM interaction.
- If: obj:
None
, initializes with a default ChatAgent. (default: :obj:None
)
init
Initialize pipeline with templates and language model agent.
Parameters:
- templates (Type[EvolInstructTemplates]): Template class containing evolution strategy configurations. (default: :obj:
EvolInstructTemplates
) - agent (Optional[ChatAgent]): Preconfigured chat agent instance. Creates a default ChatAgent if not provided. (default: :obj:
None
)
_resolve_evolution_method
Resolve evolution method key to concrete implementation.
Parameters:
- method_key (str): Input method identifier. Can be: - Direct method key from templates.EVOL_METHODS - Strategy name from templates.STRATEGY keys
Returns:
str: Resolved method key from EVOL_METHODS
_get_evolution_methods
Get list of evolution methods based on input specification.
Parameters:
- method (Union[str, List[str]]): Specification for method selection. Can be: - Strategy name for methods from that strategy - Specific method name - List of method specifications
- num_generations (int): Number of methods to return.
Returns:
List[str]: List of resolved method names
_generate_single_evolution
Generate a single evolved prompt from a seed prompt.
Parameters:
- prompt (str): The seed prompt to evolve.
- method (str): The evolution method key to use.
- return_method (bool): If True, returns method along with prompt.
Returns:
Tuple[str, str]: Evolved prompt and method
_generate_multiple_evolutions
Generate multiple evolved versions of a prompt.
Parameters:
- prompt (str): Seed prompt to evolve.
- method (Union[str, List[str]]): Evolution method specification.
- num_generations (int): Candidates to generate per iteration.
- keep_original (bool): Whether to keep the original prompt.
- num_threads (int): Number of threads for parallel processing.
Returns:
List[Tuple[str, str]]: List of (evolved_prompt, method) pairs
_generate_iterative_evolutions
Generate iterative evolutions of a prompt with scoring.
Parameters:
- prompt (str): Seed prompt to evolve.
- evolution_spec (Union[str, List[Union[str, List[str]]]]): Evolution method specification. If a list is provided and num_iterations is None, then num_iterations is set to the length of the list.
- num_generations (int): Candidates to generate per iteration.
- num_iterations (Optional[int]): Number of evolution iterations. Defaults to the length of evolution_spec.
- keep_original (bool): Include original prompt in results.
- scorer (Optional[BaseScorer]): Scoring model for candidate.
- num_threads (int): Number of threads for parallel processing.
Returns:
Dict[int, List[Dict[str, Any]]]: Evolution results per iteration, where each candidate is represented as a dict with keys: “instruction”, “method”, and “scores”.
generate
Evolve a batch of prompts through iterative refinement.
Parameters:
- prompts (List[str]): Seed prompts to evolve.
- evolution_spec (Union[str, List[Union[str, List[str]]]]): Evolution method specification. If a list is provided and num_iterations is None, then num_iterations is set to the length of the list.
- num_generations (int): Candidates to generate per iteration.
- num_iterations (Optional[int]): Number of evolution iterations. Defaults to the length of evolution_spec.
- keep_original (bool): Include original prompts in results.
- scorer (Optional[BaseScorer]): Scoring model for candidate.
- num_chunks (int): Number of parallel processing chunks.
- retry_limit (int): Max retries for failed generations.
- retry_delay (float): Delay between retries in seconds.
- num_threads (int): Number of threads for parallel processing.
Returns:
List[Dict[int, List[Dict[str, Any]]]]: Evolution results.