Pipeline for evolving prompts using the Evol-Instruct methodology.Supports custom templates defining evolution strategies and methods. The
pipeline leverages language models to iteratively refine prompts through
specified evolution strategies.Parameters:
templates (Type[EvolInstructTemplates]): Template class containing evolution strategy and method definitions. Must provide EVOL_METHODS and STRATEGY attributes. (default: :obj:EvolInstructTemplates)
agent (Optional[ChatAgent]): Chat agent instance for LLM interaction.
If: obj:None, initializes with a default ChatAgent. (default: :obj:None)
def _get_evolution_methods(self, method: Union[str, List[str]], num_generations: int = 2):
Get list of evolution methods based on input specification.Parameters:
method (Union[str, List[str]]): Specification for method selection. Can be: - Strategy name for methods from that strategy - Specific method name - List of method specifications
num_generations (int): Number of methods to return.
Generate iterative evolutions of a prompt with scoring.Parameters:
prompt (str): Seed prompt to evolve.
evolution_spec (Union[str, List[Union[str, List[str]]]]): Evolution method specification. If a list is provided and num_iterations is None, then num_iterations is set to the length of the list.
num_generations (int): Candidates to generate per iteration.
num_iterations (Optional[int]): Number of evolution iterations. Defaults to the length of evolution_spec.
keep_original (bool): Include original prompt in results.
scorer (Optional[BaseScorer]): Scoring model for candidate.
num_threads (int): Number of threads for parallel processing.
Returns:Dict[int, List[Dict[str, Any]]]: Evolution results per iteration,
where each candidate is represented as a dict with keys:
“instruction”, “method”, and “scores”.
Evolve a batch of prompts through iterative refinement.Parameters:
prompts (List[str]): Seed prompts to evolve.
evolution_spec (Union[str, List[Union[str, List[str]]]]): Evolution method specification. If a list is provided and num_iterations is None, then num_iterations is set to the length of the list.
num_generations (int): Candidates to generate per iteration.
num_iterations (Optional[int]): Number of evolution iterations. Defaults to the length of evolution_spec.
keep_original (bool): Include original prompts in results.
scorer (Optional[BaseScorer]): Scoring model for candidate.
num_chunks (int): Number of parallel processing chunks.
retry_limit (int): Max retries for failed generations.
retry_delay (float): Delay between retries in seconds.
num_threads (int): Number of threads for parallel processing.