evotoolkit is an LLM-driven code evolution library that enables automated generation and optimization of code through evolutionary algorithms guided by Large Language Models. The library provides a unified framework for tackling diverse optimization tasks, from generating high-performance hardware kernels to discovering adversarial attack algorithms.
The library's primary use case is Ascend C operator generation, where it implements a sophisticated multi-agent system that transforms Python reference implementations into optimized Ascend C operators through signature analysis, multi-round design planning, knowledge retrieval, and iterative code generation. Beyond CANN operators, evotoolkit supports evolutionary optimization for CUDA kernels, prompt engineering, scientific symbolic regression, and adversarial attack discovery.
For installation instructions, see Installation and Dependencies. For a minimal usage example, see Quick Start. For detailed information about CANN operator generation, see CANN Operator Generation.
Sources: pyproject.toml6-8 src/evotoolkit/task/cann_init/__init__.py4-8
evotoolkit is structured around a core abstraction layer that unifies evaluation and task configuration, with domain-specific implementations for different optimization problems. The architecture separates evolution orchestration from task-specific logic, enabling extensibility through well-defined interfaces.
The BaseTask abstract class (src/evotoolkit/core/base_task.py18-132) provides the universal interface that all optimization tasks must implement. Task implementations inherit from BaseTask and provide domain-specific evaluation logic. The evotoolkit.solve() function orchestrates the evolution process by interfacing with LLM providers and applying mutation/crossover operations to generate and evaluate candidate solutions.
Sources: src/evotoolkit/core/base_task.py1-132 pyproject.toml42-85 High-Level Diagram 1
All optimization tasks in evotoolkit inherit from the BaseTask abstract class, which unifies task evaluation and configuration into a single interface. This design eliminates the need for separate evaluator and config objects, simplifying the API while maintaining flexibility.
| Method | Purpose | Return Type |
|---|---|---|
evaluate_code(candidate_code) | Evaluate a code string | EvaluationResult |
evaluate_solution(solution) | Evaluate a Solution object with metadata | EvaluationResult |
get_base_task_description() | Get task description for prompt generation | str |
make_init_sol_wo_other_info() | Create initial solution | Solution |
get_task_type() | Get task type identifier (e.g., 'Python', 'Cuda') | str |
get_task_info() | Get task metadata dictionary | dict |
The evaluate_code() method (src/evotoolkit/core/base_task.py52-65) provides a simple interface for tasks that only need a code string. For more complex tasks requiring additional metadata (such as tiling configurations, block dimensions, or compilation flags), the evaluate_solution() method (src/evotoolkit/core/base_task.py67-86) accepts a Solution object that encapsulates both code and supplementary information in its other_info dictionary.
Sources: src/evotoolkit/core/base_task.py18-132
For detailed information on the Solution and EvaluationResult data structures, see Solution and Evaluation. For guidance on creating custom tasks, see Creating Custom Tasks.
evotoolkit provides five domain-specific task implementations, each targeting a different optimization problem. The library uses optional dependency groups to manage task-specific requirements.
| Task | Module | Purpose | Dependency Group | Key Features |
|---|---|---|---|---|
| CANNInitTask | evotoolkit.task.cann_init | Generate Ascend C operators from Python | (built-in) | Multi-agent planning, knowledge retrieval, template system |
| AdversarialAttackTask | evotoolkit.task.adversarial_attack | Evolve adversarial attack algorithms | adversarial_attack | Black-box attacks, robustness evaluation, transfer attacks |
| CUDATask | evotoolkit.task.cuda_engineering | Optimize CUDA kernels | cuda_engineering | Kernel compilation, performance measurement |
| PromptTask | evotoolkit.task.prompt_engineering | Optimize LLM prompts | prompt_engineering | LLM-based evaluation |
| ScientificRegressionTask | evotoolkit.task.scientific_regression | Symbolic regression for scientific data | scientific_regression | Parameter optimization, CSV datasets |
The CANNInitTask (src/evotoolkit/task/cann_init/cann_init_task.py) is the most sophisticated task implementation, featuring a four-phase pipeline that transforms Python reference implementations into optimized Ascend C operators:
The CANN system includes specialized components:
Sources: src/evotoolkit/task/cann_init/__init__.py1-54 pyproject.toml42-85 examples/cann_init/README.md16-22 High-Level Diagram 2
For comprehensive documentation of the CANN system, see CANN Operator Generation. For other task types, see Other Task Types.
evotoolkit tasks integrate with the library's evolutionary optimization framework through method-specific interfaces. The framework handles population management, mutation, crossover, and selection operations while delegating evaluation to task-specific logic.
The evotoolkit.solve() function accepts a BaseTask instance and a method-specific configuration object. It orchestrates the evolution loop, using the task's make_init_sol_wo_other_info() method to generate initial candidates, LLM-driven mutation/crossover to explore the solution space, and the task's evaluate_solution() method to assess fitness.
Sources: src/evotoolkit/core/base_task.py18-132 High-Level Diagram 1
For details on LLM integration, see LLM Integration. For examples of using the evolution framework, see Examples and Tutorials.
The following diagram illustrates how data flows through evotoolkit, from initial task definition to evaluated solutions:
Solution (src/evotoolkit/core/solution.py):
sol_string: The main code to be evaluated (e.g., kernel source code)other_info: Task-specific metadata dictionary (e.g., {"block_dim": [16, 16]} for CUDA tasks)EvaluationResult (src/evotoolkit/core/solution.py):
valid: Boolean indicating whether the solution compiled/ran successfullyscore: Numeric fitness score (higher is better)additional_info: Detailed metrics (e.g., compilation time, accuracy, performance)For CANN tasks specifically, the other_info dictionary can be wrapped in a CANNSolutionConfig typed object (src/evotoolkit/task/cann_init/data_structures.py) that provides structured access to CANN-specific parameters like project_path, compile_only, and skip_correctness.
Sources: src/evotoolkit/core/base_task.py27-47 src/evotoolkit/core/solution.py High-Level Diagram 5
The evotoolkit repository is organized into the following key directories:
evotoolkit/
├── src/evotoolkit/
│ ├── core/ # Core abstractions (BaseTask, Solution, etc.)
│ ├── task/ # Task implementations
│ │ ├── cann_init/ # CANN operator generation (most complex)
│ │ ├── adversarial_attack/ # Adversarial attack evolution
│ │ ├── cuda_engineering/ # CUDA kernel optimization
│ │ ├── prompt_engineering/ # Prompt optimization
│ │ └── scientific_regression/ # Symbolic regression
│ ├── method/ # Evolution method implementations
│ └── utils/ # Shared utilities
├── examples/ # Usage examples and tutorials
│ ├── cann_init/ # CANN operator examples (ReLU, Softmax, SDPA)
│ └── adversarial_attack/ # Adversarial attack examples
└── tests/ # Test suites
The CANN operator generation system (src/evotoolkit/task/cann_init/) contains several subsystems:
agent/: Multi-agent collaboration (JointBranch, prompt generators)knowledge_base/: API/operator indexing and retrievaltemplates/: Code template systembackend/: Compilation and execution utilitiesSources: src/evotoolkit/task/cann_init/__init__.py4-24 examples/cann_init/README.md1-68
Sources: All sections
Refresh this wiki
This wiki was recently refreshed. Please wait 7 days to refresh again.