apunkt/MetaGPT
1
0
Fork 0
mirror of https://github.com/FoundationAgents/MetaGPT.git synced 2026-05-24 14:15:17 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update

Browse source
This commit is contained in:
didi 2024-10-22 11:08:08 +08:00
parent fcc5e19160
commit 5aa62b76ce
13 changed files with 71 additions and 105 deletions
Download patch file Download diff file Expand all files Collapse all files

6
examples/aflow/optimize.py
View file

@ -4,7 +4,7 @@
# @Desc : Entrance of AFlow.
from metagpt.ext.aflow.scripts.optimizer import Optimizer
from metagpt.ext.aflow.scripts.evaluator import DatasetType, QuestionType, OptimizerType
from metagpt.ext.aflow.scripts.optimizer import DatasetType, QuestionType, OptimizerType
from metagpt.ext.aflow.data.download_data import download
from metagpt.configs.models_config import ModelsConfig
from typing import Literal
@ -15,13 +15,13 @@ from typing import Literal
# OptimizerType = Literal["Graph", "Test"]
# When you fisrt use, please download the datasets and initial rounds; If you want to get a look of the results, please download the results.
# download(["datasets", "results", "initial_rounds"])
download(["datasets", "initial_rounds"])
# Crucial Parameters
dataset: DatasetType = "GSM8K" # Ensure the type is consistent with DatasetType
sample: int = 4 # Sample Count, which means how many workflows will be resampled from generated workflows
question_type: QuestionType = "code" # Ensure the type is consistent with QuestionType
optimized_path: str = "examples/aflow/scripts/optimized" # Optimized Result Save Path
optimized_path: str = "metagpt/ext/aflow/scripts/optimized" # Optimized Result Save Path
initial_round: int = 1 # Corrected the case from Initial_round to initial_round
max_rounds: int = 20
check_convergence: bool = True

8
examples/aflow/readme.md
View file

@ -25,7 +25,7 @@ ## Datasets
## Quick Start
1. Configure your search in `optimize.py`:
- Open `examples/aflow/scripts/optimize.py`
- Open `metagpt/ext/aflow/scripts/optimize.py`
- Set the following parameters:
```python
dataset = "HumanEval" # Choose from: "HumanEval", "MBPP", "GSM8K", "MATH", "HotpotQA", "DROP" or your custom dataset name
@ -37,19 +37,19 @@ ## Quick Start
max_rounds = 20 # Maximum number of optimization rounds
```
- Adjust these parameters according to your specific requirements and dataset
2. Set up parameters in `config/config2.yaml` (see `examples/aflow/config2.example.yaml` for reference)
2. Set up parameters in `config/config2.yaml` (see `metagpt/ext/aflow/config2.example.yaml` for reference)
3. Set the operator you want to use in `optimize.py` and in `xxxx`
4. Download the init round of six datasets and put them in `xxxxxx`
5. Add your custom dataset and corresponding evaluation function:
- Create a new Python file in the `examples/aflow/benchmark/` directory, named `{custom_dataset_name}.py`
- Create a new Python file in the `metagpt/ext/aflow/benchmark/` directory, named `{custom_dataset_name}.py`
- Implement the following key functions in this new file:
- `load_data`: for loading the dataset
- `evaluate_problem`: for evaluating a single problem solution
- `evaluate_all_problems`: for evaluating all problems
- `save_results_to_csv`: for saving evaluation results
- `optimize_{custom_dataset_name}_evaluation`: main evaluation function that integrates the above functionalities
- Add your custom dataset name and config val_list in `examples/aflow/scripts/evaluator.py`
- Add your custom dataset name and config val_list in `metagpt/ext/aflow/scripts/evaluator.py`
## License