From e41f9342cdeb9d5f8efafe61d7da8e1376715e69 Mon Sep 17 00:00:00 2001 From: Cyzus Chi Date: Mon, 28 Oct 2024 17:14:26 +0800 Subject: [PATCH] update readme --- metagpt/ext/sela/README.md | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/metagpt/ext/sela/README.md b/metagpt/ext/sela/README.md index c8a1adea8..c8df4eeba 100644 --- a/metagpt/ext/sela/README.md +++ b/metagpt/ext/sela/README.md @@ -7,7 +7,7 @@ ## 1. Data Preparation - Download Datasets：https://deepwisdom.feishu.cn/drive/folder/RVyofv9cvlvtxKdddt2cyn3BnTc?from=from_copylink - Download and prepare datasets from scratch: ``` -cd expo/data +cd data python dataset.py --save_analysis_pool python hf_data.py --save_analysis_pool ``` @@ -60,16 +60,36 @@ #### Setup #### Run -- `python run_experiment.py --exp_mode mcts --task titanic --rollouts 10` - -If the dataset has reg metric, remember to use `--low_is_better`: - -- `python run_experiment.py --exp_mode mcts --task house-prices --rollouts 10 --low_is_better` +- Examples + ``` + python run_experiment.py --exp_mode mcts --task titanic --rollouts 10 + python run_experiment.py --exp_mode mcts --task house-prices --rollouts 10 --low_is_better + ``` -In addition to the generated insights, include the fixed insights saved in `expo/insights/fixed_insights.json` -- `--use_fixed_insights` +- `--rollouts` - The number of rollouts + +- `--use_fixed_insights` - In addition to the generated insights, include the fixed insights saved in `expo/insights/fixed_insights.json` +- `--low_is_better` - If the dataset has reg metric, remember to use `--low_is_better` + +- `--from_scratch` - Do not use pre-processed insight pool, generate new insight pool based on dataset before running MCTS, facilitating subsequent tuning to propose search space prompts + +- `--role_timeout` - The timeout for the role + - This feature limits the duration of a single simulation, making the experiment duration more controllable (for example, if you do ten rollouts and set role_timeout to 1,000, the experiment will stop at the latest after 10,000s) + + +- `--max_depth` - The maximum depth of MCTS, default is 4 (nodes at this depth directly return the previous simulation result without further expansion) + +- `--load_tree` - If MCTS was interrupted due to certain reasons but had already run multiple rollouts, you can use `--load_tree`. + - For example: + ``` + python run_experiment.py --exp_mode mcts --task titanic --rollouts 10 + ``` + - If this was interrupted after running three rollouts, you can use `--load_tree`: + ``` + python run_experiment.py --exp_mode mcts --task titanic --rollouts 7 --load_tree + ``` #### Ablation Study @@ -112,7 +132,7 @@ ## 5. Baselines ### AIDE #### Setup - +The version of AIDE we use is dated September 30, 2024 ``` git clone https://github.com/WecoAI/aideml.git git checkout 77953247ea0a5dc1bd502dd10939dd6d7fdcc5cc