apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-23 17:01:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
doc-to-lora/scripts/main_exp
History
51616 cddb1238ff self-gen data viewer
2025-10-10 08:09:46 +00:00
..
eval python api + remove generate_multi_lora + rename config path + test eval scripts 2025-10-01 13:11:54 +00:00
0-download_data.py python api + remove generate_multi_lora + rename config path + test eval scripts 2025-10-01 13:11:54 +00:00
1-train.sh python api + remove generate_multi_lora + rename config path + test eval scripts 2025-10-01 13:11:54 +00:00
gen_data.sh self-gen data viewer 2025-10-10 08:09:46 +00:00
gen_data_test.sh python api + remove generate_multi_lora + rename config path + test eval scripts 2025-10-01 13:11:54 +00:00
README.md python api + remove generate_multi_lora + rename config path + test eval scripts 2025-10-01 13:11:54 +00:00

README.md

D2L pipeline

Data

You can either download the generated data (recommended) or generate them by youself.

# download training data (recommended)
uv run bash scripts/main_exp/0-download_data.sh

Generating data from scratch can take very long if not parallelized across multiple gpus.

# generate training data (takes very long if not parallelized across multiple gpus)
# optional: use the command below for generating data from scratch
# uv run bash scripts/main_exp/gen_data.sh

Training

Simply run the training script.

# train
uv run bash scripts/main_exp/1-train.sh

Evaluation

All evaluation scripts for reproducing the main results in the paper are included in eval directory.