apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-23 17:01:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
doc-to-lora/scripts/main_exp
History
51616 8093817c34 icml cleaned
2026-02-13 06:02:29 +00:00
..
eval icml cleaned 2026-02-13 06:02:29 +00:00
0-download_data.py icml cleaned 2026-02-13 06:02:29 +00:00
1-train.sh mistral config + script 2025-10-13 07:45:47 +00:00
2-train-chunk.sh mistral config + script 2025-10-13 07:45:47 +00:00
gen_data.sh self-gen data viewer 2025-10-10 08:09:46 +00:00
README.md python api + remove generate_multi_lora + rename config path + test eval scripts 2025-10-01 13:11:54 +00:00
train-cross-enc-chunk-slurm.sh vlm ctx-encoder + slrum scripts + qwen self-gen 2025-10-27 06:20:26 +00:00
train-cross-enc-slurm.sh vlm ctx-encoder + slrum scripts + qwen self-gen 2025-10-27 06:20:26 +00:00
train_no_qa.sh icml cleaned 2026-02-13 06:02:29 +00:00

README.md

D2L pipeline

Data

You can either download the generated data (recommended) or generate them by youself.

# download training data (recommended)
uv run bash scripts/main_exp/0-download_data.sh

Generating data from scratch can take very long if not parallelized across multiple gpus.

# generate training data (takes very long if not parallelized across multiple gpus)
# optional: use the command below for generating data from scratch
# uv run bash scripts/main_exp/gen_data.sh

Training

Simply run the training script.

# train
uv run bash scripts/main_exp/1-train.sh

Evaluation

All evaluation scripts for reproducing the main results in the paper are included in eval directory.