apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-23 17:01:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
doc-to-lora/data
History
Rujikorn Charakorn ee9039295e
LoRA separate bias param + merger + toy ds (#15) 
* separate bias + head weight init

* smaller bias init + sum with 1/sqrt(combined_r) scaling + better  lora merge logic

* ok init

* chunked ctx magic num working

* configs

* no transpose + only sum aggregation

* fix cli

* cli

* 32-256 data

* example cli + smaller dataset

* toy ctx magic num ds size + quantize + remove head bias

* configs removed

* require lora_bias

* Merge remote-tracking branch 'origin/main' into new-arch
2025-08-29 11:55:24 +09:00
..
eval Refactor_and_improve_data (#7) 2025-08-04 20:52:37 +09:00
build_drop_compact.py compact script update 2025-07-21 09:04:53 +00:00
build_pwc_compact.py compact script update 2025-07-21 09:04:53 +00:00
build_ropes_compact.py fix compact builder (import) 2025-08-11 14:59:03 +09:00
build_squad_compact.py fix compact builder (import) 2025-08-11 14:59:03 +09:00
create_test_data.py Store top k logits tests and scripts (#2) (to be refactored + integrated with data pipeline) 2025-07-09 13:18:15 +09:00
download_facts_grounding.py Refactor_and_improve_data (#7) 2025-08-04 20:52:37 +09:00
download_fineweb_edu.py data scripts + readme 2025-06-06 09:48:42 +00:00
download_generated_fineweb_qa.py download generated fw qa 2025-06-26 16:24:18 +09:00
download_self_gen_qa.py download self gen qa 2025-06-26 16:30:04 +00:00
download_self_gen_qa_logprobs.py Add distillation training (#5) 2025-07-29 15:32:06 +09:00
generate_ctx_kv.py toy ctx nums and multi-lora training (#11) 2025-08-18 18:38:07 +09:00
generate_ctx_magic_number.py LoRA separate bias param + merger + toy ds (#15) 2025-08-29 11:55:24 +09:00
generate_ctx_numbers.py toy ctx nums and multi-lora training (#11) 2025-08-18 18:38:07 +09:00
generate_fav.py install + data folder for fav num 2025-06-19 04:21:52 +09:00
generate_fav_num_big.py data gen scripts 2025-05-30 03:11:03 +00:00
generate_fw_edu_augment_vllm.py data scripts + readme 2025-06-06 09:48:42 +00:00
generate_fw_edu_qa_v2.py small adjustment data gen (remove empty) 2025-07-03 11:02:46 +00:00
generate_fw_edu_qa_v2_repeat.py small adjustment data gen (remove empty) 2025-07-03 11:02:46 +00:00
generate_fw_edu_qa_v3.py q gen can take ds_names + self_gen summ/struct/cot prob + ctx qa v3 data 2025-08-28 17:42:43 +00:00
generate_fw_edu_qa_vllm.py data gen scripts 2025-05-30 03:11:03 +00:00
generate_negative_natural_questions.py q gen can take ds_names + self_gen summ/struct/cot prob + ctx qa v3 data 2025-08-28 17:42:43 +00:00
generate_pretrain_from_fw_qa.py data scripts + readme 2025-06-06 09:48:42 +00:00
prepare_distillation_dataset.py Add distillation training (#5) 2025-07-29 15:32:06 +09:00
q_generation_prompts.py Improved training data generation (#14) 2025-08-28 21:03:53 +09:00
self_generate_qa.py q gen can take ds_names + self_gen summ/struct/cot prob + ctx qa v3 data 2025-08-28 17:42:43 +00:00
store_top_logits.py Store top k logits tests and scripts (#2) (to be refactored + integrated with data pipeline) 2025-07-09 13:18:15 +09:00