apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-23 17:01:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
doc-to-lora/configs
History
Rujikorn Charakorn ee9039295e
LoRA separate bias param + merger + toy ds (#15) 
* separate bias + head weight init

* smaller bias init + sum with 1/sqrt(combined_r) scaling + better  lora merge logic

* ok init

* chunked ctx magic num working

* configs

* no transpose + only sum aggregation

* fix cli

* cli

* 32-256 data

* example cli + smaller dataset

* toy ctx magic num ds size + quantize + remove head bias

* configs removed

* require lora_bias

* Merge remote-tracking branch 'origin/main' into new-arch
2025-08-29 11:55:24 +09:00
..
gemma-3-1b-it/toy_exp toy ctx nums and multi-lora training (#11) 2025-08-18 18:38:07 +09:00
medium_exp per-rank bias + skip interleave if cached+ add bnb 2025-08-09 15:20:37 +00:00
small_exp per-rank bias + skip interleave if cached+ add bnb 2025-08-09 15:20:37 +00:00
toy_exp LoRA separate bias param + merger + toy ds (#15) 2025-08-29 11:55:24 +09:00
context_numbers_10.yaml refactor after distillation (#6) 2025-07-30 18:38:51 +09:00
context_numbers_10_self_gen.yaml Refactor_and_improve_data (#7) 2025-08-04 20:52:37 +09:00
fw_qa_v2_level_0.yaml qa data exp 2025-06-26 15:30:26 +09:00
fw_qa_v2_level_1.yaml qa data exp 2025-06-26 15:30:26 +09:00
fw_qa_v2_level_2.yaml qa data exp 2025-06-26 15:30:26 +09:00
fw_qa_v2_level_3.yaml clean up + configs + scripts 2025-06-24 21:51:33 +09:00
pwc_tiny.yaml Add distillation training (#5) 2025-07-29 15:32:06 +09:00
qa_short_ctx.yaml small configs 2025-07-21 09:03:58 +00:00
qa_short_ctx_compact.yaml small configs 2025-07-21 09:03:58 +00:00
qa_short_ctx_self_gen.yaml tiny configs + fix dataset gen (remove repeated response + qa template for compact ds) + add ropes and drop eval ds 2025-06-30 16:10:52 +09:00
qa_short_ctx_self_gen_no_fw_qa.yaml Add distillation training (#5) 2025-07-29 15:32:06 +09:00
qa_short_ctx_self_gen_squad.yaml Add distillation training (#5) 2025-07-29 15:32:06 +09:00
squad.yaml transformers ver + squad only example 2025-06-20 21:38:26 +09:00
squad_compact.yaml small configs 2025-07-21 09:03:58 +00:00
squad_fw_qa_v2_level_3.yaml clean up + configs + scripts 2025-06-24 21:51:33 +09:00
squad_hotpot_qa.yaml squad_hotpot config 2025-06-19 23:08:14 +09:00