doc-to-lora/configs/toy_exp
Rujikorn Charakorn ee9039295e
LoRA separate bias param + merger + toy ds (#15)
* separate bias + head weight init

* smaller bias init + sum with 1/sqrt(combined_r) scaling + better  lora merge logic

* ok init

* chunked ctx magic num working

* configs

* no transpose + only sum aggregation

* fix cli

* cli

* 32-256 data

* example cli + smaller dataset

* toy ctx magic num ds size + quantize + remove head bias

* configs removed

* require lora_bias

* Merge remote-tracking branch 'origin/main' into new-arch
2025-08-29 11:55:24 +09:00
..
ctx_magic_number_32_256.yaml LoRA separate bias param + merger + toy ds (#15) 2025-08-29 11:55:24 +09:00