apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-23 17:01:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
doc-to-lora/src/ctx_to_lora
History
Rujikorn Charakorn ee9039295e
LoRA separate bias param + merger + toy ds (#15) 
* separate bias + head weight init

* smaller bias init + sum with 1/sqrt(combined_r) scaling + better  lora merge logic

* ok init

* chunked ctx magic num working

* configs

* no transpose + only sum aggregation

* fix cli

* cli

* 32-256 data

* example cli + smaller dataset

* toy ctx magic num ds size + quantize + remove head bias

* configs removed

* require lora_bias

* Merge remote-tracking branch 'origin/main' into new-arch
2025-08-29 11:55:24 +09:00
..
data LoRA separate bias param + merger + toy ds (#15) 2025-08-29 11:55:24 +09:00
modeling LoRA separate bias param + merger + toy ds (#15) 2025-08-29 11:55:24 +09:00
__init__.py monkey patch liger kernel for gemma-3-1b-it 2025-06-19 04:44:24 +09:00
configs.py LoRA separate bias param + merger + toy ds (#15) 2025-08-29 11:55:24 +09:00
eval_utils.py LoRA separate bias param + merger + toy ds (#15) 2025-08-29 11:55:24 +09:00
hooks.py rearrange + move to uv (#1) 2025-05-27 21:18:15 +09:00
metrics.py toy ctx nums and multi-lora training (#11) 2025-08-18 18:38:07 +09:00
model_loading.py LoRA separate bias param + merger + toy ds (#15) 2025-08-29 11:55:24 +09:00
monkey_patch.py monkey patch liger kernel for gemma-3-1b-it 2025-06-19 04:44:24 +09:00
pooling.py rearrange + move to uv (#1) 2025-05-27 21:18:15 +09:00
trainer.py toy ctx nums and multi-lora training (#11) 2025-08-18 18:38:07 +09:00
utils.py fixing non-packed collator + packing working and trainable 2025-06-24 17:35:50 +09:00
viz_utils.py llm comparator 2025-06-17 11:35:04 +00:00