5 commits

Author	SHA1	Message	Date
51616	8093817c34	icml cleaned	2026-02-13 06:02:29 +00:00
51616	a4f4dfd522	magic num eval upto 128k	2025-09-06 02:21:15 +09:00
51616	3505b006ad	toy ctx magic num eval upto 32k + remove configs	2025-09-01 05:57:06 +00:00
Rujikorn Charakorn	ee9039295e	LoRA separate bias param + merger + toy ds (#15) ... * separate bias + head weight init * smaller bias init + sum with 1/sqrt(combined_r) scaling + better lora merge logic * ok init * chunked ctx magic num working * configs * no transpose + only sum aggregation * fix cli * cli * 32-256 data * example cli + smaller dataset * toy ctx magic num ds size + quantize + remove head bias * configs removed * require lora_bias * Merge remote-tracking branch 'origin/main' into new-arch	2025-08-29 11:55:24 +09:00
Rujikorn Charakorn	c5e9bc769d	toy ctx nums and multi-lora training (#11) ... * multi-lora trainable toy number repeat dataset * per rank bias init * remove head_bias +simplify merge + skip perplexities metric * ctx_numbers train example * self-gen ctx numbers example	2025-08-18 18:38:07 +09:00