51616
696b45c51b
icml rebuttal
2026-06-15 04:31:47 +00:00
51616
8093817c34
icml cleaned
2026-02-13 06:02:29 +00:00
51616
d6c7e3ab03
fix tracker + recall/precision logging + add generative adapter
2025-11-10 16:54:25 +09:00
51616
b6679ba755
iclr cleanup
2025-09-30 14:53:21 +00:00
51616
ad18d38374
iclr 2026 (3)
2025-09-28 15:46:09 +00:00
51616
76bf6981ba
scaling gen lora + q gen round for cd + gsm8k
2025-09-18 16:02:22 +09:00
51616
8a6c2cfd42
add t2l
2025-09-15 04:04:32 +09:00
51616
3105fac6dc
remove num_gen_q
2025-09-15 01:37:28 +09:00
51616
0e46a7c568
add llmlingua + different q model for cd
2025-09-13 23:52:07 +09:00
51616
0e05644d6a
iterative mode for hypernet
2025-09-12 19:05:05 +09:00
51616
7d95db6672
context distillation eval + timer
2025-09-12 16:34:03 +09:00
Rujikorn Charakorn
ee9039295e
LoRA separate bias param + merger + toy ds ( #15 )
...
* separate bias + head weight init
* smaller bias init + sum with 1/sqrt(combined_r) scaling + better lora merge logic
* ok init
* chunked ctx magic num working
* configs
* no transpose + only sum aggregation
* fix cli
* cli
* 32-256 data
* example cli + smaller dataset
* toy ctx magic num ds size + quantize + remove head bias
* configs removed
* require lora_bias
* Merge remote-tracking branch 'origin/main' into new-arch
2025-08-29 11:55:24 +09:00
Rujikorn Charakorn
ec955a935d
chunked ctx eval ( #9 )
...
* got burned by floating point precision again :)))))))))))
* split-ctx working eval (batch_size=1)
* batched lora aggregation (sum + mean) eval + finegrain eval len bins
* fix ctx_ids assert
2025-08-11 18:44:26 +09:00
Rujikorn Charakorn
891c0bd256
Refactor_and_improve_data ( #7 )
...
* faster slice
* add facts + ctx_qa
* new configs
* new scripts
* intx_sft.py to train.py
* add kaggle for downloading facts
* max_new_tokens cli for eval
* generate negative_nq
* scripts + configs
* default vals
* max_val_samples_per_ds=500
* more efficient layer-to-layer ctx encoder
* use_per_ctx_average_loss
* faster processing
* small exp distill
* scripts
* more robust watcher
* per-module l1_norm avg
* per-ctx average loss
* clear_gpu
2025-08-04 20:52:37 +09:00
Rujikorn Charakorn
610750e6fb
refactor after distillation ( #6 )
2025-07-30 18:38:51 +09:00
51616
375c50757c
eval generative only + scripts
2025-06-30 16:41:26 +09:00
51616
dcda29bae6
watcher + partial refactor eval + debug train w/ pwc and hotpot
2025-06-02 09:02:20 +00:00
