26 commits

Author	SHA1	Message	Date
51616	2ed418bc22	vlm ctx-encoder + slrum scripts + qwen self-gen	2025-10-27 06:20:26 +00:00
51616	a074fd3305	self-gen data misaligned fixed + niah reproducible + vllm-tokenizers version combo	2025-10-08 05:17:50 +00:00
51616	3b798c670c	python api + remove generate_multi_lora + rename config path + test eval scripts	2025-10-01 13:11:54 +00:00
51616	b6679ba755	iclr cleanup	2025-09-30 14:53:21 +00:00
51616	86ba2bc4a5	self-gen data import	2025-09-13 23:52:24 +09:00
51616	650395874f	toy exp w/ self-gen + use double bos for eval (following vllm 0.8.4)	2025-09-07 17:16:20 +09:00
51616	a269aff51d	fix empty q_list	2025-08-29 04:50:18 +00:00
51616	9d82014dc2	q gen can take ds_names + self_gen summ/struct/cot prob + ctx qa v3 data	2025-08-28 17:42:43 +00:00
Rujikorn Charakorn	c5e9bc769d	toy ctx nums and multi-lora training (#11) ... * multi-lora trainable toy number repeat dataset * per rank bias init * remove head_bias +simplify merge + skip perplexities metric * ctx_numbers train example * self-gen ctx numbers example	2025-08-18 18:38:07 +09:00
Rujikorn Charakorn	891c0bd256	Refactor_and_improve_data (#7) ... * faster slice * add facts + ctx_qa * new configs * new scripts * intx_sft.py to train.py * add kaggle for downloading facts * max_new_tokens cli for eval * generate negative_nq * scripts + configs * default vals * max_val_samples_per_ds=500 * more efficient layer-to-layer ctx encoder * use_per_ctx_average_loss * faster processing * small exp distill * scripts * more robust watcher * per-module l1_norm avg * per-ctx average loss * clear_gpu	2025-08-04 20:52:37 +09:00
Rujikorn Charakorn	610750e6fb	refactor after distillation (#6)	2025-07-30 18:38:51 +09:00
ShinnosukeUesakaSakana	d99fa8732b	Add distillation training (#5) ... - work with vllm self-gen data (everything is tokenized during self-gen) - fix chat template! - not use liger kernel for distillation (for both training and eval)	2025-07-29 15:32:06 +09:00
51616	c39e44cc90	self-gen can now take --temp and --closed_qa_prob	2025-07-10 02:22:11 +00:00
51616	2bce7d164d	new self-gen exp (not using self-gen for qa ds with perfect label + small exp + remove "refusal" prompt)	2025-07-09 16:59:25 +00:00
51616	c9e958f963	self-gen w/ closed_qa format + perceiver scripts + no strip response	2025-07-08 14:21:43 +00:00
51616	1a03b01a52	small adjustment data gen (remove empty)	2025-07-03 11:02:46 +00:00
51616	f8a81a1eea	self_gen temp fix + lv1 temp varying configs	2025-07-01 15:30:28 +00:00
51616	92ba1ca76c	tiny data def	2025-07-01 05:52:47 +00:00
51616	4d544e538d	temp naming	2025-06-30 16:38:10 +09:00
51616	28eb85a2f3	tiny configs + fix dataset gen (remove repeated response + qa template for compact ds) + add ropes and drop eval ds	2025-06-30 16:10:52 +09:00
51616	e4cb1ffa55	add todo for self-gen logits	2025-06-27 17:43:37 +09:00
51616	1457981348	compact ds + self_gen + update rope + drop multi answer eval	2025-06-26 16:04:33 +00:00
51616	4cedfc2514	hf data	2025-06-26 16:23:14 +09:00
51616	7a889d0d7e	self-gen naming	2025-06-03 15:33:53 +00:00
51616	5aa83b96e2	self gen data for all ds both train + val	2025-06-03 14:30:38 +00:00
51616	b1f92663a7	data gen scripts	2025-05-30 03:11:03 +00:00