70 commits

Author	SHA1	Message	Date
51616	2bce7d164d	new self-gen exp (not using self-gen for qa ds with perfect label + small exp + remove "refusal" prompt)	2025-07-09 16:59:25 +00:00
51616	c9e958f963	self-gen w/ closed_qa format + perceiver scripts + no strip response	2025-07-08 14:21:43 +00:00
51616	43f47d40ad	tiny exp	2025-07-03 16:26:26 +00:00
51616	1a03b01a52	small adjustment data gen (remove empty)	2025-07-03 11:02:46 +00:00
51616	f8a81a1eea	self_gen temp fix + lv1 temp varying configs	2025-07-01 15:30:28 +00:00
51616	28eb85a2f3	tiny configs + fix dataset gen (remove repeated response + qa template for compact ds) + add ropes and drop eval ds	2025-06-30 16:10:52 +09:00
51616	19417d28a6	self_gen data name	2025-06-27 05:56:08 +00:00
51616	f07b38d5da	more configs	2025-06-27 01:57:15 +09:00
51616	1457981348	compact ds + self_gen + update rope + drop multi answer eval	2025-06-26 16:04:33 +00:00
51616	f08831d957	qa data exp	2025-06-26 15:30:26 +09:00
51616	816377b4ac	clean up + configs + scripts	2025-06-24 21:51:33 +09:00
51616	0d8e50d218	new trainng data format working non-packing	2025-06-23 23:21:22 +09:00
51616	36d78c827c	transformers ver + squad only example	2025-06-20 21:38:26 +09:00
51616	4c1e4062e1	squad_hotpot config	2025-06-19 23:08:14 +09:00
51616	b218df0feb	gemma 3	2025-06-17 11:35:12 +00:00
51616	b61086034c	data exp + packing	2025-06-17 11:34:16 +00:00
51616	bae2593cd4	fix configs	2025-06-10 15:39:56 +00:00
51616	669186ace6	gemma data exp	2025-06-06 15:29:27 +00:00
51616	6242341099	data scripts + readme	2025-06-06 09:48:42 +00:00
51616	a9a164e2de	clean up configs + scripts + chat_templates (+ %generation% tag)	2025-06-06 04:56:45 +00:00
51616	8b15901fb9	self-gen data naming + num_proc to 2	2025-06-03 15:32:38 +00:00
51616	5aa83b96e2	self gen data for all ds both train + val	2025-06-03 14:30:38 +00:00
51616	dcda29bae6	watcher + partial refactor eval + debug train w/ pwc and hotpot	2025-06-02 09:02:20 +00:00
51616	0ff762cdff	medium fw qa 3 + fallback to manual cache dataset	2025-05-10 15:31:08 +00:00
51616	dde5f5cb4b	load data from disk then convert to iterable	2025-05-09 13:11:34 +00:00
51616	bd61787486	allow streaming data	2025-05-09 10:59:08 +00:00
51616	dfac7a51b3	fw_qa_3_mini	2025-05-09 02:14:26 +00:00
51616	49214d34df	add per rank latent query + per layer hyper head	2025-04-19 07:03:11 +00:00
51616	ff7bf56029	add booksum and gov_report	2025-02-18 10:43:02 +00:00
51616	584b2bfb44	fw_qa_xl + modernbert	2025-02-14 14:59:25 +00:00
51616	4bad251877	default warmup + decay	2025-01-28 21:36:19 +00:00
51616	cf742f8fbd	everything :)	2025-01-22 12:59:22 +00:00
51616	930a22b2c0	openmathintx ft ds	2025-01-20 09:18:14 +00:00
51616	c983a3a0d8	more data	2025-01-20 09:18:07 +00:00
51616	c22c98764e	add gsm8k + finetuning + more pretrain data	2025-01-19 19:05:01 +00:00
51616	4f4aecdb00	add squad and gemma	2025-01-18 12:36:12 +00:00
51616	cce43deb64	restructure	2025-01-17 17:41:10 +00:00
51616	d952c54eee	repeat only shorter ctxs	2025-01-14 20:10:07 +00:00
51616	273b35378f	custom ctx_encoder!	2025-01-13 16:45:26 +00:00
51616	302ed4dbdd	include new data!	2025-01-13 14:21:41 +00:00
51616	108907590e	batch size for hotpot	2025-01-10 20:06:29 +00:00
51616	b9394c0fd6	warmup=0.1	2025-01-10 18:40:56 +00:00
51616	4d50ad84fa	hotpot_qa fix!!! (strip) + ds_kwargs + easier ds names	2025-01-10 17:37:13 +00:00
51616	440aff4151	added hotpot_qa	2025-01-10 10:42:27 +00:00
51616	3d1321a947	add neg prompt + fix repeat (on works on unique ctx now)	2025-01-09 14:21:35 +00:00
51616	eb522f678d	add validation set when not exist	2025-01-09 10:39:05 +00:00
51616	2381b6a93e	add repeat prompt	2025-01-08 16:17:07 +00:00
51616	6c579fa373	limit max test sample + config test up to 256 number	2025-01-07 16:57:23 +00:00
51616	2a4e5742ca	data + configs testing model capability	2025-01-07 14:44:51 +00:00
51616	c7f7a2f788	new configs	2025-01-07 12:25:47 +00:00