apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-26 17:11:02 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

chunked ctx eval (#9)

Browse source 
* got burned by floating point precision again :)))))))))))

* split-ctx working eval (batch_size=1)

* batched lora aggregation (sum + mean) eval + finegrain eval len bins

* fix ctx_ids assert
This commit is contained in:
Rujikorn Charakorn 2025-08-11 18:44:26 +09:00 • committed by GitHub
parent 537ab4bb65
commit ec955a935d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 474 additions and 67 deletions
Download patch file Download diff file Expand all files Collapse all files

2
train.py
View file

@ -259,6 +259,8 @@ def main():
ctx_model_max_len=ctx_model_max_len,
ctx_tokenizer=ctx_tokenizer,
add_ctx_to_chat=add_ctx_to_chat,
max_ctx_chunk_len=ctx_args.max_ctx_chunk_len,
max_ctx_chunk_num=ctx_args.max_ctx_chunk_num,
use_kl_loss=ctx_args.use_kl_loss,
)
splits = ["train"]