doc-to-lora

mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-23 17:01:04 +02:00

Hypernetworks that update LLMs to remember factual information https://arxiv.org/abs/2602.15902

ai ai-agent hypernetworks llm llm-agent lora machine-learning memory

Find a file

51616 d4f01c999e more metrics		2024-12-22 18:52:32 +00:00
configs	more metrics	2024-12-22 18:52:32 +00:00
hyperlora	more metrics	2024-12-22 18:52:32 +00:00
icae_v2	formatting	2024-12-20 10:18:30 +00:00
.gitignore	line length to 89	2024-12-19 13:08:22 +00:00
.pre-commit-config.yaml	init commit	2024-12-18 13:05:19 +00:00
install.sh	refactor config + schedule free adam + liger kernel	2024-12-21 12:39:03 +00:00
pyproject.toml	line length to 89	2024-12-19 13:08:22 +00:00
README.md	add lora ft cmd to readme	2024-12-20 15:09:16 +00:00
requirements.txt	refactor config + schedule free adam + liger kernel	2024-12-21 12:39:03 +00:00

README.md

Project doc

Finetuning the base model with LoRA adaptor

python hyperlora/intx_sft.py configs/default.yaml --model_name_or_path=meta-llama/Llama-3.2-1B-Instruct --num_train_epochs=100 --per_device_train_batch_size=8 --per_device_eval_batch_size=32 --overwrite_output_dir=True