apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-23 17:01:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
Hypernetworks that update LLMs to remember factual information https://arxiv.org/abs/2602.15902
50 commits 2 branches 0 tags 47 MiB
Find a file
51616 a88dbd135f perceiver working w/ context_numbers_medium
2024-12-24 17:19:06 +00:00
configs fix tokenizer pad token generation warning 2024-12-24 13:44:51 +00:00
data/raw_datasets formatting 2024-12-24 07:50:08 +00:00
hyperlora perceiver working w/ context_numbers_medium 2024-12-24 17:19:06 +00:00
icae_v2 formatting 2024-12-20 10:18:30 +00:00
webui formatting 2024-12-24 07:50:08 +00:00
.gitignore line length to 89 2024-12-19 13:08:22 +00:00
.pre-commit-config.yaml init commit 2024-12-18 13:05:19 +00:00
install.sh refactor config + schedule free adam + liger kernel 2024-12-21 12:39:03 +00:00
pyproject.toml line length to 89 2024-12-19 13:08:22 +00:00
README.md add lora ft cmd to readme 2024-12-20 15:09:16 +00:00
requirements.txt add webui requirements 2024-12-23 15:17:15 +00:00

README.md

Project doc

Finetuning the base model with LoRA adaptor

python hyperlora/intx_sft.py configs/default.yaml --model_name_or_path=meta-llama/Llama-3.2-1B-Instruct --num_train_epochs=100 --per_device_train_batch_size=8 --per_device_eval_batch_size=32 --overwrite_output_dir=True