apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-05-05 13:02:38 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Doc-to-LoRA release

Browse source
This commit is contained in:
51616 2026-02-27 03:47:04 +00:00
commit 1abe8ae16d
92 changed files with 22131 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

10
data/download_fineweb_edu.py Normal file
View file

@ -0,0 +1,10 @@
from huggingface_hub import snapshot_download
if __name__ == "__main__":
fw_dir = "./data/raw_datasets/fineweb_edu/"
snapshot_download(
"HuggingFaceFW/fineweb-edu",
repo_type="dataset",
local_dir=fw_dir,
allow_patterns="sample/100BT/*",
)