apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-23 17:01:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

download self gen qa

Browse source
This commit is contained in:
51616 2025-06-26 16:30:04 +00:00
parent 1457981348
commit dee7df746d
2 changed files with 10 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -75,6 +75,7 @@ uv run python data/generate_fav_num_big.py
***Download a subset of generated Fineweb-QA directly***
```bash
uv run data/download_generated_fineweb_qa.py
uv run data/download_self_gen_qa.py
```
***Generate data from scratch***

9
data/download_self_gen_qa.py Normal file
View file

@ -0,0 +1,9 @@
from huggingface_hub import snapshot_download
if __name__ == "__main__":
fw_dir = "./data/raw_datasets/self_gen/"
snapshot_download(
"SakanaAI/self_gen_qa",
repo_type="dataset",
local_dir=fw_dir,
)