Oracle/Unsloth-Finetune-Template
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Clarify variable adjustment in README.md

Browse source
This commit is contained in:
Oracle 2026-06-02 17:41:18 +02:00
parent 63e1c0c486
commit 04007141bd
Signed by: Oracle
SSH key fingerprint: SHA256:x4/RtnjUyuHkdvmwNDsWSfcfF1V5PNr3OpriZqOvCX8
2 changed files with 11 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

20
README.md
View file

@ -102,7 +102,7 @@ Generates synthetic training data using a GGUF model via llama.cpp. Run this if
| `top_k` | Top-k sampling threshold | `50` |
| `min_p` | Minimum probability threshold | `0.05` |
The model expects output in the format:
The script expects output in the format:
```
Question: <generated question>
@ -115,14 +115,14 @@ bash scripts/generate-data.sh
### 2. scripts/finetune.sh
Fine-tunes a model using Unsloth with LoRA adapters. Saves LoRA weights to `./model/`.
Fine-tunes a model using Unsloth with LoRA adapters. Saves LoRA adapter to `./model/`.
**Edit `finetune.py`:**
| Variable | Description | Example |
|---|---|---|
| `DATA_PATH` | Path to training Parquet file | `./data/output.parquet` |
| `OUTPUT_DIR` | Directory to save LoRA adapters | `./model` |
| `OUTPUT_DIR` | Directory to save LoRA adapters (leave at default) | `./model` |
| `BATCH_SIZE` | Per-device batch size | `2` |
| `GRADIENT_ACCUMULATION_STEPS` | Gradient accumulation steps | `8` |
| `LEARNING_RATE` | Training learning rate | `2e-4` |
@ -142,9 +142,9 @@ Merges LoRA adapters into the base model, saves the merged model, then converts
| Variable | Description | Example |
|---|---|---|
| `BASE_MODEL_PATH` | Path to the base model | `""` (empty to load from HuggingFace) |
| `LORA_DIR` | Path to LoRA adapters | `./model` |
| `MERGED_MODEL_PATH` | Output directory for merged model | `./merged_model` |
| `BASE_MODEL_PATH` | Path to the base model (same as model_name in finetune.py) | `"Qwen/Qwen3.5-2B"` |
| `LORA_DIR` | Path to LoRA adapters (leave at default) | `./model` |
| `MERGED_MODEL_PATH` | Output directory for merged model (leave at default) | `./merged_model` |
```bash
bash scripts/merge-and-convert.sh
@ -158,7 +158,7 @@ Runs the converted GGUF model using llama.cpp's CLI interface for inference.
| Variable | Description | Example |
|---|---|---|
| Model path | Path to the GGUF file | `./merged_model/model.gguf` |
| Model path | Path to the GGUF file (gguf file name will vary based on base model) | `./merged_model/model.gguf` |
```bash
bash scripts/run-model.sh
@ -171,7 +171,7 @@ bash scripts/run-model.sh
./merged_model/ ← Merged HF model + GGUF file (from merge-and-convert.sh)
llama.cpp/ ← llama.cpp repository (created by setup.sh)
scripts/ ← Individual pipeline step scripts
setup.sh ← Setup script (venv + llama.cpp build)
setup.sh ← Setup script (venv + llama.cpp build/symlink)
run-pipeline.sh ← Run full pipeline (finetune → merge/convert → run)
```
@ -197,9 +197,9 @@ Common issues:
### llama-cpp-python install fails
- Ensure llama.cpp is built successfully first
- Ensure llama.cpp is built successfully first (or build it yourself if you want to use a backend other than CUDA, ROCm or Vulkan)
- Try CPU-only install first to verify: `pip install llama-cpp-python`
- Check [llama-cpp-python docs](https://llama-cpp-python.readthedocs.io/en/latest/) for other backends
- Check [llama-cpp-python docs](https://llama-cpp-python.readthedocs.io/en/latest/)
## Project Structure

2
finetune.py
View file

@ -1,4 +1,4 @@
# CRITICAL: Import unsloth BEFORE any other packages
# CRITICAL: Import unsloth BEFORE any other packages so all optimizations are applied (Unsloth recommendation)
import os
import warnings
from unsloth import FastLanguageModel, is_bfloat16_supported