From 04007141bd35a70876f8a434be97fa50ec246429 Mon Sep 17 00:00:00 2001 From: Oracle Date: Tue, 2 Jun 2026 17:41:18 +0200 Subject: [PATCH] Clarify variable adjustment in README.md --- README.md | 20 ++++++++++---------- finetune.py | 2 +- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index cac3fbf..8094c3d 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,7 @@ Generates synthetic training data using a GGUF model via llama.cpp. Run this if | `top_k` | Top-k sampling threshold | `50` | | `min_p` | Minimum probability threshold | `0.05` | -The model expects output in the format: +The script expects output in the format: ``` Question: @@ -115,14 +115,14 @@ bash scripts/generate-data.sh ### 2. scripts/finetune.sh -Fine-tunes a model using Unsloth with LoRA adapters. Saves LoRA weights to `./model/`. +Fine-tunes a model using Unsloth with LoRA adapters. Saves LoRA adapter to `./model/`. **Edit `finetune.py`:** | Variable | Description | Example | |---|---|---| | `DATA_PATH` | Path to training Parquet file | `./data/output.parquet` | -| `OUTPUT_DIR` | Directory to save LoRA adapters | `./model` | +| `OUTPUT_DIR` | Directory to save LoRA adapters (leave at default) | `./model` | | `BATCH_SIZE` | Per-device batch size | `2` | | `GRADIENT_ACCUMULATION_STEPS` | Gradient accumulation steps | `8` | | `LEARNING_RATE` | Training learning rate | `2e-4` | @@ -142,9 +142,9 @@ Merges LoRA adapters into the base model, saves the merged model, then converts | Variable | Description | Example | |---|---|---| -| `BASE_MODEL_PATH` | Path to the base model | `""` (empty to load from HuggingFace) | -| `LORA_DIR` | Path to LoRA adapters | `./model` | -| `MERGED_MODEL_PATH` | Output directory for merged model | `./merged_model` | +| `BASE_MODEL_PATH` | Path to the base model (same as model_name in finetune.py) | `"Qwen/Qwen3.5-2B"` | +| `LORA_DIR` | Path to LoRA adapters (leave at default) | `./model` | +| `MERGED_MODEL_PATH` | Output directory for merged model (leave at default) | `./merged_model` | ```bash bash scripts/merge-and-convert.sh @@ -158,7 +158,7 @@ Runs the converted GGUF model using llama.cpp's CLI interface for inference. | Variable | Description | Example | |---|---|---| -| Model path | Path to the GGUF file | `./merged_model/model.gguf` | +| Model path | Path to the GGUF file (gguf file name will vary based on base model) | `./merged_model/model.gguf` | ```bash bash scripts/run-model.sh @@ -171,7 +171,7 @@ bash scripts/run-model.sh ./merged_model/ ← Merged HF model + GGUF file (from merge-and-convert.sh) llama.cpp/ ← llama.cpp repository (created by setup.sh) scripts/ ← Individual pipeline step scripts -setup.sh ← Setup script (venv + llama.cpp build) +setup.sh ← Setup script (venv + llama.cpp build/symlink) run-pipeline.sh ← Run full pipeline (finetune → merge/convert → run) ``` @@ -197,9 +197,9 @@ Common issues: ### llama-cpp-python install fails -- Ensure llama.cpp is built successfully first +- Ensure llama.cpp is built successfully first (or build it yourself if you want to use a backend other than CUDA, ROCm or Vulkan) - Try CPU-only install first to verify: `pip install llama-cpp-python` -- Check [llama-cpp-python docs](https://llama-cpp-python.readthedocs.io/en/latest/) for other backends +- Check [llama-cpp-python docs](https://llama-cpp-python.readthedocs.io/en/latest/) ## Project Structure diff --git a/finetune.py b/finetune.py index 90e7ada..82fae8e 100644 --- a/finetune.py +++ b/finetune.py @@ -1,4 +1,4 @@ -# CRITICAL: Import unsloth BEFORE any other packages +# CRITICAL: Import unsloth BEFORE any other packages so all optimizations are applied (Unsloth recommendation) import os import warnings from unsloth import FastLanguageModel, is_bfloat16_supported