Fine-Tuning Expert gives Claude a five-step routine for adapting language models: prepare and validate the dataset, choose a parameter-efficient method, train while watching validation loss, evaluate against the base model and deploy a merged, quantized model.
Five reference files cover LoRA and PEFT, dataset preparation, hyperparameter tuning, evaluation metrics and deployment optimization. The SKILL.md carries a complete LoRA training example with a QLoRA variant and an adapter merge.
What You Get
- A dataset preparation script with schema checks, token-length statistics and deduplication
- A training configuration with TrainingArguments and LoraConfig, commented
- An evaluation script reporting perplexity, task metrics and latency
- A design rationale for the method, rank and learning rate
- Steps to merge adapters, quantize and serve the model
Common Use Cases
- Adapting an open model to an instruction-following dataset
- Training on a single GPU with QLoRA four-bit quantization
- Comparing a tuned model with the base model on held-out data
- Exporting a merged model to GGUF or a serving container
What Makes It Unique
Each step has a checkpoint: dataset errors are fixed before training starts, validation loss must fall during training, and a model is not deployed before a held-out evaluation and a latency benchmark. The rules also forbid merging adapters with mismatched rank, base model or target modules and require learning rate warmup.
Under The Hood
The skill is a SKILL.md with five reference files and no scripts, and the references load by topic. The examples download base models and datasets from Hugging Face, train on local GPUs, write checkpoints and adapter folders to disk and can push merged models to a hub repository. The deployment reference runs llama.cpp conversion and quantization programs through subprocess calls and includes a container setup for a text generation server. Training runs can report to Weights and Biases. The workflow also expects a validate_dataset.py script that the agent writes.