LLM Fine-Tuning

Fine-tune large language models for domain-specific tasks. Covers full fine-tuning, LoRA, QLoRA, dataset preparation, evaluation, deployment, and the patterns that produce specialized models without the cost of training from scratch.

LLM fine-tuning is the process of adapting a pre-trained large language model to a specific task, domain, or dataset by continuing training on new data. It matters when the base model’s general knowledge is insufficient—when prompt engineering alone fails to achieve required precision, coverage, or consistency. It is the go-to technique when domain-specific terminology, style, or logic must be embedded into the model, and when performance gains from in-context learning are outpaced by cost, latency, or need for consistency across large volumes of text.

Preparing the Training Dataset

A fine-tuning job begins with a dataset structured as a sequence of input-output pairs. Each example is a JSON object with prompt and completion fields, optionally including system_prompt and metadata. The dataset is stored as a single file in a standard format: one JSON object per line (JSONL), with each line representing a training example.

{"prompt": "Summarize the following text in one sentence:\n\n", "completion": "The project delivered a scalable API service with reduced latency and improved error handling."}
{"prompt": "Translate the following to French:\n\n", "completion": "Traduire le texte suivant en français :\n\n"}

The dataset must be cleaned and validated. Common failures include mismatched prompt lengths, missing completions, and inconsistent formatting in the prompt field. A silent but pervasive error is the use of \n\n in the prompt where \n suffices, leading to misaligned tokenization. The completion field should be trimmed and normalized: whitespace differences between expected and actual outputs cause divergent gradients and unstable convergence.

To validate a dataset, use jq:

jq -c 'select(.prompt == null or .completion == null)' data.jsonl

This command identifies examples with missing fields. Use python to compute the average token count per example using Hugging Face’s tokenizers:

from tokenizers import Tokenizer
from pathlib import Path

tokenizer = Tokenizer.from_pretrained("bert-base-uncased")
dataset_path = Path("data.jsonl")

total_tokens = 0
example_count = 0

for line in dataset_path.open():
    example = json.loads(line)
    prompt_tokens = tokenizer.encode(example["prompt"]).ids
    completion_tokens = tokenizer.encode(example["completion"]).ids
    total_tokens += len(prompt_tokens) + len(completion_tokens)
    example_count += 1

print(f"Average tokens per example: {total_tokens / example_count:.1f}")

Choosing a Training Configuration

Fine-tuning requires a configuration file describing hyperparameters, model architecture, and training schedule. The configuration is a YAML file with standard keys, using the same structure across all workflows.

model: "meta-llama/Llama-2-7b-hf"
output_dir: "checkpoints/llama-2-7b-finetuned"
per_device_train_batch_size: 8
gradient_accumulation_steps: 4
num_train_epochs: 3
learning_rate: 2e-5
warmup_steps: 500
weight_decay: 0.01
logging_steps: 100
save_steps: 1000
eval_steps: 1000
fp16: true
max_length: 1024

The max_length key controls the maximum number of tokens in both prompt and completion. If max_length is too small, examples are truncated; if too large, memory usage grows and gradients become sparse. A common failure is setting max_length without padding strategy: the model sees sequences of varying lengths, but the input tensor is not padded to the maximum across the batch.

The gradient_accumulation_steps parameter controls effective batch size. With per_device_train_batch_size: 8 and gradient_accumulation_steps: 4, the model computes gradients over 32 total tokens per update, but only updates weights every 4 forward passes. This leads to smoother convergence, but a mismatch between gradient_accumulation_steps and save_steps causes checkpoints to be saved at irregular intervals.

Running the Training Job

Training is launched using a script that orchestrates data loading, model initialization, and optimizer setup. The most common interface is a command-line tool that accepts a configuration file and a dataset path.

python -m transformers.trainer \
    --config_file config.yaml \
    --train_dataset_path data.jsonl \
    --eval_dataset_path eval.jsonl \
    --output_dir checkpoints/llama-2-7b-finetuned

The transformers library from Hugging Face is used here, but the workflow is generic: the script reads the YAML file, instantiates the model, loads the tokenizer, and sets up a Trainer instance. The dataset is loaded using datasets.Dataset.from_json or load_dataset("json", data_files=...).

A key failure mode is mismatched tokenizer and model. The tokenizer must be compatible with the model’s vocabulary. When using a model from Hugging Face Hub, the tokenizer is typically included in the model directory. To verify, use:

ls -la checkpoints/llama-2-7b-finetuned/tokenizer_config.json

If the file is missing or has incorrect vocab_size, max_length, or pad_token, the model will misalign tokens. A silent error: the model processes input, but the pad_token is not recognized, causing padding values to be treated as regular tokens.

The training job logs to standard output and files. The following error appears when the model fails to load a checkpoint:

RuntimeError: Error(s) in loading state_dict for LlamaForCausalLM:
    Unexpected key(s) in state_dict: "model.embed_tokens.weight", "model.layers.0.self_attn.q_proj.weight"
    Missing keys in state_dict: "model.embed_tokens.weight"

This indicates that the checkpoint was saved with a different model architecture than expected—likely from a different version of the transformers library.

Evaluating Fine-Tuned Models

After training, the model must be evaluated on a held-out dataset. Evaluation is done by computing the average loss and accuracy across a set of examples.

from transformers import pipeline, AutoModelForCausalLM, AutoTokenizer
from datasets import load_dataset

model = AutoModelForCausalLM.from_pretrained("checkpoints/llama-2-7b-finetuned")
tokenizer = AutoTokenizer.from_pretrained("checkpoints/llama-2-7b-finetuned")

eval_dataset = load_dataset("json", data_files="eval.jsonl", split="train")

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0)

results = []
for example in eval_dataset:
    prompt = example["prompt"]
    expected = example["completion"]
    generated = pipe(prompt, max_length=1024, num_return_sequences=1)[0]["generated_text"]

    # Compute similarity
    generated = generated[len(prompt):].strip()
    accuracy = 1.0 if generated == expected else 0.0
    results.append(accuracy)

print(f"Average accuracy: {sum(results) / len(results):.3f}")

The evaluation pipeline must align the prompt and completion fields. A common failure: generated_text includes the input prompt, but the evaluation expects only the completion. The solution: slice the output from len(prompt) onward.

For a more robust evaluation, compute the sequence-level accuracy using transformers’ compute_metrics function:

def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    # Remove padding and special tokens
    predictions = np.argmax(predictions, axis=2)
    labels = np.where(labels != -100, labels, 0)  # -100 is padding in labels

    # Flatten
    predictions = predictions.flatten()
    labels = labels.flatten()

    # Compute accuracy
    accuracy = np.mean(predictions == labels)
    return {"accuracy": accuracy}

This function is passed to the Trainer as eval_metrics.

Deploying the Fine-Tuned Model

Once trained, the model is ready to be deployed. The model directory contains:

To serve the model, use uvicorn with a simple API endpoint.

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from transformers import pipeline
import torch

app = FastAPI()

class PromptRequest(BaseModel):
    prompt: str
    max_length: int = 1024
    temperature: float = 1.0


model = pipeline(
    "text-generation",
    model="checkpoints/llama-2-7b-finetuned",
    device=0,
    torch_dtype=torch.bfloat16,
)

@app.post("/generate")
async def generate(request: PromptRequest):
    try:
        result = model(
            request.prompt,
            max_length=request.max_length,
            temperature=request.temperature,
            num_return_sequences=1,
        )
        return {"generated_text": result[0]["generated_text"]}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

Deploy with uvicorn main:app --host 0.0.0.0 --port 8000.

A common failure: the model is deployed without torch_dtype set to bfloat16, leading to high memory usage and slower inference. The model loads in float32, but inference is slow and memory-bound.

Handling Common Failure Modes

gradient_checkpointing: true
print(tokenizer.bos_token_id, tokenizer.eos_token_id, tokenizer.pad_token_id)

If any are None, the model will fail to process sequences correctly.

Monitoring and Iterating

After deployment, monitor the model’s input-output pairs. Log the prompt, generated_text, and input_ids for each request. A failure occurs when the model generates null or "" for a prompt: check that pad_token_id is set and that input_ids are not all pad_token_id.

Use tensorboard to visualize training curves. Run:

tensorboard --logdir checkpoints/llama-2-7b-finetuned

The events.out.tfevents.* file contains loss, accuracy, and learning rate metrics. A flat accuracy curve indicates a learning rate that is too low.

Iterate: when new data arrives, retrain the model with a new dataset. Use --resume_from_checkpoint to continue training from the latest checkpoint:

python -m transformers.trainer \
    --config_file config.yaml \
    --train_dataset_path new_data.jsonl \
    --resume_from_checkpoint checkpoints/llama-2-7b-finetuned

This command loads the model, optimizer state, and training progress, enabling seamless updates.

Fine-tuning is not a one-time process. It is a continuous loop of data collection, training, evaluation, deployment, and feedback. Each step must be reliable, and each failure must be visible and actionable. The model is never “done”—it is always learning.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.