Fine Tuning Large Language Models, Part 1

If you've spent any time around large language models (LLMs), you've probably heard the term “fine-tuning” used to mean about six different things. Some people use it to describe writing a really good prompt. Others use it to describe attaching a small classifier on top of a frozen model. Others mean updating every single weight in a 7-billion-parameter model. All three are real techniques, they solve different problems, and confusing them is the single most common reason beginners waste GPU hours (or a weekend of CPU time) on the wrong approach. This article is a practical, hands-on guide to the entire spectrum—from the lightest-weight option (feature extraction, where the pretrained model never changes at all) through partial fine-tuning, full fine-tuning, and parameter-efficient fine-tuning with LoRA. It is written to be beginner-accessible: every concept is explained before it's used, and every code example is runnable on ordinary hardware. It is also platform-agnostic. Whether you're on a Windows PC with an NVIDIA GPU, a Mac with Apple silicon, or a Linux workstation, the same Python code will work, because everything here is built on the Hugging Face ecosystem (transformers, datasets, peft, accelerate), which automatically detects and uses whatever hardware you have—CUDA, Apple's Metal Performance Shaders (MPS), or plain CPU.

By the end of this article, you will understand:

• What actually happens, mathematically and practically, when you “fine-tune” a model
• When to use feature extraction, partial fine-tuning, full fine-tuning, or LoRA—and why the choice matters for your budget and your timeline
• How to prepare a dataset for training
• How to run each of these techniques with working code
• How to evaluate whether your fine-tuned model actually improved
• How to export your model, regardless of what platform you trained it on

Why Fine-Tune at All?

Modern LLMs are extraordinarily capable generalists. A well-crafted prompt against a frontier model can often get you 90% of the way to what you need, and prompting has a huge advantage: it costs nothing but your time, and you can iterate in seconds. So, the first honest thing to say about fine-tuning is this—it should rarely be your first move.

Reach for fine-tuning when one or more of these is true:

• The task requires a specific output format or style that prompting can't reliably enforce (e.g., a support ticket classifier that must always return one of 12 exact category labels).
• You need the behavior “baked in” so you don't have to repeat a long instruction in every prompt, which saves tokens and latency at scale.
• The task involves patterns that are hard to describe in words but easy to demonstrate with examples (e.g., matching your company's specific tone, or recognizing domain-specific entities).
• You need the model to run without internet access, on-device, or air-gapped, and a smaller fine-tuned model can match a much larger general-purpose model on your narrow task.
• Retrieval-augmented generation (RAG) isn't enough—RAG is great for giving a model new facts, but poor at teaching it new behaviors or styles.

If none of those apply, try prompting or RAG first. Fine-tuning is a bigger investment of time, data, and compute, and it's easy to end up with a model that's worse than the base model at everything except the narrow thing you trained it on (a phenomenon called catastrophic forgetting, which we'll return to later).

Neural Network and Transformer Fundamentals

Before diving deeper into fine-tuning techniques, it helps to ground a few terms that get used constantly from here on—“parameters,” “weights,” “attention”—with a quick, concrete picture of what they actually are. If you're already comfortable with neural network internals, feel free to skip this; if terms like “trainable parameters” or “attention layer” have always been a little fuzzy, this section is for you.

What a Neural Network Actually Computes

At its core, a neural network is a chain of simple mathematical steps. Data flows in through an input layer, passes through one or more hidden layers, and produces a result at the output layer. Each connection between two neurons carries a weight—a number that scales how much influence one neuron has on the next—and each neuron typically adds a bias, a number that shifts its output up or down before an activation function decides how much of the signal to pass forward.

Training a neural network means repeatedly showing it examples, measuring how wrong its output is (the loss), and using an optimizer to nudge every weight and bias slightly in the direction that would have made the output less wrong—a process called backpropagation. Do this enough times across enough examples, and the weights and biases settle into values that make the network good at the task.

Figure 1 shows a simple neural network with several layers, along with the weights and biases.

Figure 1: A neural network showing input, hidden, and output layers connected by weighted edges

Trainable Parameters

“Parameters” simply means the full collection of weights and biases a model can adjust during training. When you see a model described as “7B” or “671B,” that number is a count of these trainable parameters—671 billion individual weights and biases, in the case of a very large model. This is worth internalizing early, because everything in this article is really a conversation about how many of those parameters you allow gradient descent to touch, and which ones:

• Feature extraction touches zero of the pretrained model's parameters—it trains a small new set from scratch instead.
• Partial fine-tuning touches a subset—typically the parameters in the last few layers.
• Full fine-tuning touches all of them.
• LoRA touches none of the original parameters, but adds a small new set of its own alongside them.

Every technique in this article is a different answer to that same question, and keeping the raw parameter count in mind is a useful sanity check throughout: a model description that says “trainable params: 739,590 / 67,694,596” is telling you, directly, how much of the model's original 67.7 million parameters were left completely untouched.

A First Look Inside Attention

Modern transformer-based models—which is to say, essentially every model discussed in this article—are built from repeated attention layers. An attention layer is what lets the model figure out which parts of the input are relevant to which other parts when producing a prediction, and it's worth having at least a rough picture of its mechanics, because LoRA specifically targets these layers, and it's much easier to understand why once you've seen how attention works. Figure 2 shows how an attention layer processes a token.

Figure 2: Diagram showing a token converted into Query, Key, and Value vectors combined via the attention formula

Each token in the input is converted into three vectors:

• Query (Q): what this token is “looking for” in the rest of the sequence
• Key (K): what each token has to “offer” as a potential match
• Value (V): the actual information passed along once a match is found

Attention scores how much each token should attend to every other token by comparing queries against keys, then uses those scores to combine the values into a new representation:

Attention(Q, K, V) = 
    softmax( Q · K^T / sqrt(d) ) · V

You don't need to memorize this formula to fine-tune models effectively, but the intuition matters: the Q/K/V projection matrices are where the model “decides what to pay attention to,” which is precisely why LoRA (and most PEFT techniques) inject their small trainable matrices at exactly these projections (q_proj, k_proj, v_proj, or their model-specific equivalents like q_lin/v_lin) rather than elsewhere in the network. Small, targeted updates to the mechanism that decides “what matters” turn out to be a remarkably efficient lever for adapting model behavior—much more efficient than nudging every weight in the network by the same small amount.

Fine-Tuning Fundamentals for Beginners

Before diving into code, it helps to see where fine-tuning actually sits in the bigger picture. Fine-tuning isn't a standalone technique—it's one branch of a broader idea called transfer learning—taking a model that's already learned useful patterns from massive amounts of data, and adapting that knowledge to a new, more specific task rather than starting from scratch. Transfer learning itself splits into two approaches (see Figure 3).

Feature extraction is the lighter-touch option—freeze the pre-trained model entirely and train only a small new “head” on top of it.

Fine-tuning goes further, unfreezing some or all of the pre-trained weights so the model itself can adapt, not just the layer sitting on top of it.

Figure 3: Fine- tuning is part of transfer learning

Within fine-tuning, there's a further choice to make (see Figure 4): update every weight (full fine-tuning), update a small, cleverly chosen subset of parameters (parameter-efficient fine-tuning, or PEFT—the category LoRA belongs to), or leave the model's weights untouched entirely and instead learn a trainable prompt that steers its behavior (prompt-based fine-tuning).

Understanding this hierarchy up front makes it much easier to see how these techniques relate to one another and where each one fits. In this article, we'll focus on feature extraction, along with full and partial fine-tuning, working through the mechanics and tradeoffs of each. Prompt-based fine-tuning is a different approach entirely—one we'll save for the next article.

Figure 4 😗* Three approaches to fine-tuning**—full fine-tuning updates every weight, parameter-efficient fine-tuning (like LoRA) updates a small, targeted subset, and prompt-based fine-tuning leaves weights frozen and trains prompt embeddings instead.

What a Pretrained Model Already Knows

A pretrained language model—whether it's BERT, DistilBERT, GPT-style, or a modern instruction-tuned model like Llama or Qwen—has already been trained on a massive amount of text. During that pretraining process, the model's parameters (its weights) were adjusted, layer by layer, so that the model became good at predicting missing or next words. Along the way, it developed internal representations of grammar, facts, reasoning patterns, and—critically for us—general-purpose features: numerical representations of meaning that transfer well to new tasks.

This is the key insight that makes everything in this article possible: you almost never need to train a model from scratch. You start from a checkpoint that already understands language, and you adapt it.

The Four Ways to Adapt a Pretrained Model

There is a spectrum of techniques for adapting a pretrained model to a new task, and they differ mainly in how many parameters you allow to change (see Table 1).

We'll walk through all four in this article, in increasing order of how much of the model you touch, because understanding feature extraction first makes it much easier to understand what fine-tuning is actually doing differently. Figure 5 summarizes how much of the model gets trained when you use the various fine-tuning techniques.

Figure 4: Bar chart showing the proportion of a model's parameters left trainable by feature extraction, prompt tuning, LoRA, partial fine-tuning, and full fine-tuning.

A Concrete Mental Model: The Model as a Feature Pipeline

Think of a transformer model as a pipeline with two conceptual parts:

• The backbone (encoder/decoder layers): dozens of transformer blocks that progressively transform raw input tokens into rich numerical representations. Early layers tend to capture low-level patterns (syntax, local word relationships); later layers capture higher-level, more task-relevant patterns (semantics, sentiment, entities).
• The head: a small final layer (or few layers) that maps those representations to your actual output—a class label, a next-token probability distribution, a similarity score, and so on.

When you pretrain a model, both the backbone and the head (in the pretraining sense—usually a “predict the next/missing token” head) get trained together on enormous amounts of generic text. When you adapt the model to your task, you have a choice about which parts to touch:

• Feature extraction = keep the backbone frozen exactly as pretrained, throw away the pretraining head, and train a brand-new small head from scratch on your data.
• Partial fine-tuning = keep most of the backbone frozen, but unfreeze the last handful of backbone layers plus the new head, and train all of those together.
• Full fine-tuning = unfreeze the entire backbone plus the head, and train everything.
• LoRA = keep the entire backbone frozen (like feature extraction), but instead of only training a new head, inject small trainable low-rank matrices alongside the frozen weight matrices throughout the backbone, and train only those—which turns out to approximate full fine-tuning surprisingly well, at a tiny fraction of the trainable-parameter count.

Keep this mental model in mind. Every technique in this article is really just a different answer to the question: how much of the pipeline do we let gradient descent (the algorithm that updates a model's weights during training) touch?

A simple analogy. Full training from scratch is like learning a language from zero—memorizing an alphabet, grammar rules, and vocabulary before you can say anything at all. Fine-tuning, by contrast, starts from someone already fluent in the language, who is now picking up the specific jargon of a new field—medical terminology, legal contracts, or restaurant-review slang. That's why fine-tuning needs so much less data and time than pretraining: the hard, general-purpose work is already done.

Choosing Your Approach: A Decision Framework

Here is a simple set of questions to work through before you write any training code:

• Step 1—Can a well-crafted prompt solve this? If yes, stop here. Prompting is cheaper and faster to iterate on than anything in these steps.
• Step 2—Is this fundamentally a “give the model new facts” problem, not a “teach the model a new skill/style” problem? If yes, try retrieval-augmented generation before fine-tuning.
• Step 3—Is your task “extract a representation and make a decision” (classification, similarity, clustering, ranking)? If yes, start with feature extraction. It's the cheapest option, it's fast to experiment with, and it often gets surprisingly close to fine-tuned performance because pretrained representations are already very good.
• Step 4—Did feature extraction underperform, and do you have a moderate amount of labeled data (low thousands of examples)? Move to partial fine-tuning or LoRA. Try LoRA first in almost all cases—modern evidence consistently shows LoRA recovers 95–99% of full fine-tuning quality on most tasks, at a fraction of the memory footprint, and because the base weights stay frozen, you keep the original model's general capabilities largely intact (much lower risk of catastrophic forgetting).
• Step 5—Do you have a large, high-quality dataset, a real reason to believe LoRA's reduced parameter budget is a bottleneck, and a GPU with plenty of memory (or patience for a long CPU/MPS run)? Only then reach for full fine-tuning.

In practice, for the vast majority of real-world projects—including nearly everything you'll build as a developer rather than an ML research lab—the winning path is: try feature extraction first, and if that's insufficient, go straight to LoRA. Full fine-tuning is a tool for the minority of cases where you have both the data and the hardware budget to justify it, and this article will show you how to make that judgment with actual numbers rather than guesswork.

Setting Up Your Environment (Windows, Mac, or Linux)

One of the nicest things about building on the Hugging Face stack is that you write the same code regardless of platform, and the underlying libraries pick the fastest available hardware automatically.

Install Python and Create a Conda Environment

This article uses conda (https://docs.conda.io) to manage the environment and Jupyter Notebook to run the code interactively—both work identically on Windows, macOS, and Linux, which keeps the whole setup platform-agnostic. If you don't already have conda, install Miniconda first.

Create an isolated environment with Python 3.11 and Jupyter installed together:

conda create -n FineTuning python=3.11 jupyter
conda activate FineTuning

This one pair of commands is identical whether you're in Windows PowerShell/Command Prompt, macOS Terminal, or a Linux shell—one of the practical advantages of conda over venv for a platform-agnostic workflow, since it manages the Python interpreter itself rather than relying on whatever's already on your system PATH.

With the environment active, launch Jupyter:

jupyter notebook

Every code example in this article is written as a self-contained cell (or short sequence of cells) and is meant to be run directly inside a notebook under this FineTuning environment.

To deactivate the environment when you're done, or switch back to it later in a new terminal session:

conda deactivate      
conda activate FineTuning

Install PyTorch for Your Hardware

This is the one step where platform matters, and even here (Listing 1) it's a single command from the PyTorch website that installs the right build automatically detected for your OS. Run these from a terminal with the FineTuning environment active (or prefix each line with ! to run it directly from a Jupyter cell):

Listing 1: Installing PyTorce for the various platforms

# Windows / Linux with an NVIDIA GPU (CUDA 12.1 example)
pip install torch --index-url     
    https://download.pytorch.org/whl/cu121
# macOS (Apple Silicon)—MPS acceleration is built 
# into the default build
pip install torch
# Any platform, CPU-only fallback
pip install torch --index-url 
    https://download.pytorch.org/whl/cpu

You do not need to write any device-specific code beyond this install step. As you'll see below, a single helper function detects whichever accelerator is present.

Install the Rest of the Stack

With the FineTuning environment still active, install the following packages using the pip command:

pip install transformers datasets 
    accelerate peft evaluate 
    scikit-learn sentencepiece

• transformers—model architectures, tokenizers, and the Trainer API.
• datasets—loading, splitting, and preprocessing data efficiently.
• accelerate—abstracts away device placement so the same script runs on CPU, CUDA, or MPS.
• peft—Hugging Face's Parameter-Efficient Fine-Tuning library, which implements LoRA and related techniques.
• evaluate—standard metrics (accuracy, F1, BLEU, ROUGE, etc.).
• scikit-learn—handy for quick metric calculations and data splitting.

A quick sanity check worth running as your first Jupyter cell, to confirm the environment is wired up correctly before going any further:

import torch, transformers, datasets, peft

print("PyTorch:", torch.__version__)
print("Transformers:", transformers.__version__)
print("Datasets:", datasets.__version__)
print("PEFT:", peft.__version__)

For my machine (a Mac Studio), I have the following versions installed:

PyTorch: 2.13.0
Transformers: 5.13.0
Datasets: 5.0.0
PEFT: 0.19.1

A Device-Agnostic Helper

Every script in this article starts with this small helper, which detects the best available device regardless of platform:

import torch
def get_device():
    if torch.cuda.is_available():
        # Windows/Linux with NVIDIA GPU
        return torch.device("cuda") 
    if torch.backends.mps.is_available():
        # Apple silicon Mac
        return torch.device("mps")
    # Fallback, works everywhere
    return torch.device("cpu") 
device = get_device()
print(f"Using device: {device}")

If you are running this on a Mac, you should see this:

Using device: mps

From here on, every example in this article uses device instead of hardcoding cuda or mps, which is what makes the code portable. The Hugging Face Trainer API goes a step further and handles device placement for you automatically via accelerate, so in most of the examples below you won't even need to call .to(device) yourself.

A Note on Hardware Expectations

Fine-tuning is genuinely more comfortable with a discrete GPU, but you are not locked out without one:

• NVIDIA GPU (Windows/Linux): the fastest path for everything in this article, especially full fine-tuning.
• Apple silicon Mac (M1/M2/M3/M4/M5): MPS acceleration handles feature extraction, partial fine-tuning, and LoRA comfortably on models up to a few billion parameters, thanks to unified memory. Full fine-tuning of larger models is possible but slower and more memory-constrained than on an equivalent NVIDIA GPU.
• CPU only: feature extraction and small-model LoRA runs are realistic (if slow); full fine-tuning of anything beyond a small model is impractical for iteration, though fine for an overnight run on a small dataset.

Every code example in this article was chosen to be runnable, end to end, on all three of the above—with training time being the main variable.

Verifying Your GPU Is Actually Detected (Windows/Linux)

If you're on Windows or Linux with an NVIDIA GPU, it's worth confirming your driver and CUDA setup are working before installing PyTorch, since a mismatched CUDA version is a common source of confusing “no GPU found” errors later. Run:

nvidia-smi

This prints your driver version, the CUDA version your driver supports, and current GPU memory usage—for example, a line like CUDA Version: 12.5 tells you which PyTorch CUDA build to install. Once PyTorch is installed, confirm it can actually see the GPU from Python:

import torch
# should print True
print(torch.cuda.is_available()) 
# should print your GPU's name
print(torch.cuda.get_device_name(0)) 

If torch.cuda.is_available() returns False despite nvidia-smi working correctly, the most common cause is a PyTorch build installed for the wrong CUDA version—reinstall using the exact index URL matching your driver's supported CUDA version from the PyTorch website.

Preparing Your Data

Good data preparation matters more than any hyperparameter choice you'll make later. This section covers the practical mechanics of getting text data into a shape all four techniques can use.

Loading and Inspecting Data

We'll use the Hugging Face datasets library throughout, since it handles large datasets efficiently and integrates directly with Trainer. The following code snippet loads the Yelp Review Full dataset:

from datasets import load_dataset
dataset = \
    load_dataset("Yelp/yelp_review_full")
print(dataset)
print(dataset["train"][0])

The Yelp Review Full dataset is a benchmark text classification dataset introduced in the 2015 “Character-level Convolutional Networks for Text Classification” paper, containing 650,000 training and 50,000 test examples of real Yelp business reviews, each labeled with a star rating from 1 to 5 (encoded as 0-4) and perfectly balanced across all five classes. It's a step up in difficulty from binary sentiment datasets like IMDB or SST-2, since predicting the exact star rating (rather than just positive/negative) requires distinguishing between genuinely ambiguous adjacent classes, like 3 versus 4 stars, making it a good dataset for illustrating why accuracy alone can understate a model's real performance on ordinal, fine-grained classification tasks.

To load your own data instead of a public dataset, use the following code snippet:

from datasets import load_dataset
dataset = load_dataset(
    "csv",
    data_files={"train": "train.csv", 
                "test": "test.csv"}
)

Your CSV just needs consistent column names—typically something like text and label for classification tasks.

Splitting Data Properly

Never train and evaluate on the same data. If your source doesn't already have a train/test split, create one, and hold out a validation set as well so you can tune hyperparameters without peeking at your final test set:

split = dataset["train"].train_test_split(
    test_size=0.2, seed=42
)
train_val = split["train"].train_test_split(
    test_size=0.1, seed=42
)
train_dataset = train_val["train"]
val_dataset = train_val["test"]
test_dataset = split["test"]
print(
    f"Train: {len(train_dataset)}, "
    f"Val: {len(val_dataset)}, "
    f"Test: {len(test_dataset)}"
)

The above code snippet splits the dataset into training, validation, and test sets—first setting aside 20% as a held-out test set, then splitting the remaining 80% again to carve out 10% of that as a validation set, leaving the rest for training.

For the Yelp Review Full dataset, you will see the following:

Train: 468000, Val: 52000, Test: 130000

Tokenization

Tokenization converts raw text into the numerical IDs a model consumes. Always use the tokenizer that matches your chosen model checkpoint—mixing tokenizers is a common beginner mistake that silently produces garbage results.

The following code snippet loads the tokenizer for distilbert-base-uncased and applies it to both the training and validation sets, padding or truncating every review to a fixed length of 256 tokens so the batches can be fed into the model as uniformly shaped tensors.

from transformers import AutoTokenizer

model_name = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)


def tokenize_function(examples):
    return tokenizer(
        examples["text"],
        padding="max_length",
        truncation=True,
        max_length=256,
    )


tokenized_train = train_dataset.map(tokenize_function, batched=True)
tokenized_val = val_dataset.map(tokenize_function, batched=True)

A few beginner-friendly notes on the parameters above:

• truncation=True cuts off text longer than max_length, since transformer models have a fixed maximum input length.
• padding="max_length" pads shorter sequences up to that same length so they can be batched together as a single tensor. (For efficiency at scale, dynamic padding via a data collator is often preferred over fixed-length padding.)
• max_length is a tradeoff: longer sequences capture more context but cost more memory and time, roughly quadratically for the attention mechanism.

Dynamic Padding for Efficiency

Padding every example to a single fixed max_length wastes compute on short examples. A DataCollator pads each batch only as much as its longest member needs:

from transformers import 
    DataCollatorWithPadding
data_collator = \
    DataCollatorWithPadding(
        tokenizer=tokenizer)

You'll pass this collator into Trainer later and skip fixed-length padding at tokenization time (padding=False or omit it) when using this approach.

Data Quality Checklist

Before training anything, it's worth running through this checklist—problems here are the single most common cause of disappointing fine-tuning results, far more often than the choice of technique or hyperparameters:

• Label consistency: are labels applied the same way across your whole dataset? Inconsistent labeling is invisible in aggregate metrics but caps your model's ceiling.
• Class balance: if you're doing classification, check the distribution of labels. Severe imbalance may need oversampling, undersampling, or class-weighted loss.
• Duplicate leakage: make sure the same (or near-identical) examples don't appear in both train and test splits.
• Length distribution: check whether your max_length choice is truncating a meaningful fraction of your examples.
• **Minimum viable dataset size: **as a rough beginner-friendly guideline, feature extraction can work with a few hundred examples per class; LoRA and partial fine-tuning usually want at least a low thousand total examples; full fine-tuning benefits from tens of thousands or more to really outperform the lighter-weight options.

Feature Extraction: Adapting a Model Without Changing It

Feature extraction is the most under-appreciated technique in this entire article. It's the cheapest, fastest, and—for many classification and similarity tasks—surprisingly close to what full fine-tuning would achieve. It deserves to be your default first attempt, not an afterthought.

The Idea Behind Feature Extraction

You take a pretrained model, run your data through it, and keep the output representations (embeddings) from one of its internal layers—almost always the final hidden layer, or a pooled summary of it. You never update a single weight inside the pretrained model. Instead, you train a small, separate model (often just logistic regression or a tiny neural network) on top of those fixed embeddings.

With feature extraction, instead of fine-tuning, you usually train a small model—logistic regression or a tiny neural net—on top of fixed, frozen embeddings.

This works because a well-pretrained model's internal representations already encode a huge amount of general-purpose linguistic and semantic structure. Your job is just to teach a lightweight classifier to read the parts of that structure relevant to your task.

A simple analogy. Imagine hiring a talented graduate. With feature extraction, you hire them, don't retrain them at all, and simply ask them to summarize or analyze information using the general knowledge they already have—you make your own decisions based on their output. With fine-tuning, you hire the same graduate but then give them additional training specific to your company: they learn your terminology, your style, your expectations, and over time they perform better on your exact needs than a generalist ever could. Neither approach is strictly “better”—it depends on whether you need a fast, cheap generalist opinion, or a slower, more expensive specialist.

Extracting Embeddings with a Sentence Transformer

For most text classification and similarity tasks, a sentence-embedding model—trained specifically so that semantically similar sentences produce nearby vectors—is the best starting point. Let's see how this is done. First, install the sentence-transformers package:

!pip install sentence-transformers

sentence-transformers is a Python library built on top of Hugging Face Transformers, used to generate dense vector embeddings for sentences, paragraphs, or short texts.

The following code snippet loads the all-MiniLM-L6-v2 sentence-transformers model—a small, fast, CPU-friendly choice—and encodes all the review texts in train_dataset into fixed-size embeddings in batches of 32, printing the resulting array's shape as (num_examples, embedding_dim), where embedding_dim is 384 for this model:

from sentence_transformers import (
    SentenceTransformer,
)
import numpy as np
model = SentenceTransformer(
    "all-MiniLM-L6-v2"
)  # small, fast, CPU-friendly
texts = train_dataset["text"]
embeddings = model.encode(
    texts,
    show_progress_bar=True,
    batch_size=32,
)
print(embeddings.shape)
# (num_examples, embedding_dim)

all-MiniLM-L6-v2 is deliberately chosen here as a beginner-friendly starting point: it's small enough to run comfortably on CPU, and it produces high-quality general-purpose sentence embeddings.

Extracting Embeddings from Any Transformer Backbone

Sometimes you don't want to use sentence-transformers—maybe you need a model it doesn't support, like one fine-tuned on your own data, or a decoder-only model. In that case, you can load the model directly from Hugging Face's transformers library instead and build the sentence embedding yourself.

Here's the difference: all-MiniLM-L6-v2 was specially trained so that calling .encode() hands you one clean vector per sentence, ready to use. A general-purpose model like distilbert-base-uncased wasn't trained that way—it only outputs one vector per* *word, not one per sentence. So you have to combine all those word-level vectors into a single sentence vector yourself. This works, but the result usually isn't as good as an embedding from a model designed specifically for that purpose.

The code below (see Listing 2) combines the word vectors using a method called mean pooling—it simply averages the vectors for every word in the sentence into one vector, while making sure padding (the empty filler added to short sentences so they match the batch length) isn't included in that average.

Listing 2: Extracting mean-pooled embeddings from DistilBERT

from transformers import (
    AutoTokenizer,
    AutoModel,
)
import torch
model_name = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(
    model_name
)
model = AutoModel.from_pretrained(
    model_name
).to(device)
# freeze inference mode—no grads, no dropout
model.eval()  
def extract_embeddings(texts, batch_size=32):
    all_embeddings = []
    with torch.no_grad():
        for i in range(0, len(texts), batch_size):
            batch = texts[i:i + batch_size]
            inputs = tokenizer(
                batch,
                padding=True,
                truncation=True,
                max_length=256,
                return_tensors="pt",
            ).to(device)
            outputs = model(**inputs)
            # Mean-pool last hidden state across tokens
            # for a single vector per example
            last_hidden = outputs.last_hidden_state
            mask = (
                inputs["attention_mask"]
                .unsqueeze(-1)
                .expand(last_hidden.size())
                .float()
            )
            summed = torch.sum(last_hidden * mask, 
                dim=1)
            counts = torch.clamp(
                mask.sum(dim=1), min=1e-9
            )
            pooled = summed / counts
            all_embeddings.append(pooled.cpu().numpy())
    return np.concatenate(all_embeddings, axis=0)
train_embeddings = extract_embeddings(
    train_dataset["text"]
)
val_embeddings = extract_embeddings(
    val_dataset["text"]
)

Two details worth calling out:

• model.eval() switches off dropout and other training-only behaviors, which matters even though we're not training this model—we want deterministic, stable representations.
• torch.no_grad() tells PyTorch not to track operations for backpropagation, which significantly speeds up inference and reduces memory use, since we have no intention of ever computing gradients through this frozen backbone.
• Mean pooling averages the per-token representations (weighted by the attention mask, so padding tokens are excluded) into a single fixed-size vector per example. This is a simple, effective default; some models (like those trained with a [CLS] token objective) work just as well or better using only the first token's representation instead.

The end result is two NumPy arrays of sentence embeddings—one per review—ready to feed into a classifier. Specifically:

train_embeddings: shape (len(train_dataset), 768)—one 768-dimensional vector per training review (768 because that's distilbert-base-uncased's hidden size).

val_embeddings: shape (len(val_dataset), 768)—same thing, for the validation set.

Each row is a single fixed-size vector that represents the meaning of one review, produced by mean-pooling DistilBERT's per-token hidden states.

Training the Head

With embeddings in hand, training a classifier is now a completely ordinary machine learning problem—no GPU, no transformer library, needed for this step at all. As Listing 3 shows, you built a star-rating classifier, capable of detecting the sentiment expressed in a piece of text.

Listing 3: Training the feature extraction head

from sklearn.linear_model import (
    LogisticRegression,
)
from sklearn.metrics import (
    accuracy_score,
    classification_report,
)
classifier = LogisticRegression(max_iter=1000)
classifier.fit(
    train_embeddings, train_dataset["label"]
)
val_predictions = classifier.predict(
    val_embeddings
)
print(
    f"Validation accuracy: "
    f"{accuracy_score(val_dataset['label'], 
        val_predictions):.4f}"
)
print(
    classification_report(
        val_dataset["label"], val_predictions
    )
)

In practice, expect classifier accuracy from these mean-pooled DistilBERT embeddings to fall noticeably short of what you'd get from all-MiniLM-L6-v2.

With the classifier built, you can now use it to perform sentiment analysis, as shown in Listing 4.

Listing 4: Running the trained classifier on new text

new_texts = [
    "The battery life on this laptop is amazing",
    "Customer support never responded to my email",
]
new_embeddings = extract_embeddings(new_texts)
predictions = classifier.predict(new_embeddings)
probabilities = classifier.predict_proba(
    new_embeddings
)
for text, pred, prob in zip(
  new_texts, predictions, probabilities
):
    print(f"Text: {text}")
    print(f"Predicted label: {pred}")
    print(f"Confidence: {prob.max():.4f}\n")

You should see output resembling the following, though your exact confidence scores will vary:

Text: The battery life on this laptop is
amazing
Predicted label: 4
Confidence: 0.7991

Text: Customer support never responded to
my email
Predicted label: 0
Confidence: 0.9928

For a slightly more expressive head than logistic regression, a small neural network works too, and stays fast to train because it only has to learn a mapping from a fixed embedding size (e.g., 384 or 768 dimensions) to your number of classes:

import torch.nn as nn
class EmbeddingClassifier(nn.Module):
    def __init__(
        self, embedding_dim,
        num_classes, hidden_dim=128
    ):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(
                embedding_dim, hidden_dim
            ),
            nn.ReLU(),
            nn.Dropout(0.2),
            nn.Linear(
                hidden_dim, num_classes
            )
        )
    def forward(self, x):
        return self.net(x)

When Feature Extraction Is Enough

In practice, feature extraction tends to work well when:

• Your task is close in nature to what the pretrained model already does well (general semantic understanding, topic classification, sentiment).
• You have a modest amount of labeled data (hundreds to low thousands of examples).
• You need fast iteration—swapping classifiers, trying different pooling strategies, or experimenting with class balancing all take seconds, not hours, because the expensive part (embedding extraction) only needs to run once and can be cached.
• You want a lightweight, cheap-to-serve solution—a logistic regression head on top of cached embeddings is trivial to deploy compared to serving an entire fine-tuned transformer.

It tends to fall short when your task requires the model to develop genuinely new internal representations—for example, recognizing a highly specialized vocabulary the pretrained model has rarely or never seen, or performing multi-step reasoning patterns that weren't present in pretraining. That's the signal to move on to partial or full fine-tuning, covered next.

Once feature extraction hits its ceiling, the next question is how much of the backbone to unfreeze. This is a genuine spectrum, not a binary choice, and understanding the tradeoff is more useful than memorizing a rule.

Full Fine-Tuning

In full fine-tuning, every parameter in the model—every attention weight, every feed-forward layer, every embedding—is unfrozen and updated by gradient descent on your task's loss function. This gives the model maximum flexibility to reshape its internal representations around your specific task.

• Advantages: highest ceiling on task performance, especially when your task differs substantially from the pretraining distribution—no architectural constraints—every part of the model can adapt.
• Costs memory: you need to store gradients and optimizer states (e.g., Adam keeps two extra values per parameter) for every parameter, which for a model with N parameters typically means 4-6x N in memory just for training, on top of the model weights themselves.
• Risk of catastrophic forgetting: aggressively updating every weight on a narrow dataset can degrade the model's general capabilities outside your task.
• Storage: each fine-tuned checkpoint is a full copy of the model (hundreds of MB to tens of GB), which becomes expensive if you need many task-specific variants.

Partial Fine-Tuning (Layer Freezing)

Partial fine-tuning freezes most of the backbone—typically the earlier layers, which tend to capture more generic, transferable patterns—and only unfreezes the later layers (which capture more task-specific, higher-level patterns) plus the new task head.

• Advantages: Meaningfully cheaper than full fine-tuning: fewer parameters need gradients and optimizer states tracked.
• Lower risk of catastrophic forgetting, since most of the model's original knowledge is architecturally protected from being overwritten: A useful middle ground when feature extraction underperforms but full fine-tuning's cost isn't justified.
• Costs: Lower ceiling than full fine-tuning on tasks that genuinely need early-layer adaptation.
• Requires a judgment call about how many layers to unfreeze, which typically takes some experimentation.

Figure 6 shows the difference between full fine-tuning vs partial fine-tuning.

Figure 5: Two neural network diagrams side by side: full fine-tuning with all layers highlighted as trainable, and partial fine-tuning with only the last layers highlighted

Freezing Layers in Code

Listing 5 shows to freeze all but the last few transformer blocks of a Hugging Face model—the mechanics are the same regardless of model architecture, you're just setting requires_grad = False on parameters you don't want updated.

Listing 5: Freezing the layer in a model that you do not want to update

from transformers import (
    AutoModelForSequenceClassification,
)
model = (
    AutoModelForSequenceClassification
    .from_pretrained(
        "distilbert-base-uncased",
        num_labels=5,
    )
).to(device)
# First, freeze everything
for param in model.parameters():
    param.requires_grad = False
# Then, selectively unfreeze the last N
# transformer layers plus the classification head
num_layers_to_unfreeze = 2
# DistilBERT's transformer blocks live
# under model.distilbert.transformer.layer
total_layers = len(
    model.distilbert.transformer.layer
)
for layer in model.distilbert.transformer.layer[
    total_layers - num_layers_to_unfreeze:
]:
    for param in layer.parameters():
        param.requires_grad = True
# Always unfreeze the classification head —
# it's randomly initialized and must
# learn from scratch
for param in model.classifier.parameters():
    param.requires_grad = True
for param in model.pre_classifier.parameters():
    param.requires_grad = True
# Sanity check: how many parameters
# will actually be trained?
trainable = sum(
    p.numel() for p in model.parameters()
    if p.requires_grad
)
total = sum(
    p.numel() for p in model.parameters()
)
print(
    f"Trainable: {trainable:,} / {total:,} "
    f"({100 * trainable / total:.2f}%)"
)

Every model architecture names its internal layers slightly differently (model.distilbert.transformer.layer for DistilBERT, model.bert.encoder.layer for BERT, model.model.layers for many decoder-only models). A reliable way to find the right attribute name for any model is to print its structure once:

print(model)

Look for the repeated block of transformer layers in the printed module tree.

Figure 7 shows how the code model.distilbert.transformer.layer maps onto the printed module tree, tracing down from distilbert→ transformer → layer to reach the ModuleList of 6 TransformerBlock layers that the loop then iterates over to selectively unfreeze.

Figure 6: Mapping model.distilbert.transformer.layer to DistilBERT's printed module structure

Choosing How Many Layers to Unfreeze

There's no universal formula, but a sensible beginner-friendly starting point is to unfreeze the last 20–30% of the backbone's layers and evaluate from there. Practically, treat “number of unfrozen layers” as a hyperparameter—start with the head only (equivalent to feature extraction with a trainable head), then progressively unfreeze more layers, watching validation performance at each step. Diminishing or negative returns tell you when to stop.

Full Fine-Tuning Walkthrough

With the conceptual groundwork in place, here's a complete, runnable full fine-tuning example using the Hugging Face Trainer API—the same code works unmodified on Windows/CUDA, Mac/MPS, or CPU, because Trainer (via accelerate) handles device placement for you.

Loading the Model

The following code snippet loads a pretrained DistilBERT model and adapts it for classification, plus the matching tokenizer:

from transformers import (
    AutoModelForSequenceClassification,
    AutoTokenizer,
)
model_name = "distilbert-base-uncased"
num_labels = 5   # e.g., 1-5 star ratings
tokenizer = AutoTokenizer.from_pretrained(
    model_name
)
model = (
    AutoModelForSequenceClassification
    .from_pretrained(
        model_name, num_labels=num_labels
    )
)

Notice we do not manually move the model to device here—Trainer will handle that for us later.

Defining Metrics

Once your model starts training, you need a way to know if it's actually getting better—that's where metrics come in. Hugging Face's Trainer doesn't score your model's predictions on its own; instead, it expects you to hand it a compute_metrics function that it will call automatically after every evaluation pass. This function receives the model's raw predictions and the true labels, and returns a dictionary of scores you care about.

Below, you define one that reports two common classification metrics: accuracy (the percentage of predictions that were exactly right) and weighted F1 (a balance of precision and recall that accounts for class imbalance—useful since our Yelp review ratings aren't necessarily evenly distributed across all five stars):

import numpy as np
import evaluate
accuracy_metric = evaluate.load("accuracy")
f1_metric = evaluate.load("f1")
def compute_metrics(eval_pred):
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    accuracy = accuracy_metric.compute(
        predictions=predictions,
        references=labels,
    )
    f1 = f1_metric.compute(
        predictions=predictions,
        references=labels,
        average="weighted",
    )
    return {
        "accuracy": accuracy["accuracy"],
        "f1": f1["f1"],
    }

The compute_metrics is called automatically after each evaluation pass during training. Here are some of the important features of this function:

logits, labels = eval_pred—the Trainer hands back raw model outputs (logits, one score per class) and the true labels.

np.argmax(logits, axis=-1)—converts the 5 raw logit scores per example into a single predicted class (0-4), by picking the highest-scoring one.

accuracy_metric.compute(...)—the fraction of predictions that exactly matched the true label.

f1_metric.compute(..., average="weighted")—F1 score per class, then averaged across classes weighted by how many examples are in each class. This matters more than accuracy for datasets where some classes are rarer, since accuracy alone can hide poor performance on minority classes.

The function returns a dict, which is what Trainer expects—these numbers show up in your training logs and progress bars at each evaluation step.

Given this ties back to our 5-star Yelp classification setup, weighted F1 is a reasonable default since your classes are balanced—but it's worth knowing macro (unweighted average across classes) is the stricter choice if you ever want every class treated equally regardless of size.

Configuring and Running Training

With the model, tokenized datasets, data collator, and metrics function all in place, the last step is to tell the Trainer exactly how to train—things like how many passes to make over the data, how fast to learn, and when to check performance. This is handled by TrainingArguments, a settings object where each argument controls one aspect of the training loop. Once configured, we hand these settings—along with the model, datasets, and compute_metrics function from earlier—to the Trainer itself, which orchestrates the whole process. Calling trainer.train() is what actually kicks off full fine-tuning—the model's weights will be updated batch by batch, epoch by epoch, with evaluation and checkpointing happening automatically along the way.

Here's how to start the training process:

from transformers import (
    TrainingArguments,
    Trainer,
)
training_args = TrainingArguments(
    output_dir="./full-finetune-output",
    eval_strategy="epoch",
    save_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=32,
    num_train_epochs=3,
    weight_decay=0.01,
    load_best_model_at_end=True,
    metric_for_best_model="f1",
    logging_steps=50,
    report_to="none",
    # disable external experiment trackers
    # unless you use one
)
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_train,
    eval_dataset=tokenized_val,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
)
trainer.train()

Here are some of the important parts of the code:

eval_strategy="epoch" / save_strategy="epoch"—run evaluation and save a checkpoint at the end of every epoch. These two must match when using load_best_model_at_end.

learning_rate=2e-5—a small learning rate typical for fine-tuning pretrained transformers; too high and you risk destroying the pretrained weights (“catastrophic forgetting”).

load_best_model_at_end=True + metric_for_best_model="f1"—after training finishes, the Trainer reloads the checkpoint with the best validation F1 across all epochs, not just whatever the last epoch happened to produce.

weight_decay=0.01—a small L2 regularization penalty to reduce overfitting.

report_to="none"—disables logging to services like Weights & Biases or TensorBoard.

data_collator—batches and pads examples dynamically at train time.

trainer.train() is what kicks off the loop—forward pass, loss computation, backward pass, optimizer step, repeated for num_train_epochs=3 over tokenized_train, with evaluation against tokenized_val after each epoch using the compute_metrics function from before.

Saving and Reloading

Training the model (it took many hours on my Mac Studio!) is only half the job—you also want to make sure it can be saved, reloaded, and used later without needing to retrain from scratch. The code below saves both the fine-tuned model's weights and its tokenizer to a local folder, then immediately reloads them into fresh objects. This “round trip” isn't strictly necessary for using the model, but it's a good sanity check: if the reload works without errors, you know the model was saved correctly and is ready to be shared, deployed, or used for inference.

trainer.save_model("./my-fine-tuned-model")
tokenizer.save_pretrained("./my-fine-tuned-model")

from transformers import AutoModelForSequenceClassification, AutoTokenizer

# confirms that the fine-tuned model
# can be loaded
reloaded_model = AutoModelForSequenceClassification.from_pretrained("./my-fine-tuned-model")
reloaded_tokenizer = AutoTokenizer.from_pretrained("./my-fine-tuned-model")

This saves the fine-tuned model to disk, then reloads it as a fresh, standalone object.

trainer.save_model(...)—saves the model's weights (with the fine-tuned classification head included) plus its config to the given folder. Since load_best_model_at_end=True was set earlier, this saves the best checkpoint by validation F1, not necessarily the final epoch's weights.

tokenizer.save_pretrained(...)—saves the tokenizer's vocabulary and config to the same folder. This step is easy to forget, but necessary—without it, anyone loading the model later has no matching tokenizer and would have to guess or re-download the original one.

The reload step—demonstrates that the saved folder is fully self-contained. AutoModelForSequenceClassification.from_pretrained("./my-fine-tuned-model") doesn't need num_labels passed in this time, since that's now baked into the saved config from the fine-tuning step.

This pattern—save both model and tokenizer to the same directory, then reload both from that directory—is the standard way to persist a fine-tuned Hugging Face model so it can be shared, deployed, or reused later without repeating the training run.

What to Watch During Training

Keep an eye on the relationship between training loss and validation loss/metric across epochs:

• Both improving together: healthy training—keep going.
• Training loss improving, validation loss worsening: overfitting. Consider fewer epochs, more data, weight decay, or dropout.
• Neither improving: learning rate may be too low, or too high and diverging (check for NaN losses, which signal the latter).
• Sudden validation collapse on tasks the base model previously handled well: catastrophic forgetting—consider partial fine-tuning or LoRA instead, or a lower learning rate with fewer epochs.

Testing the Fine-Tuned Model

Now, to actually test the model on new text, you run inference: feed it a review, get back logits, and convert those into a predicted star rating. Here's a simple way to do that with your reloaded model and tokenizer:

import torch
# put the model in evaluation mode
reloaded_model.eval()
def predict(text):
    inputs = reloaded_tokenizer(
        text,
        return_tensors="pt",
        truncation=True,
        padding=True,
    )
    with torch.no_grad():
        outputs = reloaded_model(**inputs)
    logits = outputs.logits
    predicted_class = torch.argmax(
        logits, dim=-1
    ).item()
    return predicted_class
# try it on a sample review
sample = (
    "This is a fantastic restaurant!")
print(predict(sample))

The above code prints out 4, which is a reasonable value!

Parameter-Efficient Fine-Tuning with Low-Rank Adaptation (LoRA)

LoRA (Low-Rank Adaptation) is the flagship technique in Hugging Face's PEFT (Parameter-Efficient Fine-Tuning) library, and for most developers, it's the best default choice once feature extraction isn't enough. Rather than updating every weight in the model (as is done in full fine-tuning), PEFT methods like LoRA freeze the pre-trained weights and train only a small set of additional parameters alongside them. LoRA approximates the effect of full fine-tuning while training a tiny fraction of the parameters—often under 1-2% of the model's total parameter count—which makes it feasible on a single consumer GPU, an Apple silicon Mac, or even a capable CPU for smaller models.

PEFT (Parameter-Efficient Fine-Tuning) is a family of techniques for adapting large pretrained models without retraining every parameter—the model's original weights stay frozen, and only a small set of additional parameters gets trained. LoRA (Low-Rank Adaptation) is the most widely used PEFT technique: it injects a pair of small, low-rank matrices alongside a frozen weight matrix, learning just enough to steer the model's behavior at a fraction of the cost of full fine-tuning.

The Idea Behind LoRA

Every weight matrix inside a transformer (for instance, the matrices that compute attention queries and values) is normally updated directly during fine-tuning. LoRA's insight is that the change needed to adapt a weight matrix to a new task can often be well-approximated by a low-rank matrix—the product of two much smaller matrices. Instead of updating the full weight matrix W, LoRA freezes W entirely and learns two small matrices A and B such that the effective update is W + BA, where A and B have far fewer total parameters than W itself (see Figure 8).

Figure 7:

Figure 8: Diagram showing a frozen weight matrix W with small trainable low-rank matrices A and B added alongside it

Practically, this means:

• The original pretrained weights never change, which keeps the model's general capabilities intact and dramatically lowers catastrophic forgetting risk.
• Only the small A/B matrices need gradients and optimizer states tracked, which is why LoRA uses so much less memory than full fine-tuning.
• You can store just the LoRA adapter (often a few MB to a few hundred MB, versus a full multi-GB model checkpoint) and swap different task-specific adapters onto the same frozen base model.

A simple analogy. If full fine-tuning is rewriting an entire company's strategy from the ground up, and partial fine-tuning is updating a handful of departments, LoRA is closer to placing sticky notes on the “decision-making desk”—the attention layers—telling it how to handle your specific situation, without touching anything else about how the company runs. The company (model) still operates exactly as it always has everywhere else; only the specific decision points you've annotated behave differently.

Setting Up LoRA with PEFT

The following code snippets sets up LoRA fine-tuning—a parameter-efficient alternative to full fine-tuning where you freeze the entire pretrained model and only train small “adapter” matrices injected into specific layers:

from transformers import (
    AutoModelForSequenceClassification,
)
from peft import (
    LoraConfig,
    get_peft_model,
    TaskType,
)
base_model = (
    AutoModelForSequenceClassification
    .from_pretrained(
        "distilbert-base-uncased",
        num_labels=5,
    )
)
lora_config = LoraConfig(
    task_type=TaskType.SEQ_CLS,
    r=8,
    # rank of the low-rank matrices —
    # the key size/quality tradeoff knob
    lora_alpha=16,
    # scaling factor, conventionally
    # set to 2x the rank
    lora_dropout=0.1,
    target_modules=["q_lin", "v_lin"],
    # which weight matrices to
    # attach LoRA adapters to
)
model = get_peft_model(
    base_model, lora_config
)
model.print_trainable_parameters()

Running this typically prints something like:

trainable params: 741,893 || all params: 
    67,699,210 || trainable%: 1.0959

That's the entire story of why LoRA is so efficient: over 98% of the model's parameters are frozen and untouched. Here's what the above code snippet is doing:

• r (rank)—controls the capacity of the adapter. Common values range from 4 to 64. Lower rank means fewer trainable parameters and a smaller adapter file, but less expressive capacity. Start at 8 and increase if you're underfitting.
• lora_alpha—a scaling factor applied to the adapter's output. The conventional rule of thumb is lora_alpha = 2 * r, though this is worth treating as tunable.
• lora_dropout—standard dropout applied within the adapter during training, to reduce overfitting—typically 0.05–0.1.
• target_modules—which weight matrices inside each transformer block get an adapter attached. Attention query/value projections (q_lin/v_lin for DistilBERT; q_proj/v_proj for many decoder-only models like Llama/Qwen) are the most common and often sufficient choice; some setups also target key projections and the feed-forward layers for extra capacity.

Training a LoRA Model

Training looks identical to full fine-tuning—you still use Trainer—because PEFT's get_peft_model wraps the base model so that gradient updates automatically only flow to the adapter parameters:

from transformers import (
    TrainingArguments,
    Trainer,
)
training_args = TrainingArguments(
    output_dir="./lora-output",
    eval_strategy="epoch",
    save_strategy="epoch",
    learning_rate=1e-4,
    # LoRA typically tolerates/needs a
    # higher LR than full fine-tuning
    per_device_train_batch_size=16,
    per_device_eval_batch_size=32,
    num_train_epochs=5,
    # LoRA often benefits from a few
    # more epochs than full fine-tuning
    weight_decay=0.01,
    load_best_model_at_end=True,
    metric_for_best_model="f1",
    logging_steps=50,
    report_to="none",
)
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_train,
    eval_dataset=tokenized_val,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
)
trainer.train()

Saving and Reusing LoRA Adapters

One of LoRA's most practical advantages is that saving only stores the small adapter, not the whole model:

# typically a few MB, not GBs
model.save_pretrained("./my-lora-adapter")

To reload it later, load the original base model, then apply the adapter on top:

from transformers import (
    AutoModelForSequenceClassification,
)
from peft import PeftModel
base_model = (
    AutoModelForSequenceClassification
    .from_pretrained(
        "distilbert-base-uncased",
        num_labels=5,
    )
)
model = PeftModel.from_pretrained(
    base_model,
    "./my-lora-adapter",
)

This adapter pattern is extremely useful in practice: you can keep a single copy of a large base model on disk and swap in different lightweight adapters for different tasks, rather than storing a full model copy per task.

Merging Adapters for Deployment

For serving, it's often convenient to “bake” the adapter into the base model's weights, producing a single standalone model with no PEFT dependency at inference time:

merged_model = model.merge_and_unload()
merged_model.save_pretrained(
    "./my-merged-model"
)

After merging, merged_model behaves like an ordinary transformers model—this is the version you'd typically export for deployment.

Using the Merged Model

Here's how to run inference with the merged model—since merge_and_unload() folds the LoRA weights directly into the base model, it now behaves exactly like a normal transformers model, with no PEFT wrapper needed at inference time:

from transformers import AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
text = "The food was amazing!"
inputs = tokenizer(text, return_tensors="pt", truncation=True)

merged_model.eval()
with torch.no_grad():
    outputs = merged_model(**inputs)

predicted_id = outputs.logits.argmax(dim=-1).item()
print(f"Predicted class: {predicted_id}")

The code snippet above prints out:

Predicted class: 4

LoRA for Generative (Decoder-Only) Models

Everything so far used a classification model as the example, but LoRA works just as well—and is arguably used more often—with generative, decoder-only models like Llama, Qwen, and Mistral. These are the models behind chatbots and instruction-following assistants, and LoRA is commonly used to fine-tune them for a specific task or domain (say, teaching a model to answer questions about your company's product docs).

The good news: nothing about how LoRA works changes. You're still freezing the base model and training a small A/B pair alongside it. Only two things need to change in the code—the task type, and which layers the adapter attaches to:

# ========================================= 
# 1. Load base model and attach LoRA adapter 
# =========================================
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
)
from peft import (
    LoraConfig,
    get_peft_model,
    TaskType,
)
# a beginner-friendly, modest-sized
# instruct model
model_name = "Qwen/Qwen2.5-1.5B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(
    model_name
)
base_model = (
    AutoModelForCausalLM
    .from_pretrained(model_name)
)
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    target_modules=[
        "q_proj",
        "v_proj",
        "k_proj",
        "o_proj",
    ],
    # decoder-only models use
    # q_proj/v_proj/k_proj/o_proj
    # instead of DistilBERT's
    # q_lin/v_lin
)
model = get_peft_model(
    base_model, lora_config
)
model.print_trainable_parameters()

The above code snippet uses the same LoRA pattern as before but shifted to a decoder-only causal language model (Qwen2.5-1.5B-Instruct) instead of DistilBERT-for-classification.

A few differences worth calling out against the earlier DistilBERT example:

AutoModelForCausalLM instead of AutoModelForSequenceClassification—this loads the model's language-modeling head (predicting the next token) rather than a classification head, since Qwen is a generative model, not a classifier.

task_type=TaskType.CAUSAL_LM instead of TaskType.SEQ_CLS—tells PEFT to wire up the adapters appropriately for a causal LM task rather than a classification task.

target_modules=["q_proj", "v_proj", "k_proj", "o_proj"]—decoder-only models like Qwen use different attribute names for their attention projections (q_proj, k_proj, v_proj, o_proj) than DistilBERT's q_lin/v_lin. This ties directly back to your earlier point: every architecture names its internal layers differently, and print(model) is how you'd confirm these names for any new model. Note this example also adapts k_proj and o_proj in addition to query/value, unlike the DistilBERT example which only targeted q_lin/v_lin.

r=16, lora_alpha=32—a higher rank than the DistilBERT example's r=8. Larger causal LMs are often given a bit more adapter capacity, though this is a tunable choice rather than a fixed rule.

At this point, model is a wrapped version of Qwen with fresh, untrained LoRA matrices attached. By PEFT's default initialization, the B matrix starts at all zeros, so the adapter currently contributes nothing—the wrapped model behaves identically to the plain base model until it's trained. model.print_trainable_parameters() confirms just how small the trainable slice is: typically well under 1% of the model's total parameters.

Instruction-tuning data typically looks nothing like the labeled reviews from the classification example. Instead of a single label per example, each row is a prompt/response pair: an instruction and the response you want the model to learn to produce. Before this data can be tokenized, it needs to be formatted into the exact chat template the model expects—the special role markers and turn boundaries that tell Qwen where the user's turn ends and the assistant's turn begins. tokenizer.apply_chat_template handles this automatically, so you never have to hand-write those tokens yourself. Below, a small sample dataset stands in for your real one—swap in your actual instruction/response pairs when adapting this for your own project (see Listing 6).

Listing 6: Building and formatting the instruction-tuning dataset

# =========================================
# 2. Build and format the training data
# =========================================
from datasets import Dataset
# a small, concrete example dataset —
# replace with your real dataset
raw_dataset = Dataset.from_list([
    {
        "instruction": (
            "Summarize this review in one "
            "sentence: 'The pho was rich "
            "and flavorful, but we waited "
            "45 minutes just to order.'"
        ),
        "response": (
            "Great pho, but painfully "
            "slow service."
        ),
    },
    {
        "instruction": (
            "What star rating (1-5) fits "
            "this review: 'Cold food, "
            "rude staff, would not "
            "return.'"
        ),
        "response": "1 star.",
    },
    {
        "instruction": (
            "Extract the main complaint "
            "from this review: 'Loved "
            "the ambiance and cocktails, "
            "but the music was way too "
            "loud to hold a conversation.'"
        ),
        "response": "The music was too loud.",
    },
    {
        "instruction": (
            "Is this review positive, "
            "negative, or mixed: 'Best "
            "tacos in town, hands down. "
            "Parking is a nightmare "
            "though.'"
        ),
        "response": "Mixed.",
    },
    {
        "instruction": (
            "Suggest a one-line reply a "
            "restaurant owner could post "
            "in response to this review: "
            "'Waited an hour for a table "
            "even with a reservation.'"
        ),
        "response": (
            "We're sorry for the wait—"
            "we're working on improving "
            "our reservation system."
        ),
    },
    {
        "instruction": (
            "Summarize this review in one "
            "sentence: 'Portions were "
            "small for the price, but "
            "everything was fresh and "
            "beautifully plated.'"
        ),
        "response": (
            "Fresh, well-presented food, "
            "though portions run small "
            "for the price."
        ),
    },
])
def format_example(example):
    messages = [
        {
            "role": "user",
            "content": example["instruction"],
        },
        {
            "role": "assistant",
            "content": example["response"],
        },
    ]
    return {
        "text": tokenizer.apply_chat_template(
            messages, tokenize=False
        )
    }
formatted_dataset = raw_dataset.map(
    format_example
)

One detail worth calling out: the chat template is model-specific. The same messages list will produce different-looking formatted text depending on which tokenizer you load—Qwen's format differs from Llama's, which differs from Mistral's. If you ever swap base models, this step will silently produce different training text without any code change needed elsewhere. formatted_dataset now holds formatted chat text, but the model still can't train on raw strings—everything needs to be converted into token IDs first. This step also introduces the biggest conceptual shift from the classification example: instead of a single integer label, the labels here are the entire token sequence itself. A causal language model is trained to predict the next token at every position, so labels is simply a copy of input_ids; AutoModelForCausalLM handles the necessary shift internally during loss computation, so you don't need to do it by hand:

# =========================================
# 3. Tokenize and split the data
# =========================================
def tokenize_example(example):
    return tokenizer(
        example["text"],
        truncation=True,
        max_length=512,
    )
tokenized_dataset = formatted_dataset.map(
    tokenize_example,
    remove_columns=(
        formatted_dataset.column_names
    ),
)
# split into train and validation sets
split_dataset = (
    tokenized_dataset.train_test_split(
        test_size=0.1
    )
)
tokenized_train = split_dataset["train"]
tokenized_val = split_dataset["test"]

Two details are easy to miss here. First, remove_columns drops the original instruction, response, and text string columns after tokenizing—Trainer expects a dataset of pure tensors with no leftover string columns, and skipping this step produces a confusing crash later. Second, max_length=512 silently truncates anything longer; if your model produces oddly cut-off completions during training, this is often why.

A data collator's job is to assemble individual tokenized examples into batches, padding shorter sequences so every example in a batch is the same length. For causal language modeling, DataCollatorForLanguageModeling with mlm=False is the standard choice—the mlm=False flag is essential, since it tells the collator this is next-token prediction, not the masked-token prediction that BERT-style models like DistilBERT use during pre-training. There's also a Qwen-specific quirk to handle: many decoder-only models don't ship with a distinct pad token, since padding was never needed for pure text generation. Without setting one, Trainer will raise an error the moment it tries to batch examples together:

# =========================================
# 4. Data collator
# =========================================
from transformers import (
    DataCollatorForLanguageModeling,
)
# Qwen doesn't set a pad token by
# default; reuse the EOS token instead
if tokenizer.pad_token is None:
    tokenizer.pad_token = (
        tokenizer.eos_token
    )
data_collator = (
    DataCollatorForLanguageModeling(
        tokenizer=tokenizer,
        mlm=False,
        # mlm=False tells the collator
        # this is causal (next-token)
        # LM, not masked LM like BERT
    )
)

With the data pipeline in place, the last setup step is telling Trainer how to train. Most of these arguments will look familiar from the classification example, but two things change for generative fine-tuning. First, evaluation now tracks loss (effectively perplexity) rather than accuracy or F1—there's no compute_metrics function here, since there's no single “correct class” to score against. Second, batch size drops from the classification example's 16/32 down to 4: a 1.5-billion-parameter decoder model needs meaningfully more memory per example than DistilBERT, and 4 is a safer default for a single consumer GPU or an Apple silicon Mac. Raise it if you have more VRAM to spare.

# =========================================
# 5. Training arguments and Trainer
# =========================================
from transformers import (
    TrainingArguments,
    Trainer,
)
# for generative models, eval loss
# (perplexity) is the usual metric,
# not accuracy or F1
training_args = TrainingArguments(
    output_dir="./lora-qwen-output",
    eval_strategy="epoch",
    save_strategy="epoch",
    learning_rate=1e-4,
    per_device_train_batch_size=4,
    per_device_eval_batch_size=4,
    num_train_epochs=5,
    weight_decay=0.01,
    load_best_model_at_end=True,
    logging_steps=10,
    report_to="none",
)
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_train,
    eval_dataset=tokenized_val,
    data_collator=data_collator,
)

Before running trainer.train(), it's worth generating a response from the untrained model first—this gives us a baseline to compare against once training completes, making LoRA's effect visible rather than abstract. The generate_reply helper below wraps the full generation flow: build a chat-formatted prompt, tokenize it, generate a response, and decode only the newly generated tokens (not the echoed prompt, which generate() includes by default). add_generation_prompt=True is the key difference from the formatting step in the earlier section—instead of building a complete user-and-assistant exchange for training, it stops right after the user's turn and adds the special tokens that signal “now it's your turn to respond.”

You will see something like the following:

Before training:
"Sorry about the wait; please enjoy your 
meal while we make it hot."

Don't expect broken or nonsensical output here. Since Qwen2.5-1.5B-Instruct is already a capable instruction-following model, and the fresh LoRA adapter contributes essentially nothing to the forward pass, this “before” response will look like ordinary Qwen output—generic, but coherent. The real difference training makes will show up in style and domain-specificity, not basic fluency.

This is the step that actually updates the LoRA adapter's weights. Everything before this point was setup—architecture, data, and configuration—but no learning happens until trainer.train() runs. It reads batches from tokenized_train, computes the loss, and backpropagates only through the small number of trainable LoRA parameters, leaving the base model's weights untouched throughout.

# =========================================
# 7. Train
# =========================================
trainer.train()

Now we run the exact same prompt through the exact same model object—but the adapter's weights have changed. Reusing generate_reply from earlier makes the comparison direct: any difference in output reflects what the LoRA adapter learned during training, not a change in model architecture or prompt.

# =========================================
# 8. Test the model after training
# =========================================
# after training: same model object,
# same prompt, but the adapter
# weights have now been updated
print("After training:")
print(generate_reply(model, test_review))

As our training sample is small, you should not see much change in the response—or worse, you may see the model latching onto phrases from the one or two training examples it managed to memorize, rather than genuinely learning a general style. With a properly sized dataset of hundreds or thousands of examples, this same before/after comparison would show something more meaningful: the “after” response staying on-topic with the actual complaint, while adopting the tone, structure, and voice consistent across your training data—evidence that the adapter learned a pattern, not just a handful of answers by rote.

The final step is saving the trained adapter to disk. Because LoRA only trains a small number of additional parameters, save_pretrained here writes out just the adapter weights—typically a few megabytes—not a full copy of the 1.5-billion-parameter base model. This is what makes LoRA adapters so easy to share, version, and swap between different fine-tuned “personalities” for the same base model.

# =========================================
# 9. Save the adapter
# =========================================
model.save_pretrained(
    "./my-qwen-lora-adapter"
    # typically a few MB, not GBs
)

From here, the adapter can be reloaded later with PeftModel.from_pretrained(base_model, "./my-qwen-lora-adapter"), or permanently folded into the base model's weights with model.merge_and_unload() for deployment as an ordinary, PEFT-free model.

Taken together, this walkthrough shows the full LoRA fine-tuning loop for a generative model: attach a lightweight adapter to a frozen base model, format instruction/response pairs into the model's chat template, tokenize and train, then compare generation before and after. The toy example—just a handful of examples and a deliberately small run—was enough to prove the mechanics work, but too little data to generalize; instead, it overfit and echoed its training examples almost verbatim. With a properly sized, diverse dataset—hundreds or thousands of examples showing the same task done well and consistently—that same mechanism stops memorizing individual answers and starts learning the underlying pattern: a particular voice, structure, and tone the model applies to new inputs it's never seen before. That's the real promise of LoRA fine-tuning—not teaching a model new facts, but teaching it to consistently respond in a specific style, for a fraction of the cost of full fine-tuning.

Summary

This article walked through the full spectrum of ways to adapt a pretrained language model to a new task—not as four unrelated techniques, but as different answers to the same question: how much of the model do you let gradient descent touch?

We started with the theory needed to make the rest of the article click: what a neural network actually computes, what “parameters” and “attention” mean concretely, and how to transfer learning splits into feature extraction and fine-tuning. From there, we worked through each technique hands-on:

Feature extraction froze the pretrained backbone entirely and trained only a small new head on top of cached embeddings—the cheapest, fastest option, and often surprisingly competitive when your task is close to what the model already does well.

Partial fine-tuning unfroze just the last few backbone layers, giving the model some room to adapt while keeping most of its pretrained knowledge intact and lowering the risk of catastrophic forgetting.

Full fine-tuning updated every parameter in the model—the most powerful option, but also the most expensive, and the one most prone to overwriting what the model already knew.

LoRA froze the entire base model and injected small, trainable low-rank matrices into its attention projections, recovering most of full fine-tuning's benefit at a fraction of the trainable-parameter count—for both classification models like DistilBERT and generative, decoder-only models like Qwen.

Along the way, the decision framework from earlier in the article is worth repeating as a takeaway on its own: try prompting first, reach for feature extraction when your task is close to classification or similarity, move to LoRA when you need more adaptation than that, and save full fine-tuning for the minority of cases where you have both the data and the hardware budget to justify it. For nearly every real-world project, that path—feature extraction, then LoRA if needed—will get you further than reaching straight for full fine-tuning ever would.

What I deliberately left out of this article is prompt-based fine-tuning—leaving the model's weights untouched entirely and instead training a small set of prompt embeddings to steer its behavior. That's a different approach with its own tradeoffs, and it's where we'll pick up in part 2.