The vocabulary, without the hand-waving.

A small trained module attached to a frozen base model.

An adapter holds the parameters learned during parameter-efficient fine-tuning, separate from the base weights. Because adapters are small, many can be stored cheaply and swapped at serving time against a single copy of the base model in memory — which is what makes per-customer or per-task models economically practical.

Losing previously learned general ability while training hard on a narrow new task.

A model fine-tuned aggressively on one narrow task can lose competence it previously had, including basic instruction following. Parameter-efficient methods reduce the risk because the base weights stay frozen. Mixing some general-purpose examples into the training set and keeping the learning rate modest also help.

A saved snapshot of model weights at a point during training.

Checkpoints let training resume after interruption and let you evaluate intermediate states rather than only the final one. The last checkpoint is not always the best: if a run starts overfitting, an earlier checkpoint may score higher on held-out data, which is why checkpoints should be selected on evaluation results rather than by recency.

The maximum number of tokens a model can attend to in one request.

The context window bounds prompt plus response together. Exceeding it means truncation, usually silent, and the dropped content is often the part that mattered. Larger windows are not free: attention cost grows with length, and models frequently attend less reliably to material in the middle of a very long context.

Evaluation examples leaking into the training data, inflating measured scores.

If test examples appear in training data, the model can score well by recall rather than capability, and the evaluation stops predicting real-world behaviour. Contamination creeps in through duplicated records, near-duplicates that exact-match deduplication misses, and scraped corpora that already contain public benchmarks. Deduplicating against your eval set before training is the basic defence.

Removing duplicate and near-duplicate examples from a dataset.

Duplicates cause a model to over-weight whatever they contain and inflate evaluation scores when they straddle the train/test split. Exact-match deduplication is straightforward; near-duplicate detection, usually via hashing or embedding similarity, catches the reworded and reformatted copies that exact matching misses.

Learning directly from preferred/rejected response pairs, with no separate reward model.

DPO reformulates preference learning so the model can be optimised straight from pairs of preferred and rejected responses, skipping the reward model and RL loop that RLHF requires. In practice it is markedly simpler to run and more stable, and gets comparable results on many alignment tasks, which is why it is often the default choice now.

A numeric vector representing text so that similar meanings sit close together.

Embeddings turn text into vectors whose geometric closeness approximates semantic similarity, which is what makes search by meaning rather than keyword possible. They underpin retrieval, clustering, deduplication and recommendation. Embeddings from different models are not comparable, so re-embedding a corpus is required when changing embedding model.

One complete pass of the training process over the whole dataset.

Epoch count is one of the first hyperparameters to tune. Too few and the model has not learned the task; too many and it begins memorising the training set at the expense of generalisation. On small fine-tuning datasets the useful range is often just a few epochs, and the right number is decided by watching validation loss rather than by rule of thumb.

A repeatable test that measures whether a model does the job you need.

An eval is a fixed set of inputs with a scoring method, run identically against every checkpoint so results are comparable. Public benchmarks measure general capability; a task-specific eval built from your own data is what actually predicts whether a model will work for you. Building one early is the highest-leverage thing most teams skip.

Continuing training on a pre-trained model so it adapts to a narrower task or domain.

Fine-tuning takes a model that already learned general language structure and trains it further on a smaller, targeted dataset. It is the standard way to change how a model behaves — its format, tone, or handling of domain-specific inputs — rather than what facts it can recall. The cost is far lower than pre-training because the model starts from useful weights instead of noise.

A single-file model format designed for efficient local CPU and GPU inference.

GGUF packages weights, metadata and tokenizer into one file that local inference runtimes can memory-map and run, commonly in quantised form. It is the usual target when shipping a model to run on a laptop, a workstation or edge hardware rather than in a datacentre.

Checks around a model that constrain what goes in and what comes out.

Guardrails run outside the model — filtering or masking inputs, validating outputs against a schema or policy, and blocking or rerouting responses that fail. Because they are deterministic code rather than learned behaviour, they enforce rules that prompting alone cannot guarantee, which is why safety-critical constraints belong here rather than in a system prompt.

A training setting you choose in advance, rather than one the model learns.

Learning rate, batch size, epoch count, LoRA rank and weight decay are all hyperparameters. They are chosen before the run and are not updated by gradient descent, so finding good values means searching — manually, by grid or random search, or with an automated optimiser that proposes configurations based on earlier results.

Running a trained model to produce output.

Inference is where nearly all lifetime compute cost accumulates, since a model is trained once and queried continuously. Its economics are shaped by batching, quantization, caching and hardware choice, and the metrics that matter — time to first token, tokens per second, and cost per request — are different from the ones that matter in training.

How large a step the optimiser takes on each weight update.

Learning rate is usually the single most consequential hyperparameter. Too high and training diverges or the model forgets its pre-trained abilities; too low and it barely moves within the compute budget. Fine-tuning normally uses a much smaller learning rate than pre-training, often with a warmup period and a decay schedule.

Using a language model to score another model’s outputs.

For open-ended outputs with no single correct answer, a strong model can be prompted to grade responses against a rubric, which scales far past human review. It carries known biases — toward longer answers, toward its own style, and toward whichever response is shown first — so judge prompts need position-swapping and periodic calibration against human ratings.

Freezing the base model and training small low-rank matrices injected alongside its weights.

LoRA adds a pair of small matrices to selected layers and trains only those, leaving the original weights untouched. Because the added matrices are low-rank, the number of trained parameters is a tiny fraction of the model, which cuts memory and produces an adapter file small enough to store and swap per task. Adapters can be merged back into the base weights for serving, or kept separate so one base model can serve many variants.

A model whose trained parameters can be downloaded and run by anyone.

Open weights means you can obtain the parameters and run them on your own infrastructure — which is what enables private deployment, pinned reproducible versions and air-gapped operation. It is not the same as open-source: training data is often undisclosed and licences may restrict commercial use, so the specific licence always needs reading.

When a model memorises its training data instead of learning to generalise.

The signature of overfitting is training loss still falling while validation loss rises. On small fine-tuning datasets it can appear within a couple of epochs. The usual remedies are fewer epochs, more or more varied data, a lower learning rate, or a parameter-efficient method that constrains how much the model can change.

The family of methods that adapt a model by training a small added subset of parameters.

PEFT is the umbrella covering LoRA, prefix tuning, prompt tuning, adapters and related techniques. All of them freeze most of the model and train something small alongside it. The practical benefits are consistent across the family: lower memory, faster iteration, small portable artefacts, and less catastrophic forgetting than updating every weight.

The reinforcement learning algorithm traditionally used to carry out RLHF.

PPO updates a policy while constraining how far each step may move from the previous policy, which keeps training from collapsing. In RLHF it is the optimiser that pushes the language model toward higher reward-model scores. It works, but it is sensitive to hyperparameters and requires holding several models in memory at once.

LoRA performed on top of a 4-bit quantised frozen base model.

QLoRA keeps the frozen base model in 4-bit precision while training LoRA adapters in higher precision. Holding the base quantised is what makes large models trainable on a single smaller GPU. The trade is throughput — de-quantising on the fly costs time per step — and some quantisation error, which matters more on precision-sensitive tasks than on style adaptation.

Representing model weights at lower numeric precision to cut memory and increase speed.

Quantization stores weights in fewer bits — 8-bit or 4-bit instead of 16-bit — shrinking the model and speeding up inference. Some accuracy is lost, and how much depends on the method and the task. Post-training quantization applies it to a finished model; quantization-aware training accounts for it during training and usually preserves more quality.

Retrieving relevant documents at query time and putting them in the prompt.

RAG searches a corpus for passages relevant to the question and supplies them as context, so the model answers from provided material rather than memory. It suits knowledge that changes often or must be cited, and it lets you add or revoke a document without retraining. Its failure mode is retrieval: if the right passage is not found, a confident wrong answer follows.

The inner dimension of the low-rank matrices, setting adapter capacity.

Rank controls how much the adapter can express. Higher rank means more trainable parameters, more memory, and a higher ceiling on adaptation; lower rank is cheaper and often sufficient. Raising rank is a common reflex when training plateaus, but a plateau far more often reflects a data problem than insufficient adapter capacity.

A model trained to score responses the way human raters would.

A reward model learns from human comparisons and then stands in for human judgement during RL training, scoring candidate responses at a rate no human panel could match. Its weakness is that it can be gamed: the policy may find responses that score well without being genuinely better, a failure usually called reward hacking.

Reinforcement Learning from Human Feedback — optimising a model against learned human preferences.

RLHF trains a reward model on human comparisons between candidate responses, then uses reinforcement learning to push the language model toward responses the reward model scores highly. It shapes qualities that are easy to recognise but hard to write down as labels, such as helpfulness or appropriate refusal. It is more complex and less stable than SFT, which is why simpler preference methods have gained ground.

A tensor storage format that loads quickly and cannot execute code on load.

Safetensors was designed to replace Python pickle checkpoints, which can run arbitrary code when deserialised. It stores tensors with a simple header, supports memory-mapped loading for fast start-up, and is safe to load from an untrusted source — which is why it has become the default for published weights.

Training on labelled input-output pairs so the model imitates the demonstrated responses.

SFT is the most direct form of fine-tuning: show the model an input and the response you wanted, and update weights to make that response more likely. It is usually the first stage after pre-training and the foundation that preference methods like DPO build on. Its ceiling is set by the quality of the demonstrations — SFT faithfully reproduces whatever is in the data, including its mistakes.

The unit of text a model actually reads and generates.

Text is split by a tokenizer into tokens — often word fragments rather than whole words. Token count, not character or word count, determines context usage and per-token pricing. Tokenizers are language-dependent, so the same meaning can cost noticeably more tokens in one language than another.

The component that converts text to token IDs and back.

A tokenizer is trained alongside its model and is not interchangeable — running a model with the wrong tokenizer produces fluent-looking nonsense. When fine-tuning, the training data must be tokenized with the same tokenizer the base model uses, and inspecting the tokenized output is a cheap way to catch data formatting bugs early.

Held-out data used to measure generalisation during training.

The validation set is never trained on, so its loss indicates whether the model is generalising or memorising. It also drives checkpoint selection and early stopping. If it overlaps with training data — a common and easily-missed error — every number it produces is optimistic and the model will disappoint in production.

A store optimised for nearest-neighbour search over embedding vectors.

Vector databases index embeddings so the closest matches to a query vector can be found quickly across millions of items, usually via approximate nearest-neighbour algorithms that trade exactness for speed. They are the retrieval layer underneath most RAG systems, typically alongside metadata filtering.