AI Training

What Is RLHF?

How raw next-token predictors get turned into helpful, well-behaved assistants.

Pretraining: How Models Learn Language

The first, most expensive training stage — where a model learns to predict text from a huge unlabeled corpus.

Fine-Tuning vs Prompting

Two very different ways to get a general-purpose model to behave the way you want for your specific use case.

What Is DPO (Direct Preference Optimization)?

A simpler alternative to full RLHF for teaching a model human preferences.

AI Hallucinations: Why Models Make Things Up

Why confident-sounding wrong answers are a natural consequence of how these models are trained, not a rare bug.

What Is Model Distillation?

How providers train small, fast models that punch above their weight by learning from a larger "teacher" model.

What Is Reinforcement Learning? The Basics Behind RLHF

Reinforcement learning is how a model learns from feedback on its own actions rather than from labeled examples — the foundation of the training stage that turns a base model into a helpful assistant.

What Is Supervised Fine-Tuning (SFT)?

The training stage right after pretraining, where a model learns from curated example conversations what a good response actually looks like.

What Is Instruction Tuning? Teaching a Model to Follow Directions

Instruction tuning is what teaches a model to actually do what you ask, instead of just continuing your text in the most statistically likely direction.

What Is a Reward Model? The Judge Behind RLHF

RLHF needs a way to score responses at scale — that's the reward model's job, trained specifically to predict what a human would prefer.

What Makes a Good AI Training Dataset?

The data a model learns from shapes everything about its behavior — here's what actually separates a good training dataset from a bad one.

What Is Gradient Descent? How Models Actually Learn

The algorithm underneath every training run — a simple idea (adjust in the direction that reduces error) repeated billions of times.

What Is a Loss Function? How AI Models Know They're Wrong

A model needs a precise, numeric way to measure "how wrong was that" before it can improve — that's what a loss function provides.

What Is Overfitting? When an AI Model Memorizes Instead of Learns

A model that performs great on its training data but falls apart on anything new has a specific, well-understood problem: overfitting.

What Is Transfer Learning? Reusing What a Model Already Knows

Instead of training a new model from scratch for every task, transfer learning reuses a model's existing knowledge as a head start.

What Is Quantization? How Models Get Smaller and Faster

Quantization shrinks a model's memory footprint by storing its numbers less precisely — trading a little accuracy for a lot of speed and lower cost.

What Is Catastrophic Forgetting in AI Models?

Teaching a model something new can sometimes make it suddenly, badly worse at something it already knew — a well-known failure mode in continued training.

What Is Synthetic Data, and Why Do AI Labs Use It to Train Models?

Some of a model's training data isn't written by humans at all — it's generated by other AI models. Here's why that's become standard practice.

What Is AI Alignment? Making Models Do What We Actually Want

A model can be technically capable and still not do what people actually intend — alignment is the ongoing effort to close that gap.

What Is a Checkpoint in AI Model Training?

Training a frontier model can take months — checkpoints are the safety net (and the comparison tool) that makes a run like that manageable.

What Is Masked Language Modeling? An Alternative to Next-Token Prediction

Before decoder-only models dominated, many language models learned by filling in blanks scattered across a sentence — a genuinely different training objective.

What Is Causal Language Modeling? The Training Objective Behind Modern LLMs

The core task nearly every modern LLM is trained on: given everything so far, predict what comes next — nothing more, nothing less.

What Is Next Sentence Prediction? An Older Auxiliary Training Task

A secondary training task from earlier language models, aimed at teaching a model to judge whether two sentences actually belong together.

What Is the Adam Optimizer? The Default Choice for Training Neural Networks

Nearly every modern neural network is trained using this specific optimization algorithm — here's what it actually does differently from plain gradient descent.

What Is AdamW? The Refinement Behind Most LLM Training Runs

A small but consequential fix to the Adam optimizer, correcting how it interacts with weight decay — now the more common choice for training large models.

What Is Stochastic Gradient Descent (SGD)?

The original, simplest version of the algorithm that trains neural networks — updating parameters based on small random batches instead of the entire dataset at once.

What Is Momentum in Optimization? Smoothing Out Noisy Training Updates

A technique that helps training keep moving consistently in a good direction, rather than reacting jerkily to every noisy, individual update.

What Is a Learning Rate Schedule? Changing How Big Each Training Step Is Over Time

The learning rate rarely stays fixed through an entire training run — a schedule deliberately changes it over time, and getting that schedule right matters a lot.

What Is a Warmup Period in Training? Starting Slow on Purpose

Most training runs deliberately start with a very small learning rate and ramp it up gradually — here's why jumping straight to full speed tends to backfire.

What Is Weight Decay? A Simple Technique to Discourage Overfitting

A small, constant nudge applied during every training step, gently pulling parameters toward smaller values — a simple but effective defense against overfitting.

What Is Gradient Clipping? A Safety Cap on Training Updates

A simple safeguard that caps how large any single training update is allowed to be, preventing one bad batch from derailing an entire training run.

What Is Batch Normalization? Stabilizing Training Using Batch Statistics

An earlier normalization technique that stabilizes training by rescaling values based on statistics computed across the current batch — with a key limitation for language models.

What Is a Scaling Law in AI? Predicting Performance Before Training

A set of empirical patterns showing that model performance improves predictably as data, parameters, and compute increase together — letting labs plan huge training runs in advance.

What Is Compute-Optimal Training? Balancing Model Size and Data

For a fixed training budget, there's a specific balance between model size and data volume that gets the best result — and it's not always "bigger model."

What Is Distributed Training? Training a Model Across Many Machines at Once

A frontier model is far too large and slow to train on a single machine — distributed training spreads the work across thousands of machines working together.

What Is Data Parallelism? Splitting the Data Across Machines

One of the core strategies for distributed training: give every machine a full copy of the model, but split the training data between them.

What Is Model Parallelism? Splitting the Model Itself Across Machines

When a model is too large to fit on a single machine, model parallelism splits the model itself across multiple machines instead of just the data.

What Is Pipeline Parallelism? Splitting a Model Into Sequential Stages

A specific form of model parallelism that splits a network into sequential stages across machines, keeping every machine busy at once instead of idle.

What Is Mixed Precision Training? Speeding Up Training With Lower Precision Numbers

Using lower-precision numbers for parts of training can dramatically speed things up — mixed precision training does this selectively, without sacrificing overall accuracy.

What Is Gradient Accumulation? Simulating a Larger Batch Size

When hardware memory can't fit a large batch all at once, gradient accumulation simulates one anyway by combining several smaller batches' worth of gradients.

What Is LoRA Fine-Tuning? Adapting a Model Without Retraining Everything

A technique that fine-tunes a model by training a small set of new parameters instead of touching the original ones — dramatically cheaper than full fine-tuning.

What Is Parameter-Efficient Fine-Tuning (PEFT)?

The broader category of techniques that adapt a large model by training only a small fraction of its parameters — of which LoRA is the best-known example.

What Is an Adapter Layer? Inserting Small Trainable Modules Into a Frozen Model

A parameter-efficient fine-tuning approach that inserts small new trainable layers directly into a frozen model, rather than modifying its existing weights.

What Is Prompt Tuning? Learning a Prompt Instead of Writing One

Instead of hand-crafting a prompt, prompt tuning learns a set of continuous input values automatically, optimized specifically to steer the model toward a target task.