AI Foundation

How Transformers Work

The neural network architecture behind every modern LLM — attention, tokens, and why it scales so well.

What Are Tokens?

The chunks of text an LLM actually reads and writes — and why they don't map 1:1 to words.

What Is a Context Window?

The model's working memory — how much text it can actually see at once, and what happens when you exceed it.

What Is Multimodality?

How models moved beyond text alone to understand and generate images, audio, and video.

Open Weight vs Closed Weight Models

What it actually means when a model is "open source" — and why most flagship models aren't.

What Is a Mixture of Experts (MoE)?

The architecture trick that lets a model have huge total capacity while only using a fraction of it per request.

What Is a Large Language Model (LLM)?

The term behind every AI chatbot, explained: what makes a language model "large," and what that size actually buys you.

What Are Embeddings? How AI Represents Meaning as Numbers

Embeddings turn words, sentences, or images into lists of numbers positioned so that similar meanings end up near each other — the foundation search and RAG are built on.

The Attention Mechanism, Explained Simply

Attention is the mechanism that lets a model weigh which earlier words actually matter for the one it's generating next — the core idea behind the transformer architecture.

What Is a Neural Network? The Basics Behind Every AI Model

Every modern AI model — LLMs included — is a neural network under the hood. Here's what that actually means, without the math.

What Are Parameters? Why "Billions of Parameters" Matters

Model announcements always mention a parameter count — here's what that number actually represents and why it's used as a rough proxy for capability.

What Is Generative AI? A Plain-English Explanation

"Generative AI" is the umbrella term behind the current AI boom — here's what actually separates it from earlier AI systems.

What Is an AI Agent? How It Differs From a Chatbot

A chatbot answers what you type. An agent can take actions on its own — using tools, browsing, running code — to actually complete a task. Here's the difference.

What Is a Foundation Model?

The term behind GPT, Claude, Gemini, and Llama — a single model trained once, then adapted for a huge range of downstream uses.

What Is RAG? How AI Tools Answer Questions About Your Data

Retrieval-augmented generation is how an AI assistant can accurately answer questions about a document you just uploaded, without retraining the underlying model.

What Is a Vector Database, and Why Does AI Need One?

A vector database is built to answer a question ordinary databases can't: "what's most similar in meaning to this?" — the engine behind semantic search and RAG.

Inference vs. Training: What Happens When You Actually Use an AI Model

Training happens once (or occasionally); inference happens every single time you send a message. Here's the difference and why it matters for cost and speed.

What Is an AI Benchmark? How Models Get Compared

Every model release comes with benchmark scores — here's what those numbers actually measure, and why they don't always predict real-world performance.

Zero-Shot Learning: How AI Handles Tasks It Was Never Explicitly Trained On

Modern AI models can often do a task correctly with zero examples given — here's why, and how it differs from few-shot prompting.

API vs. Chat Interface: Two Ways to Use the Same AI Model

The chatbot you type into and a developer's API call often hit the exact same underlying model — here's what actually differs between the two.

What Is an Encoder-Decoder Model?

The architecture behind translation and summarization: one half reads the whole input, the other half writes the output — two specialized stacks instead of one.

What Is a Decoder-Only Model?

The architecture behind GPT, Claude, and nearly every modern chat assistant: a single stack that reads and generates text in one continuous pass.

What Is a Recurrent Neural Network (RNN)?

Before transformers, this was how AI processed sequences — reading one step at a time while carrying a memory forward. Here's why it was eventually replaced.

What Is a Convolutional Neural Network (CNN)?

The architecture that made modern image recognition possible, by scanning an image for small local patterns before building up to whole-object understanding.

What Is a Diffusion Model? The Architecture Behind AI Image Generation

Most AI image generators work by learning to reverse noise — starting from static and gradually "denoising" it into a picture. Here's how that actually works.

What Is a GAN (Generative Adversarial Network)?

An earlier approach to AI image generation: two networks locked in competition, one creating fakes and one trying to catch them, each getting better because of the other.

What Is an Autoencoder?

A network trained to compress its input down and then rebuild it — a simple task that forces it to learn what actually matters about the data.

What Is a Small Language Model (SLM)?

Not every AI task needs a massive frontier model — small language models trade some raw capability for speed, cost, and the ability to run on modest hardware.

What Is a Vision-Language Model (VLM)?

The technology behind an AI that can look at a photo and answer questions about it — combining image understanding with a language model in one system.

What Is a World Model in AI?

Beyond predicting the next word, a world model tries to capture how a physical or simulated environment actually works — a growing focus for AI research beyond text.

What Is a State Space Model? An Alternative to Transformer Attention

A newer architecture family (like Mamba) that processes long sequences without full attention's cost — a different tradeoff between speed and how context gets handled.

What Is Positional Encoding? How Transformers Know Word Order

Attention alone treats input as an unordered set — positional encoding is what tells a transformer which word came first, second, and so on.

What Is Cross-Attention? Letting One Sequence Attend to Another

Self-attention looks within one sequence. Cross-attention looks between two — the mechanism that lets a decoder stay grounded in a separate input, like source text or an image.

What Is a Residual Connection? Why Deep Networks Can Train at All

A simple architectural trick — adding a layer's input back to its output — that made it possible to train networks with far more layers than before.

What Is Layer Normalization? Keeping Training Stable Across Deep Networks

A behind-the-scenes technique that rescales values inside a network at each layer, keeping training stable enough for very deep models to actually converge.

What Is a Tokenizer? How Text Becomes Numbers a Model Can Use

Before a model can process any text, it has to be broken into tokens and converted to numbers — the tokenizer is the component that does that conversion.

What Is Byte Pair Encoding (BPE)? How Modern Tokenizers Are Built

The algorithm behind most LLM tokenizers — it learns a vocabulary of common chunks directly from data, striking a balance between whole words and single characters.

What Is a Vocabulary in NLP? The Fixed Set of Tokens a Model Knows

Every model has a finite, fixed list of tokens it can ever produce — its vocabulary. Here's what determines that list and why its size matters.

What Is a Logit? The Raw Numbers Behind Every AI-Generated Word

Before a model picks its next word, it produces a raw score for every possible token — that score is a logit, and how it gets turned into a choice matters more than you'd think.

What Is Greedy Decoding? The Simplest Way an AI Model Generates Text

The simplest possible generation strategy — always pick the single most likely next token — and why it isn't always the best choice.

What Is Beam Search? Generating Text by Weighing Several Options at Once

Instead of committing to one token at a time, beam search keeps several candidate sequences alive in parallel — a middle ground between greedy decoding and full randomness.

What Is Function Calling? How an AI Model Triggers Real Actions

The mechanism that lets an AI model do more than talk — recognizing when a request needs a specific tool, and producing a structured call to trigger it.

What Is Tool Use in AI? Letting a Model Reach Outside Its Own Knowledge

A model's training data is frozen at a point in time — tool use is how it reaches beyond that, calling external systems to get information or take action it couldn't otherwise.

What Is the Model Context Protocol (MCP)?

A standardized way for AI applications to connect to external tools and data sources — meant to replace one-off custom integrations with a common, reusable interface.