AI Foundation

The foundation models and core concepts behind today's AI tools, explained in plain language.

  1. 1.How Transformers WorkThe neural network architecture behind every modern LLM — attention, tokens, and why it scales so well.
  2. 2.What Are Tokens?The chunks of text an LLM actually reads and writes — and why they don't map 1:1 to words.
  3. 3.What Is a Context Window?The model's working memory — how much text it can actually see at once, and what happens when you exceed it.
  4. 4.What Is Multimodality?How models moved beyond text alone to understand and generate images, audio, and video.
  5. 5.Open Weight vs Closed Weight ModelsWhat it actually means when a model is "open source" — and why most flagship models aren't.
  6. 6.What Is a Mixture of Experts (MoE)?The architecture trick that lets a model have huge total capacity while only using a fraction of it per request.
  7. 7.What Is a Large Language Model (LLM)?The term behind every AI chatbot, explained: what makes a language model "large," and what that size actually buys you.
  8. 8.What Are Embeddings? How AI Represents Meaning as NumbersEmbeddings turn words, sentences, or images into lists of numbers positioned so that similar meanings end up near each other — the foundation search and RAG are built on.
  9. 9.The Attention Mechanism, Explained SimplyAttention is the mechanism that lets a model weigh which earlier words actually matter for the one it's generating next — the core idea behind the transformer architecture.
  10. 10.What Is a Neural Network? The Basics Behind Every AI ModelEvery modern AI model — LLMs included — is a neural network under the hood. Here's what that actually means, without the math.
  11. 11.What Are Parameters? Why "Billions of Parameters" MattersModel announcements always mention a parameter count — here's what that number actually represents and why it's used as a rough proxy for capability.
  12. 12.What Is Generative AI? A Plain-English Explanation"Generative AI" is the umbrella term behind the current AI boom — here's what actually separates it from earlier AI systems.
  13. 13.What Is an AI Agent? How It Differs From a ChatbotA chatbot answers what you type. An agent can take actions on its own — using tools, browsing, running code — to actually complete a task. Here's the difference.
  14. 14.What Is a Foundation Model?The term behind GPT, Claude, Gemini, and Llama — a single model trained once, then adapted for a huge range of downstream uses.
  15. 15.What Is RAG? How AI Tools Answer Questions About Your DataRetrieval-augmented generation is how an AI assistant can accurately answer questions about a document you just uploaded, without retraining the underlying model.
  16. 16.What Is a Vector Database, and Why Does AI Need One?A vector database is built to answer a question ordinary databases can't: "what's most similar in meaning to this?" — the engine behind semantic search and RAG.
  17. 17.Inference vs. Training: What Happens When You Actually Use an AI ModelTraining happens once (or occasionally); inference happens every single time you send a message. Here's the difference and why it matters for cost and speed.
  18. 18.What Is an AI Benchmark? How Models Get ComparedEvery model release comes with benchmark scores — here's what those numbers actually measure, and why they don't always predict real-world performance.
  19. 19.Zero-Shot Learning: How AI Handles Tasks It Was Never Explicitly Trained OnModern AI models can often do a task correctly with zero examples given — here's why, and how it differs from few-shot prompting.
  20. 20.API vs. Chat Interface: Two Ways to Use the Same AI ModelThe chatbot you type into and a developer's API call often hit the exact same underlying model — here's what actually differs between the two.
  21. 21.What Is an Encoder-Decoder Model?The architecture behind translation and summarization: one half reads the whole input, the other half writes the output — two specialized stacks instead of one.
  22. 22.What Is a Decoder-Only Model?The architecture behind GPT, Claude, and nearly every modern chat assistant: a single stack that reads and generates text in one continuous pass.
  23. 23.What Is a Recurrent Neural Network (RNN)?Before transformers, this was how AI processed sequences — reading one step at a time while carrying a memory forward. Here's why it was eventually replaced.
  24. 24.What Is a Convolutional Neural Network (CNN)?The architecture that made modern image recognition possible, by scanning an image for small local patterns before building up to whole-object understanding.
  25. 25.What Is a Diffusion Model? The Architecture Behind AI Image GenerationMost AI image generators work by learning to reverse noise — starting from static and gradually "denoising" it into a picture. Here's how that actually works.
  26. 26.What Is a GAN (Generative Adversarial Network)?An earlier approach to AI image generation: two networks locked in competition, one creating fakes and one trying to catch them, each getting better because of the other.
  27. 27.What Is an Autoencoder?A network trained to compress its input down and then rebuild it — a simple task that forces it to learn what actually matters about the data.
  28. 28.What Is a Small Language Model (SLM)?Not every AI task needs a massive frontier model — small language models trade some raw capability for speed, cost, and the ability to run on modest hardware.
  29. 29.What Is a Vision-Language Model (VLM)?The technology behind an AI that can look at a photo and answer questions about it — combining image understanding with a language model in one system.
  30. 30.What Is a World Model in AI?Beyond predicting the next word, a world model tries to capture how a physical or simulated environment actually works — a growing focus for AI research beyond text.
  31. 31.What Is a State Space Model? An Alternative to Transformer AttentionA newer architecture family (like Mamba) that processes long sequences without full attention's cost — a different tradeoff between speed and how context gets handled.
  32. 32.What Is Positional Encoding? How Transformers Know Word OrderAttention alone treats input as an unordered set — positional encoding is what tells a transformer which word came first, second, and so on.
  33. 33.What Is Cross-Attention? Letting One Sequence Attend to AnotherSelf-attention looks within one sequence. Cross-attention looks between two — the mechanism that lets a decoder stay grounded in a separate input, like source text or an image.
  34. 34.What Is a Residual Connection? Why Deep Networks Can Train at AllA simple architectural trick — adding a layer's input back to its output — that made it possible to train networks with far more layers than before.
  35. 35.What Is Layer Normalization? Keeping Training Stable Across Deep NetworksA behind-the-scenes technique that rescales values inside a network at each layer, keeping training stable enough for very deep models to actually converge.
  36. 36.What Is a Tokenizer? How Text Becomes Numbers a Model Can UseBefore a model can process any text, it has to be broken into tokens and converted to numbers — the tokenizer is the component that does that conversion.
  37. 37.What Is Byte Pair Encoding (BPE)? How Modern Tokenizers Are BuiltThe algorithm behind most LLM tokenizers — it learns a vocabulary of common chunks directly from data, striking a balance between whole words and single characters.
  38. 38.What Is a Vocabulary in NLP? The Fixed Set of Tokens a Model KnowsEvery model has a finite, fixed list of tokens it can ever produce — its vocabulary. Here's what determines that list and why its size matters.
  39. 39.What Is a Logit? The Raw Numbers Behind Every AI-Generated WordBefore a model picks its next word, it produces a raw score for every possible token — that score is a logit, and how it gets turned into a choice matters more than you'd think.
  40. 40.What Is Greedy Decoding? The Simplest Way an AI Model Generates TextThe simplest possible generation strategy — always pick the single most likely next token — and why it isn't always the best choice.
  41. 41.What Is Beam Search? Generating Text by Weighing Several Options at OnceInstead of committing to one token at a time, beam search keeps several candidate sequences alive in parallel — a middle ground between greedy decoding and full randomness.
  42. 42.What Is Function Calling? How an AI Model Triggers Real ActionsThe mechanism that lets an AI model do more than talk — recognizing when a request needs a specific tool, and producing a structured call to trigger it.
  43. 43.What Is Tool Use in AI? Letting a Model Reach Outside Its Own KnowledgeA model's training data is frozen at a point in time — tool use is how it reaches beyond that, calling external systems to get information or take action it couldn't otherwise.
  44. 44.What Is the Model Context Protocol (MCP)?A standardized way for AI applications to connect to external tools and data sources — meant to replace one-off custom integrations with a common, reusable interface.
  45. 45.What Is a Multi-Agent System? When Several AI Agents Work TogetherInstead of one AI agent trying to do everything, a multi-agent system splits a task across several specialized agents that coordinate toward a shared goal.
  46. 46.What Is Agent Memory? How AI Agents Remember Beyond a Single ConversationA model itself has no persistent memory between sessions — agent memory is the system built around it that lets an agent recall relevant information over time.
  47. 47.What Is an Agentic Workflow?A structured, multi-step process where an AI agent plans, acts, checks results, and adjusts — as opposed to a single prompt-and-response exchange.
  48. 48.What Is AI Orchestration? Coordinating Models, Tools, and DataReal AI products are rarely just one model call — orchestration is the layer that coordinates models, tools, and data sources into one coherent system.
  49. 49.What Is a Prompt Template? Reusable Structure Behind Consistent AI OutputInstead of writing a fresh prompt from scratch every time, most AI products fill in a reusable template — the structure behind consistent, reliable output at scale.
  50. 50.What Is a System Message? Setting the Rules Before the Conversation StartsBefore a user ever types anything, a system message can quietly set the model's role, tone, and boundaries for the entire conversation that follows.
  51. 51.What Is Structured Output? Getting Predictable Formats From an AI ModelFree-form text is hard for code to parse reliably — structured output constrains a model's response to a specific, predictable format like JSON instead.
  52. 52.What Is a Guardrail in AI? Keeping Model Behavior Within Safe BoundsA model won't always stay within intended bounds on its own — guardrails are the checks built around it that catch or block behavior that shouldn't happen.
  53. 53.What Is Prompt Injection? The Attack That Hijacks an AI Model's InstructionsA model can't always tell the difference between its real instructions and text that's just pretending to be one — prompt injection exploits exactly that gap.
  54. 54.What Is Semantic Search? Finding Results by Meaning, Not Just KeywordsTraditional search matches literal words. Semantic search matches meaning — finding relevant results even when the query uses completely different wording.
  55. 55.What Is Hybrid Search? Combining Keyword and Semantic MatchingKeyword search and semantic search each have blind spots the other covers — hybrid search runs both and combines the results for more reliable retrieval.
  56. 56.What Is Chunking in RAG? Splitting Documents for RetrievalA whole document is usually too large and unfocused to retrieve as one unit — chunking splits it into smaller pieces that can be searched and retrieved individually.
  57. 57.What Is a Knowledge Graph? Structuring Facts as Connected RelationshipsInstead of storing facts as isolated text, a knowledge graph represents them as explicit relationships between entities — a structure some AI systems use for more precise retrieval.
  58. 58.What Is Reranking? A Second, More Careful Pass Over Search ResultsInitial retrieval is built for speed across a huge dataset — reranking is a slower, more careful second pass that reorders a shortlist for the best possible accuracy.
  59. 59.What Is Cosine Similarity? The Math Behind Comparing EmbeddingsThe specific calculation most semantic search and RAG systems use to measure how close two embeddings are — and why it focuses on direction rather than magnitude.
  60. 60.What Is Nearest Neighbor Search? Finding the Closest Matches at ScaleThe general problem behind semantic search: given a query point, quickly find the most similar points among millions — without comparing against every single one.
  61. 61.What Is BM25? The Classic Keyword Ranking Algorithm Still Used in AI SearchLong before embeddings, this formula was how search engines ranked keyword matches — and it's still the workhorse behind the keyword half of most hybrid search systems.
  62. 62.What Is Context Caching? Reusing Work to Speed Up Repeated PromptsWhen a large chunk of a prompt stays the same across many requests, context caching avoids reprocessing it every single time — cutting both latency and cost.
  63. 63.What Is Groundedness in AI? Measuring Whether an Answer Is Actually SupportedA fluent, confident-sounding answer isn't the same as a correct one — groundedness measures whether a response is actually backed by real, verifiable source material.
  64. 64.What Is LLM-as-a-Judge? Using AI to Evaluate AI Output at ScaleHuman evaluation of AI responses is slow and expensive at scale — LLM-as-a-judge uses another model to score output automatically, standing in for a human rater.
  65. 65.What Is a Model Leaderboard? How Public AI Rankings Actually WorkEvery AI model seems to top some leaderboard — here's what these rankings actually measure, and why the same model can rank very differently across different ones.
  66. 66.What Is an Elo Rating for AI Models? Ranking by Head-to-Head ComparisonBorrowed from chess, Elo ratings rank AI models by how they perform in direct, blind head-to-head comparisons judged by human preference, rather than a fixed test.
  67. 67.What Is Model Evaluation? How Teams Actually Measure AI QualityBeyond a single benchmark score, model evaluation is the broader practice of systematically measuring how well a model performs — and where, specifically, it fails.
  68. 68.What Is a Golden Dataset? The Trusted Reference Behind AI EvaluationEvery evaluation needs a trusted answer key — a golden dataset is that carefully verified reference set, used to check whether a model's output is actually correct.
  69. 69.What Is a Confusion Matrix? Seeing Exactly How a Model Gets Things WrongA single accuracy number hides a lot — a confusion matrix breaks down exactly which categories a model confuses with which, revealing patterns accuracy alone can't.
  70. 70.What Is Explainable AI (XAI)? Making Model Decisions UnderstandableA model's decisions come from math most people can't inspect directly — explainable AI is the effort to make those decisions understandable to actual humans.
  71. 71.What Is Interpretability in AI? Looking Inside the Black BoxBeyond explaining individual decisions after the fact, interpretability research tries to understand what's actually happening inside a model while it works.
  72. 72.What Is Bias in AI? How Models Inherit Skewed Patterns From DataA model doesn't invent its biases from nothing — it learns them from patterns already present in its training data, often in ways nobody explicitly intended.
  73. 73.What Is Responsible AI? The Practice of Building AI Systems DeliberatelyAn umbrella term for the practices organizations use to build and deploy AI deliberately — considering fairness, safety, and accountability, not just raw capability.
  74. 74.What Is AI Content Moderation? Using Models to Filter Unsafe ContentAI systems increasingly moderate their own output (and other platforms' content) automatically — here's how that filtering actually works, and where it still falls short.
  75. 75.What Is AI Watermarking? Marking AI-Generated Content as SuchAs AI-generated content becomes harder to distinguish from human-made content, watermarking embeds a detectable signal marking something as AI-generated.
  76. 76.What Is a Deepfake? AI-Generated Fake Media ExplainedThe term behind AI-manipulated video, images, and audio convincing enough to pass as real — and why the same underlying technology has legitimate uses too.
  77. 77.What Is the EU AI Act? Europe's Framework for Regulating AIThe first major comprehensive AI-specific regulation, sorting AI systems into risk tiers with different obligations — here's the basic shape of how it works.
  78. 78.What Is an AI Red Team? Deliberately Trying to Break a Model Before LaunchBefore a model ships, red teamers deliberately try to make it fail — the adversarial testing process meant to catch problems before real users do.
  79. 79.What Is a Jailbreak in AI? Getting a Model to Ignore Its Own RulesA jailbreak is a crafted prompt designed to get a model to bypass its safety training and produce output it would normally refuse.
  80. 80.What Is Model Serving? Running a Trained Model in ProductionA trained model sitting on disk isn't useful to anyone yet — model serving is the infrastructure that turns it into something applications can actually call.
  81. 81.What Is Latency vs. Throughput in AI? Two Different Ways to Measure SpeedA system can be fast for one user and slow overall, or the reverse — latency and throughput measure two genuinely different things, and optimizing one can hurt the other.
  82. 82.What Is a GPU, and Why Does AI Depend on It?Almost every part of modern AI — training and running models — happens on GPUs, not regular computer processors. Here's the specific reason why.
  83. 83.What Is Inference Batching? Processing Multiple Requests TogetherRunning a model on requests one at a time wastes a lot of its capacity — batching groups several together to use hardware far more efficiently.