How Generative AI Works

Every AI system covered before this part has been about analyzing or classifying existing content — is this email spam, what's in this photo. Generative AI does something genuinely different: it creates new, original content. This part covers the real, core mechanism behind how.

The real, core idea: predicting what comes next

Input:  "The coffee was roasted this"
Model's real task: predict the most likely next word

Real, plausible candidates the model might consider:
  "morning"   → very likely, given the real context
  "week"      → plausible
  "purple"    → genuinely implausible in this real context

At its real, technical core, a generative language model is trained to do one specific thing extremely well: given some real text so far, predict what real text is most likely to come next. This sounds almost too simple to explain something as capable as a modern language model — but the genuine, practical power comes from doing this one real task at an enormous scale, informed by patterns learned from vast amounts of real training text.

Generating text is really this same real prediction, repeated

Step 1: "The coffee was roasted this" → predicts "morning"
Step 2: "The coffee was roasted this morning" → predicts ","
Step 3: "The coffee was roasted this morning," → predicts "and"
Step 4: "...this morning, and" → predicts "shipped"
...continuing one real step at a time

This is genuinely how a real language model generates an entire response — not by planning the whole answer in advance the way a person might draft an essay, but by repeatedly predicting one real next piece of text (a token, covered precisely in part 8), appending it, and repeating the same real prediction process with the now-slightly-longer real context. This real, repeated process is called autoregressive generation — each new prediction depends on everything generated so far.

Where the real "creativity" actually comes from

Deterministic (always picks the single most likely next token):
  produces real, repetitive, predictable text
Sampling (picks from among several real, likely candidates,
  weighted by probability): produces real, varied, more natural text

A real language model doesn't always output the single most statistically likely next token — that would produce genuinely repetitive, robotic-feeling text. Instead, real generation typically samples from among several plausible high-probability candidates, which is exactly why asking the same real question twice can produce two genuinely different (but both reasonable) answers — a real, deliberate design choice, not inconsistent behavior or a flaw.

Beyond text: the same real principle, different content types

Text generation:  predicts the next real, likely token
Image generation: predicts real, likely pixel patterns from a
  text description, through a genuinely different technique
  (diffusion, not the same next-token approach)

This series focuses specifically on text-generating large language models, since that's the real, practical focus for building applications later in this series — but it's worth knowing image generation genuinely uses a different, real underlying technique (diffusion models, progressively refining random noise into a coherent image) rather than the same next-token prediction covered above, even though both fall under the broad "generative AI" umbrella from part 1's hierarchy.

Why this is called "generative"

Discriminative AI (part 2's traditional ML examples): classifies or
  scores existing, real input — spam or not spam, cat or not cat
Generative AI: produces genuinely new, real output that didn't
  exist before the model created it

This is the real, meaningful line dividing the two — a spam classifier discriminates between two real, existing categories; a language model generates real, original text that exists only because the model produced it. Every part from here through the rest of this series builds specifically on this generative capability, applied to real, practical text-based tasks.

Next: what LLMs actually are — explained specifically for developers, covering the real, practical shape of what you're working with when you call one.