Handling Documents That Exceed the Context Window

Part 8's summarizer works correctly for one, reasonably-sized real document. This part covers what happens — and what to actually do — when the real input is genuinely too large to fit in a single request, using a real quarterly GreenDesk ticket export as the concrete example.

The real, concrete problem

A single real support ticket thread: maybe 200 real tokens
A real, full quarter of GreenDesk ticket threads: potentially
  hundreds of thousands of real tokens — genuinely exceeding even a
  large real context window (the AI Fundamentals series' own
  coverage of this exact limit)

Attempting to send this entire real document in one request produces the exact real failure mode covered in the AI Fundamentals series' context window part — the request is rejected outright, not gracefully truncated.

Real, practical chunking

def chunk_text(text: str, chunk_size: int = 8000) -> list[str]:
    words = text.split()
    chunks = []
    current_chunk = []
    current_length = 0

    for word in words:
        current_chunk.append(word)
        current_length += len(word) + 1
        if current_length >= chunk_size:
            chunks.append(" ".join(current_chunk))
            current_chunk = []
            current_length = 0

    if current_chunk:
        chunks.append(" ".join(current_chunk))

    return chunks

This real, simple word-count-based chunker splits a large real document into smaller, genuinely request-sized pieces — a real, practical approximation (word count roughly correlating with token count, per the AI Fundamentals series' own token estimation guidance), not an exact token count, but sufficient for this real, practical purpose.

The real map-reduce summarization pattern

def summarize_long_document(full_text: str) -> str:
    chunks = chunk_text(full_text, chunk_size=8000)

    # "map" step: summarize each real chunk independently
    chunk_summaries = [summarize(chunk, max_sentences=5) for chunk in chunks]

    # "reduce" step: summarize the summaries into one final, real result
    combined = "\n\n".join(chunk_summaries)
    return summarize(combined, max_sentences=5)

This real, two-phase pattern — summarize each real piece independently, then summarize those summaries together — is genuinely how large documents get condensed without ever exceeding a single request's real context window, regardless of how large the original real document actually is.

An honest, real trade-off this technique makes

Real cost: information that only makes sense in relation to
  something in a DIFFERENT chunk can genuinely get lost — a pattern
  spanning across chunk boundaries, like "this exact issue was
  reported three separate times across the quarter," might not
  survive the map step if each mention lands in a different chunk

This is worth stating honestly rather than presenting chunking as a lossless solution — it's a genuine, practical trade-off, not a perfect substitute for a model that could process the entire real document in one pass. For GreenDesk's real quarterly ticket review, this trade-off is usually acceptable; a use case genuinely requiring precise cross-document pattern detection would need a different, more sophisticated real approach beyond this series' scope.

A real, practical chunk-size decision

Larger chunks: fewer real API calls (lower cost, per the AI
  Fundamentals series' own per-token pricing), but each chunk's
  summary is coarser
Smaller chunks: more real API calls (higher cost), but each chunk's
  summary is more precise

There's no single, universally correct real chunk size — it's a genuine, practical trade-off between cost and precision, worth tuning against real, actual documents rather than picking an arbitrary number and assuming it's correct for every use case.

Applying this back to part 8's real summarizer

def summarize(text: str, max_sentences: int = 3) -> str:
    # a real, practical guard — route automatically based on
    # actual document size, rather than requiring the caller to
    # know which function to use
    if len(text.split()) > 6000:
        return summarize_long_document(text)
    # ...part 8's original, unchanged logic for shorter real text

This real, small addition makes part 8's summarize() function genuinely robust to real input of any size, automatically routing to the chunking strategy only when actually needed — the correct, real integration point for this part's technique.

Next: building a real AI text translator with Python — another genuinely distinct AI task, this time working across real languages.