Lex takes the cluttered pile from the last two episodes and rewrites it into fewer pages. The desk is tidier and competing cues are reduced, but a required detail does not survive the rewrite.
Context bloat is redundant, irrelevant, or conflicting material that crowds useful evidence on the desk. Quality degradation from noise means competing cues make the answer less reliable even when the right evidence is present. The token budget is the application's working allocation for assembled input and allowed output. Context window limits cap the material available to one model call.
Problem: Compression cleans the desk and restores focus, but the rewrite is not a perfect copy. Details are lost.
Resolution: The reader learns that context management is an application policy with trade-offs that should be visible and checked.
Summarization, truncation, and sliding windows have different trade-offs. The application should make its policy visible and preserve a path back to authoritative sources.
Section 1: Compression, rewriting the pile
When the desk is cluttered, one option is to rewrite the pile into fewer pages. This is context summarization, sometimes called compression. An application or model produces a shorter brief intended to preserve task-relevant information. That can reduce competing cues and token use, but it does not guarantee a better answer.
But the compressed version is not the original. It is a rewrite. The AI decided which details were "key" and which were not, and that judgment is not always right. A budget breakdown with three line items might become "budget was discussed." The specific numbers are gone. The context that explained why one line item was twice the others is gone. This is lossy compression: the output is shorter, but some information is permanently lost. The rewrite cannot be reversed back into the original.
Summarization is generally lossy. Conventional lossless compression can reduce bytes for storage or transport, but the original text normally must be reconstructed before a standard text model tokenizes and processes it. Reversible prompt encodings or model-specific compression schemes may reduce representation length in some systems, but they require a compatible decoder and have their own limits. The practical brief in this story is a lossy rewrite.
Section 2: Truncation, throwing away the bottom
A faster and cruder strategy is truncation: the application removes tokens or sections rather than rewriting them. The cut is not universally "the oldest messages." A system may retain the beginning, retain the end, remove a middle region, cap each document, or apply task-specific selection. Whatever is excluded is unavailable to that model call.
This matters because any cut policy can remove essential evidence. Keeping only recent turns may lose the original scope. Keeping only the beginning may lose the final decision. Per-document caps may cut a table in half. The application should expose or document its policy and, where possible, warn when content is omitted.
Truncation is mechanically simple, but it preserves nothing about excluded content. Summarization at least attempts to retain meaning, while introducing interpretation errors. Neither should silently replace source-of-record storage.
Section 3: Sliding windows, a recency policy
A sliding window is one truncation policy that retains a moving recent span as new material arrives. It is useful for streams or recency-heavy tasks, but it is not a universal behavior of chat products. Some applications use it, some summarize, some retrieve selected history, and some reject oversized requests.
The cost is predictable: material outside the retained span is absent unless another mechanism retrieves or summarizes it. Whether the user receives a warning depends on the product. A well-designed system should make omission observable rather than treating silence as the default.
Every desk-management policy has a cost. Summarization can distort or omit details. Truncation removes whatever its cut policy excludes. Sliding windows privilege recency. Selection may miss a relevant item. The engineering question is which evidence must be preserved, how users can inspect the transformation, and how the system can return to authoritative sources.
Glossary
- Compression / summarization
- Rewriting a large amount of context into a shorter version that keeps the key points. The desk becomes tidier and the spotlight can focus, but the rewrite is not a perfect copy of the original. Details are lost. | Office analogy: Lex rewrites a pile of papers into fewer, condensed pages. | Example: An AI summarizing a 10-message conversation into a two-paragraph recap before continuing.
- Lossy vs lossless compression
- A lossy summary shortens content by omitting or generalizing information. Lossless compression preserves exact data but normally must be decoded before standard text-model processing. | Office analogy: A short brief versus a sealed archive box that must be unpacked before Lex can read it. | Example: A summary drops one budget line. A zipped source file preserves it but is not itself the prompt text.
- Truncation
- Excluding tokens or sections according to an application policy, without summarizing what was removed. | Office analogy: A cutter removes a marked portion of the stack. The policy decides which portion. | Example: A service caps each retrieved document, potentially cutting off a later section.
- Sliding window
- An optional truncation policy that retains a moving recent span and excludes older material as new content arrives. | Office analogy: A frame moves along a conveyor and shows only the papers currently inside it. | Example: A streaming assistant keeps the most recent turns while older ones require retrieval or a summary.
- Context bloat
- When the desk fills with redundant, irrelevant, or conflicting material that makes useful evidence harder to use reliably. | (origin: P02-E01)
- Quality degradation from noise
- Reduced reliability when competing cues cause the wrong relationships to become influential. | (origin: P02-E02)
Try it yourself
- Ask an AI to summarize a long document, then compare the summary to the original. Identify at least one detail that was lost in the compression.
- If a product summarizes or compacts a conversation, inspect the recap when possible and restore missing constraints explicitly.
- Do not infer a truncation policy from one weak answer. Check product documentation, warnings, request logs, or token accounting when available.
