Context Engineering Manual

Which meter is running right now, and does it look like the one I think it is?

In one line: Two of the three meters bill by the token, and only one of them looks like it does.

This is where most people get the money wrong. It is the direct answer to the $200 Later story that opens this manual, and it is why that chapter comes first.

One: the plan allowance. A subscription buys access inside an application for a flat monthly fee. The vendor meters it as sessions, rolling windows, weekly caps, or some blend the product decides, and it resets on a schedule. You do not pay per token and you generally cannot see per-token accounting. When you exhaust it, you can wait for the reset, upgrade the plan, or buy usage credits (How do usage and length limits work?).

Two: usage credits. A prepaid balance that lets a paid plan keep working past its included limit, billed at standard per-token rates (Manage usage credits for paid Claude plans). This is the hybrid, and it is the one that surprises people. It lives inside the consumer app, so it feels like part of the subscription. It bills like the developer interface, so it behaves nothing like the subscription. Everything in Part Three about token accounting applies to your credits balance, even if you have never written a line of code.

Three: the application programming interface, usually shortened to API. Per-token billing from the first token, by category: new input, cached input, output, and sometimes reasoning, plus per-call charges for certain tools. No included allowance to exhaust.

Read the controls literally

The settings that govern meter two deserve a slow read, because their names are misleading in a specific way:

Only one of those four is a brake, and it is the one shaped like a convenience toggle. I had set three of them and switched the brake off.

Advice about one meter does not automatically transfer to another. Parts One and Two are mostly about the plan allowance. Part Three is mostly about per-token billing, which means it applies to meter two as much as meter three. Where a technique crosses over, it says so.

What is sitting on the desk when the model answers me?

In one line: Your question is only one item on the model's desk.

A model does not read words the way people do. It breaks text and other input into processing units called tokens. A token might be a whole word, part of a word, a punctuation mark, or another encoded unit. Images, audio, documents, tool descriptions, and system instructions you never see also consume capacity, even when the product reports their accounting differently.

Exact token counts matter when you are paying by the token, and providers expose counting endpoints for that purpose (Claude token counting). For everyday subscription work, the more useful question is the one at the top of this chapter. What is on the desk? The answer may include:

• your current request
• earlier messages in the conversation
• uploaded documents, images, and spreadsheets
• project instructions and saved project knowledge
• search results and retrieved passages
• descriptions of the tools available
• results those tools returned
• the answer the model is producing
• internal reasoning, when the product exposes or meters it

Think of the context window as a desk. A bigger desk holds more material. Covering every inch of it with paper does not make the work better.

If you ask an analyst to review the current budget, handing over one approved workbook and a clear question is helpful. Handing over seven workbooks named final, final2, final-revised, and final-use-this-one is not more helpful. It is a small records-management crisis, and the analyst will spend the first hour on archaeology.

A model has the same problem. It can process a remarkable amount of material. It cannot know which version survived the meeting unless you tell it.

Four terms worth learning once

Input is everything the system receives, which is more than the sentence you just typed.

Output is what the model produces for you.

Context is the information available to the model for the current response.

Capacity is the practical amount of model access you have under your plan, your rate limits, your context window, and your tool allowances.

That does not mean prompts should be short. A complete 500-word instruction often beats a vague 20-word request that produces three failed drafts. The question is not how short you can make it. The question is what information raises the chance of a correct result.

If the model can hold a million tokens, why does it forget what I said forty turns ago?

In one line: A context window is a ceiling, not a target, and the reason is measurable.

The symptom you have already seen

Vendors compete partly on how much their models can accept. Anthropic's help documentation notes that context window size depends on the model, with the newest models on paid plans supporting up to 1M tokens and others 500K or 200K (How do usage and length limits work?). Large windows are valuable. They make it possible to analyze long documents, large codebases, and complex histories.

But accepting information and using it reliably are not the same thing.

Long contexts dilute the important material, bury current instructions beneath stale ones, raise latency, and make failures harder to diagnose. If a long conversation starts confidently repeating a requirement you rejected several days ago, the model is not being stubborn. The old requirement is still in the room.

Context rot

The measured version of this has a name. Chroma's technical report, Context Rot: How Increasing Input Tokens Impacts LLM Performance, evaluated 18 models including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, and found that models do not use their context uniformly. Reliability decreases as input length grows, and it does so even on simple retrieval and text-replication tasks (Context Rot).

Anthropic characterizes the effect as a performance gradient rather than a hard cliff. Models stay highly capable at long contexts but show reduced precision for retrieval and long-range reasoning compared with their performance on shorter ones (Effective context engineering for AI agents). The product documentation carries the same warning, stating plainly that more context is not automatically better (Claude context windows).

Use signal, not volume

The useful question is not "how much can this model hold?" It is "what is the smallest complete set of material that gives this task a fair chance?"

For a policy summary, that might be the current policy and the audience. For a proposal review, the proposal, the controlling requirements, and the decision criteria. For a long research effort, a compact state brief plus links to evidence that can be reopened when needed.

The desk can be large. Good operators still clear it.

Why this happens, if you want the mechanism

You can skip this section and lose nothing practical.

Anthropic describes context as a finite resource with diminishing returns, and describes models as having an attention budget that every additional token draws down. The reason is structural. These models are built on the transformer architecture, in which every token can attend to every other token, producing n squared pairwise relationships for n tokens. As the context gets longer, the model's ability to hold all those relationships gets stretched thinner. Models are also trained on distributions where short sequences are far more common than long ones, so they have less practice with long-range dependencies (Effective context engineering for AI agents).

What am I actually trying to minimize, and is it the thing I have been measuring?

In one line: A cheap answer you cannot use is not cheap.

Token minimization sounds efficient because tokens are measurable. It is also an incomplete goal.

A short, vague request may produce a weak result that needs several rewrites. A stronger model may cost more per attempt but succeed the first time. A long answer may be unnecessary. A short answer may omit the one fact the decision requires. If a cheap call fails and has to be repeated four times, it costs more than one well-designed call to a stronger model, and that is before you count your own time.

The right unit is accepted work: output that clears the real quality threshold and is actually used. Not output that looks finished. Output you used.

Efficiency = useful accepted work / AI capacity consumed

That formula includes more than model usage. It includes your time, the reviewer's time, retries, corrections, and the cost of discovering too late that the answer was polished and wrong.

The ten beginner rules

Seven that apply to almost any task:

1. Keep one conversation focused on one body of work.
2. State the outcome, the audience, the constraints, and the format you want.
3. Upload only files that can change the answer.
4. Ask for the amount of output you will actually use.
5. Use deeper reasoning only when the task warrants it, not because the control exists.
6. Correct the defective section instead of ordering another full rewrite.
7. Verify factual output before you rely on it.

Three more that pay off once the work gets longer:

1. Store reusable project information in a project or notebook when the product supports one.
2. Start a new conversation when the subject, the audience, or the governing instructions materially change.
3. Before leaving a long conversation, capture decisions, current state, unresolved issues, and next actions.

Fix the defect, not the document

Weak correction:

Try again. Make it better.

Better correction:

Keep the structure and the opening. The recommendation section is too
generic and lacks a decision criterion. Replace only that section.
Compare the options by cost, risk, and time.

The second instruction preserves the good work, names the defect, and defines the correction. The first instruction buys another lottery ticket.

Does this task actually need the strongest model, or do I just feel safer using it?

In one line: Match the tool to the difficulty before the third retry, not after it.

Not every task deserves the largest model, the deepest reasoning setting, every connected tool, and an answer long enough to need its own table of contents.

Classify the task first

Do not spend frontier capability on routine formatting. Do not force a small model through work it keeps failing. Move up or down based on observed quality, not on which model sounds most serious.

Control the response length

Output consumes capacity too. Ask for what you will use:

Give me a five-bullet decision brief. Each bullet should be one sentence.
End with the decision required.

That beats requesting a "comprehensive analysis" when what you need is an opening for tomorrow's meeting.

What to do when you are near a limit

1. Stop exploratory prompting and name the deliverable.
2. Ask for a compact state summary if the conversation is long.
3. Move routine edits to a faster or lower-capacity model if one is available.
4. Remove files and background that cannot change the answer.
5. Request shorter outputs.
6. Finish one coherent task before opening three more.
7. Wait for the plan's reset instead of buying capacity reflexively.
8. If you turn on usage credits to finish something, treat it as a decision with a number attached, not a setting. Check the auto-reload state, not just the spend limit. That is the step I skipped.
9. Upgrade only when an efficient workflow still hits a real constraint.

Anthropic's own usage guidance says message length, attachment size, conversation length, tool use, model choice, and artifact activity all affect plan usage (Claude usage-limit best practices). verify current The exact allowance varies by plan and demand, so treat the product's current usage indicator as the operational source of truth, not any number in this manual.

Is this conversation still cheaper to continue than to rebuild?

In one line: Continuity helps until history begins arguing with the present.

The choice is not "new chats are good" or "new chats are wasteful." Both can be true on the same afternoon. A clean continuing conversation saves the cost of rebuilding context. A contaminated one charges rent for every obsolete instruction it carries forward.

Continue the current conversation when

• the goal is unchanged
• earlier decisions still govern the work
• the same files are still relevant
• the next request is a direct continuation
• the thread is still coherent

Start a new conversation when

• the goal or the audience changes
• old instructions conflict with the new task
• the transcript is mostly abandoned exploration
• the model keeps confusing earlier requirements with current ones
• you want a clean comparison or an independent review
• a milestone is finished and a compact handoff can carry the state forward

The immortal chat

The immortal chat begins innocently. It knows the project. It remembers the early discussion. It has all the files.

Several weeks later it also contains rejected options, outdated facts, three audiences, two deliverables, and the ghost of a strategy nobody approved. It knows everything except what still matters.

Hand off state, not transcript

Before leaving a long conversation, ask for a continuation brief:

Create a continuation brief containing:
1. objective and definition of done
2. approved requirements
3. decisions and rationale
4. controlling sources and versions
5. completed work and verification status
6. unresolved issues and assumptions
7. the exact next action
8. actions that require approval

Exclude rejected brainstorming. Then stop.

The transcript is the argument. The brief is the conclusion.

Projects are shelves, not memory

A project earns its keep when work shares standing instructions, reference files, and recurring goals. It is not a reason to upload everything you own. ChatGPT Projects group chats, files, and project instructions (ChatGPT Projects). Claude Projects provide a knowledge base and shift to retrieval automatically when project knowledge gets large (Claude Projects).

Use separate projects when the governing instructions, the confidentiality boundary, the audience, or the source library differs. One project per body of work, not one project per person.

If two of these files disagree, which one would I defend in a meeting, and have I told the model?

In one line: An unlabeled pile of files is not evidence. It is furniture.

Upload with a reason

For every file, finish this sentence:

The model needs this file because it contains ________ that could change the answer.

If you cannot fill in the blank, leave the file out for now.

Tell the model what job each source has

Use policy.pdf as the controlling source for requirements.
Use proposal.docx as the draft under review.
Use voice-samples.md only for tone.
Treat archived-notes.docx as historical, not current policy.
If the sources conflict, identify the conflict instead of choosing silently.

A model cannot infer that the signed policy defeated the attractive slide deck in last Tuesday's meeting. Without an authority order, it may blend them, split the difference, or choose whichever sentence appears most convenient.

Split, summarize, or keep whole

Split a file when only one section is relevant and it divides cleanly. Summarize when the decisions matter but the wording does not. Keep the full source when exact language, tables, citations, or cross-section relationships matter.

Retrieve instead of stuffing

Retrieval means finding the relevant fragments at the moment of the request instead of loading an entire library into every prompt. It suits large or changing collections. It also adds a failure mode: the system can retrieve the wrong passage or miss the right one entirely. Whenever the answer depends on retrieved evidence, ask for citations or source locations so a person can check.

Point at things instead of pasting them

This is the consumer version of a technique that has become standard in agent design. Rather than pre-loading everything, keep lightweight identifiers such as file paths, links, or saved queries, and load the actual content only when the task calls for it. Anthropic describes this as a just-in-time approach and notes that the metadata of the reference itself carries signal: a file named test_utils.py in a tests folder implies something different from the same filename in src/core_logic/. Folder hierarchies, naming conventions, and timestamps help both people and models decide when something is worth opening (Effective context engineering for AI agents).

You already do this. You do not memorize your filing cabinet. You remember which drawer has the contract.

A worked example

The better request is not successful because it is longer. It succeeds because every file has a job, the authority is clear, the audience is known, and the boundary is explicit.

Which of my AI habits have I actually tested, and which just sounded reasonable?

In one line: Most expensive AI habits come from a belief that sounded right.

Myth: Shorter prompts are always better. Complete prompts reduce retries. Remove irrelevant detail, not necessary detail. Anthropic makes the same point about system prompts: minimal does not mean short, it means the smallest set of information that fully describes the expected behavior (Effective context engineering for AI agents).

Myth: The biggest context window is the best model. Context size is one capability among quality, latency, tools, cost, and reliability. See Chapter 3.

Myth: Always use the smartest model. Use the least intensive model that reliably clears the quality threshold.

Myth: Maximum reasoning means maximum quality. More reasoning adds cost and delay without improving simple work, and it encourages over-analysis when the success criteria are weak.

Myth: Never start a new chat. Continuity is useful right up until stale context becomes interference.

Myth: Starting a new chat always saves capacity. Rebuilding all the relevant context can cost more than continuing a clean conversation. Both directions have a wrong answer.

Myth: Projects remember everything perfectly. Project behavior depends on the product, the plan, the settings, the retrieval, and the quality of what you put in. A project is a shelf. It is not a memory.

Myth: A big context window means the model read all of it carefully. It accepted all of it. Recall degrades as input grows, measurably (Context Rot).

Myth: Free plans cannot do serious work. Bounded tasks with good instructions produce real value on free tiers. Paid tiers mainly buy capacity, tools, speed, and access. Identify the constraint before you buy the cure.

Myth: If the answer is wrong, ask for a complete rewrite. Diagnose the failure instead. It is usually a missing source, an unclear criterion, the wrong audience, or a defective output contract.

If you stop here

That is the foundation. Three things to do tomorrow:

1. Before your next big task, write the five-line inventory from Chapter 10: goal, controlling sources, constraints, known decisions, required output.
2. Delete one file from a project that cannot change any answer.
3. The next time a result is wrong, name the failure before you retype the prompt.

Part Two is for when the same work comes back a second time. Part Three is for when you are billed per token, which includes usage credits. Neither is a prerequisite for the other.

Why am I retyping my job title every morning?

In one line: Put every piece of context where its useful life belongs.

This chapter does three things in order. It names the five places context can live. It explains how detailed the instructions in each place should be. Then it carries one real task through all five, so you can see where your own material would land.

Context engineering is the deliberate design of what the model sees while doing a task. Anthropic frames it as the natural progression of prompt engineering: prompt engineering asks what words to use, context engineering asks what configuration of context is most likely to produce the behavior you want (Effective context engineering for AI agents).

The five layers

Persistent Context
        ↓
Project Context
        ↓
Retrieved Context
        ↓
Task Context
        ↓
Output Contract

Persistent context. Rules that apply broadly: communication preferences, safety boundaries, default units, organizational policy, stable role expectations. Keep this layer short. A permanent instruction that matters twice a year should not quietly accompany every grocery list, architecture review, and email draft.

Project context. What one body of work shares: objectives, audience, glossary, controlling sources, voice examples, decision rights, current constraints.

Retrieved context. Evidence selected from a larger collection at the moment it becomes relevant. Narrow enough to focus the model. Traceable enough to verify.

Task context. The current assignment: what changed, what must be produced, which inputs apply, what is out of scope, what needs approval.

Output contract. The acceptance conditions: format, length, structure, audience, factual standard, citation method, definition of done.

The prompt you actually type should usually be the delta, meaning the new information this task requires. Stable material belongs in stable layers. If you are retyping your biography, glossary, and source hierarchy every morning, the problem is not your typing speed. The information is living in the wrong place.

The right altitude

Anthropic describes a Goldilocks zone for instructions. At one extreme, engineers hardcode brittle if-else logic to force exact behavior, which becomes fragile and expensive to maintain. At the other, they give vague guidance that assumes shared context the model does not have. The useful altitude is specific enough to guide behavior and flexible enough to leave the model good heuristics. Their advice on getting there: start with a minimal instruction on the best model available, then add instructions and examples based on the failure modes you actually observe, rather than preemptively (Effective context engineering for AI agents).

That last part is the discipline. Most bloated instructions are a scar from a failure that happened once.

One task, all five layers

A district analyst has to turn a quarterly enrollment file into a board memo. Here is where each piece goes.

The prompt they type is the Task row and the Contract row. Everything above those was decided once. If they are retyping the definition of "enrolled" every quarter, it is in the wrong layer.

For each thing I am about to include, what is it for, and when does it expire?

In one line: Decide what goes in by deciding what it is for.

Classify before including

A practical context inventory

Before a large task, write five lines:

Goal:
Controlling sources:
Constraints:
Known decisions:
Required output:

Then list the candidate files and remove anything that cannot change the result. This takes two minutes and routinely removes half the pile.

Examples are compressed specifications

One good example communicates tone and structure more efficiently than three paragraphs of description. Anthropic advises curating a diverse set of canonical examples rather than stuffing in a laundry list of edge cases, and describes examples as the pictures worth a thousand words for a model (Effective context engineering for AI agents).

Use examples to establish a pattern. Do not use them to smuggle last quarter's facts into this quarter's work.

Context decay

Information ages at different rates. Brand voice may hold for years. Model names, plan limits, prices, and regulations may not hold for a quarter. Put a verification date next to volatile material and set a review interval proportional to the risk.

This manual practices what it describes. Every product claim carries a date and a link, and the front matter says which parts will go stale first.

Before I touch the prompt, which of these six things actually went wrong?

In one line: Most bad answers are context failures, not knowledge failures, and they fail in recognizable ways.

These six names are mine. The underlying behaviors are not controversial, but do not go looking for this exact taxonomy in a vendor document. They are familiar to anyone who has watched an AI produce a beautiful answer to a different problem.

1. Starvation. The model lacked something it needed. Symptom: a confident, generic answer that would fit any organization. Fix: add the specific missing source, fact, or criterion.

2. Dilution. Too much material, so the signal thinned out. Symptom: it gets details wrong that are demonstrably present in the context. This is the measured effect from Chapter 3 (Context Rot). Fix: remove, summarize, or retrieve instead of stuffing.

3. Clash. Two sources disagree and nothing said which one controls. Symptom: the answer splits the difference, or picks one silently. Fix: state authority explicitly.

If the sources conflict, use the signed policy as controlling.
Treat the slide deck as explanatory, not authoritative.
List unresolved conflicts instead of choosing silently.

4. Contamination. A wrong fact entered early, was never corrected, and is now being reused as established. Symptom: an error that keeps reappearing after you fixed it. Fix: correct it at the source and restart the thread. Correcting it in turn 40 does not remove it from turns 1 through 39.

5. Drift. The context was accurate when it was written and no longer is. Symptom: last year's price, last reorganization's approval chain, a retired product name presented as current. Fix: verification dates, per Chapter 10.

6. Overload of options. Too many tools, connectors, or capabilities with overlapping purposes. Symptom: the model picks a plausible wrong tool. Anthropic names bloated tool sets as one of the most common failure modes and offers a clean test: if a human engineer cannot definitively say which tool should be used in a given situation, an agent cannot be expected to do better (Effective context engineering for AI agents). Fix: turn things off.

Why diagnosis matters

The fixes point in opposite directions. Starvation needs more context. Dilution needs less. Clash needs authority. Drift needs fresh evidence. Overload needs fewer choices.

"Make the prompt better" is not a diagnosis.

Work down the list in order. Starvation and clash account for most of what people blame on the model.

If this conversation vanished tonight, what would I have lost that was not written down anywhere else?

In one line: The chat is a workspace. Something else has to be the record.

Work in phases

A strong long-running workflow separates:

1. discovery
2. requirements
3. plan
4. execution
5. verification
6. handoff

At each milestone, compress. Preserve decisions, governing evidence, open questions, and current state. Discard the ideas you rejected, which is most of what a long transcript actually contains, unless the rejection itself matters.

The continuation brief

• objective and definition of done
• controlling instructions and sources
• decisions already approved, with rationale
• completed work and verification status
• unresolved questions
• risks and assumptions
• the exact next step
• actions that require authorization

Branch for independent thinking

Use a separate conversation for a critique, a competing strategy, or a fresh review. A reviewer that watched the entire original debate inherits the original assumptions. Give it the deliverable, the acceptance criteria, and the sources, then ask it to find defects. Do not give it the speech explaining why the current answer is probably right.

Durable state belongs outside the chat

For consequential work, the current specification, decision log, checklist, or source file lives in a repository, a document, or a task record. The conversation holds the working state. It should not be the only place a project's controlling decisions exist.

This is the same boundary that agent designers now build deliberately. A conversation is working memory. A file is the record.

If the model handed me exactly what I asked for, would I accept it, and how would I know?

In one line: Many expensive failures begin with a quality standard nobody wrote down.

A complete output contract

Deliverable: one decision memo
Audience: executive team
Length: 700 to 900 words
Structure: recommendation, evidence, risks, next action
Sources: use only the attached policy and data table
Citations: cite page or row near every factual claim
Quality bar: recommendation must identify owner, deadline, and tradeoff
Boundary: assess and report; do not send, publish, or modify systems

The boundary line matters more as tools get more capable. Say what the model may do, not only what it should produce.

Ask for uncertainty

Require the model to distinguish:

• verified fact
• reasonable inference
• assumption
• missing information

This does not make the model correct. It makes unsupported certainty easier to see. It converts a polished guess into a labeled guess, which is a different thing entirely.

Contract failures look like knowledge failures

When output is wrong in a way that feels stubborn, check the contract before you blame the model. A recommendation with no owner and no deadline is not a model that failed to reason. Often the request never asked for an owner or a deadline. The acceptance criteria stayed in the user's head and emerged one complaint at a time.

The model cannot satisfy a definition of done that nobody wrote down.

Which product handles the kind of context I need, and which handles my boundaries?

In one line: The context problem is the same everywhere. The controls are not.

The side-by-side version of this chapter is the Platform Optimization Cheat Sheet in Reference. Read this for the reasoning, use the table for the comparison.

ChatGPT

Use a regular chat for a bounded task. Use a Project when chats, files, and instructions belong to one continuing body of work (ChatGPT Projects). Keep project instructions stable and specific. Open a fresh chat inside the project for each new deliverable rather than stretching one thread across unrelated outputs.

Use stronger reasoning for decisions, hard synthesis, and ambiguous constraints. Use standard modes for drafting, summarizing, and routine transformation. When you need current facts, use web research and insist on links near the claims.

Claude

Claude is well suited to long-form writing, close reading, and sustained work over organized source material. Projects are valuable for content reused across conversations. Anthropic states that reused project content can benefit from caching, and that large project knowledge on paid plans can shift to retrieval (Claude usage-limit best practices, Claude Projects).

Do not keep one Claude conversation forever merely because it has history. Capture state at milestones. Start fresh when old exploration begins to compete with current instructions.

Gemini

Gemini is useful when very large files or genuinely multimodal material belong in one analysis. A large context window is capacity, not an invitation. Use saved instructions or Gems for reusable behavior, and keep current-task directions in the current request.

Google notes that more advanced models and higher thinking levels consume more of a user's allowance. Start at standard thinking for ordinary work and move up when the task needs it (Gemini app limits).

Microsoft Copilot Free

Appropriate for web-grounded questions, brainstorming, rewriting, summarizing pages, and general assistance. Signing in adds history and other capabilities. It does not turn general web chat into a curated business workspace. Microsoft describes the free experience as web-grounded chat at no cost, with additional features when signed in (Microsoft Copilot experiences).

Keep requests bounded. If the task depends on private organizational material, use the approved work environment and confirm what grounding and protections actually apply.

Microsoft 365 Copilot Chat

Copilot Chat changes the context problem, because account type, licensing, organizational configuration, and data access all affect what it can see. Name the source boundary you want: the web, a specific file, or work content available to you. Never assume it searched every email, site, and file. Microsoft states that Copilot Chat is included with eligible Microsoft 365 licenses and is web-grounded, and that broader work-grounded behavior depends on licensing and configuration (Microsoft Copilot experiences).

Microsoft 365 Copilot Notebooks

Use a Notebook when a task should be grounded in a curated set of references. Add only the files, pages, chats, and notes that define the assignment. Microsoft states that Copilot Notebooks uses the references added to the notebook rather than automatically reaching the user's entire OneDrive, email, Teams history, or the web (How Copilot Notebooks works).

That makes a Notebook a clean example of context engineering. The user defines the evidence boundary on purpose.

Comparative truth

These products are not the same. Plan allowances, project behavior, retrieval, reasoning controls, file limits, and data boundaries all differ. Make direct comparisons only after checking current official documentation. Durable advice belongs to context roles and acceptance criteria, not to a permanent winner.

Which of these five levers can I actually measure, and which am I just hoping about?

In one line: Five levers turn prompt craft into workload architecture.

Cache
  ↓
Reasoning Budget
  ↓
Tool Context
  ↓
Compaction and Memory
  ↓
Measurement

Cache reuses stable input instead of paying to reprocess it. Reasoning budget matches thinking effort to difficulty. Tool context exposes only the capabilities and results that help. Compaction and memory preserve state while shedding transcript weight. Measurement tells you whether any of it worked.

The first four are things you build. The fifth is the one that keeps you honest, and it is the one most often skipped. None of the four would have stopped my $200 by itself. The fifth one, actually used, would have caught it inside a week instead of after it.

What am I dividing by, and is it the number of things that actually got used?

In one line: Cost per call is easy to calculate and often answers the wrong question.

This chapter does two things. It gives you the formulas for the cost of a request and the cost of a workload, and then it walks through two worked examples that show why the second formula is the one that matters. If you skip the arithmetic, read the sentence above each block, which says what the numbers are about to prove.

General cost formula

For one request:

Request cost
= (new input tokens × new input rate)
+ (cache-write tokens × cache-write rate)
+ (cache-read tokens × cache-read rate)
+ (output tokens × output rate)
+ tool or retrieval charges
+ storage or cache-duration charges

Some vendors fold reasoning tokens into output billing. Some expose them in a separate usage field. Some tools carry per-call charges. verify current Map the provider's current invoice categories before you use this formula, because the categories are where the surprises live.

For a workload:

Workload cost
= successful request costs
+ failed request costs
+ retry costs
+ validation costs
+ fallback costs
+ human correction cost

That last line is usually the largest and is almost never in the vendor dashboard.

Worked example: architecture beats brevity

What this shows: caching a repeated prefix cut the cost of a 100-task workload by roughly two thirds, and shortening the user's question would not have come close.

These rates are hypothetical, chosen for arithmetic. They are not current vendor pricing and should not be used for budgeting.

• new input: $2 per million tokens
• cached input: $0.20 per million tokens
• output: $8 per million tokens
• each task uses 40,000 repeated input tokens, 2,000 new input tokens, and 3,000 output tokens
• 100 tasks run

Without caching:

Input = 100 × 42,000 = 4,200,000 tokens
Input cost = 4.2 × $2 = $8.40
Output = 100 × 3,000 = 300,000 tokens
Output cost = 0.3 × $8 = $2.40
Total = $10.80

With the 40,000-token prefix cached after the first request, ignoring cache-write charges for this simplified illustration:

New input = 100 × 2,000 + 40,000 = 240,000 tokens
New input cost = 0.24 × $2 = $0.48
Cache reads = 99 × 40,000 = 3,960,000 tokens
Cache-read cost = 3.96 × $0.20 = $0.792
Output cost = $2.40
Approximate total = $3.672 plus cache-write charges

The architecture created the saving. Making the user's question three words shorter would not have mattered.

Worked example: the cheaper call that costs more

What this shows: two workflows that look nearly identical on model cost alone separate completely once you count the human minutes spent on failures.

Workflow A costs $0.04 per attempt and succeeds 55 percent of the time.

Expected model cost per success = $0.04 / 0.55 = $0.073

Workflow B costs $0.07 per attempt and succeeds 92 percent of the time.

Expected model cost per success = $0.07 / 0.92 = $0.076

On model cost alone they are nearly identical. Now add a person. If each failed attempt needs five minutes of review at $60 an hour, Workflow A adds $5 in labor per hour of running, and Workflow B adds well under half that. Per-call pricing would have hidden the entire decision.

My $200 was cost-per-call thinking at the scale of a month. I knew what each reload cost. I never once asked what each accepted task cost, and so I had no idea that most of the reloads were buying retries.

Is my stable material actually first, and have I checked the usage field to see whether the cache hit?

In one line: Put the stable material first, and then verify that the cache is actually being hit.

The stable-prefix pattern

STABLE PREFIX
System instructions
Policies
Schemas
Tool definitions
Large repeated references
Canonical examples

DYNAMIC SUFFIX
Current record
Current question
Time-sensitive data
Requested output

Caching reduces the price or the latency of reprocessing eligible repeated input. It does not remove that input from the model's context window. A cached 40,000-token prefix still occupies 40,000 tokens of attention budget, with everything Chapter 3 says about that.

That distinction is worth repeating because it is the most common misunderstanding in this whole chapter. Caching is a billing optimization. It is not a context optimization.

The vendors all have it, and they are all different

OpenAI, Anthropic, and Google each document prompt or context caching, but the controls, eligibility thresholds, retention, usage fields, and prices differ (OpenAI prompt caching, Claude prompt caching, Gemini context caching).

OpenAI's documentation describes caching as reusing work when requests share a prompt prefix, with reads billed at a reduced cached-input rate, and notes that cache reuse requires the entire rendered prefix to match (OpenAI prompt caching). verify current Discount levels, minimum sizes, and retention windows are exactly the parameters most likely to have changed since this was checked.

Google documents implicit caching for current Gemini families and recommends structuring repeated requests with a similar prefix, keeping large common content at the front (Gemini context caching). The design advice converges across all three vendors even where the billing does not: common content first, variable content last.

Cache design rules

1. Put the most stable content first.
2. Put timestamps, user records, and current questions last.
3. Avoid tiny changes to the prefix. A single edited word at the top can invalidate everything after it.
4. Measure actual cache reads. Do not assume a hit.
5. Compare cache-write and storage costs against expected reuse before committing.
6. Do not cache sensitive material without reviewing retention and data controls.
7. Remember that a cached bad instruction is still a bad instruction, delivered faster and cheaper.

Cache break-even

Let:

• W = one-time cache-write cost
• R = cost of one cache read
• U = cost of sending the same tokens uncached
• N = number of later reuses

Caching is economically favorable when:

W + (N × R) is less than N × U

Therefore:

N is greater than W / (U - R)

Use current vendor rates and retention rules. Do not copy a break-even count from a different provider, or from this manual.

Part of what my $200 bought was the same project background, re-sent in full at the start of every conversation. On a subscription that is invisible. On credits it is a line item.

Of the tools I have switched on, how many could the model confidently choose between?

In one line: Every capability you expose asks the model to make another choice.

Retrieval economics

Retrieval replaces a large repeated corpus with a small set of relevant passages. Its cost includes indexing, storage, search, the retrieved tokens themselves, and the risk of missing evidence.

Measure it on two dimensions:

• precision: how much of what was retrieved was actually relevant
• recall: how much of the necessary evidence was retrieved

High precision with poor recall gives focused, incomplete answers. High recall with poor precision recreates the original pile, now with additional machinery.

Just-in-time versus pre-loaded

The field has shifted. Many applications still do embedding-based retrieval before inference. Increasingly, systems augment that with just-in-time strategies, where the agent holds lightweight references and pulls data at runtime through tools. Anthropic gives the example of Claude Code writing targeted queries and using commands like head and tail to work across large data without ever loading the full objects into context (Effective context engineering for AI agents).

The tradeoff is real. Runtime exploration is slower than retrieving something precomputed, and it takes deliberate engineering to give the model the right tools and heuristics for navigating. Without that, an agent burns context chasing dead ends. Anthropic notes that the most effective systems are often hybrids: retrieve some things up front for speed, allow autonomous exploration for the rest, and that a hybrid suits less dynamic domains such as legal or financial work (Effective context engineering for AI agents).

Tool context

Every tool definition consumes space. Expose only tools the task might need. Write descriptions that are clear about purpose, inputs, side effects, and error conditions. Anthropic's guidance is that tools should be self-contained, robust to error, and unambiguous about intended use, and that bloated, overlapping tool sets are among the most common failure modes (Effective context engineering for AI agents).

This applies to consumer work too. If you have fourteen connectors switched on and you are asking about a spreadsheet, thirteen of them are overhead and at least one is a distraction. Connecting tools was the second thing I did on my way to $200, right after exploring settings. I turned things on because turning them on was easy, not because the task in front of me needed them.

Limit large tool outputs before they reach the model. Filter fields, narrow date ranges, cap rows.

Structured outputs

Use a schema when downstream software needs predictable fields. Structured output reduces parsing failures and retries. An oversized schema also consumes context and can make simple work harder. Define only required fields, name them clearly, and validate programmatically rather than by reading.

Batch and asynchronous work

When immediate interaction is not required, batch or lower-priority processing may cost less. The tradeoff is price against latency, operational complexity, and retry handling. verify current Verify provider terms before designing around it.

Did the last escalation actually improve the result, and did anyone write that down?

In one line: More thinking is a cost, not a quality setting.

Reasoning is a budget

Use low or standard reasoning for extraction, classification, formatting, and routine summaries. Increase it for ambiguous decisions, multi-source reconciliation, difficult mathematics, architecture, and high-risk review.

Reaching for maximum capability before deciding what the work required was one of the specific habits that ran up my $200. It is easy to mistake for diligence. It is actually the opposite: a decision skipped, not a decision made carefully.

OpenAI's guidance says higher reasoning effort is not automatically better and recommends raising it only when evaluation shows a measurable quality gain (OpenAI model guidance). Google similarly warns that more advanced models and higher thinking levels consume more usage (Gemini app limits). verify current Providers expose reasoning controls differently, and billing treatment changes by model and product.

Route by task, not prestige

Choose the least expensive and least intensive route
that reliably passes the task's quality threshold.

Build a small evaluation set of representative tasks. Test candidates on accuracy, completeness, latency, token use, retry rate, and human correction time. Route by task class, not by preference. Twenty labeled examples will settle arguments that have been running for months.

Escalate on evidence

1. Start with the proven default for that task class.
2. Validate the output.
3. Escalate only on a defined failure.
4. Pass the failure evidence and the relevant state, not the entire history.
5. Record whether escalation actually improved the result.

Step five is the one everyone skips. It is the only step that tells you whether the escalation rule is worth keeping, and it is also the step most likely to be replaced by a confident opinion in a meeting.

When this conversation is summarized, what is the one thing that must survive, and where is it written down?

In one line: When the work outlasts the window, you need a strategy for what survives.

This is the longest chapter in the manual, and it covers four techniques in order: compaction, context editing, structured note-taking, and sub-agents. It then gives you a table for choosing among them, translates each one for people on a subscription, and ends where this manual stops. If you only read one section, read the subscription version, because the discipline is the same and the mechanisms are the only thing that differs.

Long-horizon tasks are the ones where total token count exceeds the context window: a large migration, a multi-week research project, anything measured in hours of continuous work. Waiting for bigger windows does not solve it, because windows of every size remain subject to the attention problem in Chapter 3. Anthropic describes three techniques that address this directly: compaction, structured note-taking, and sub-agent architectures (Effective context engineering for AI agents).

I forgot about the usage-credits setting and went back to work. Claude did not forget. That is the whole problem this chapter is about: deciding what has to survive on its own, in writing, because attention will not carry it.

Compaction

Full history
   ↓
Current working state
   ↓
Milestone summary or provider compaction
   ↓
Fresh working context
   ↓
Continue

Compaction takes a conversation approaching the window limit, summarizes it, and restarts with the summary. In Claude Code, Anthropic implements this by passing the message history back to the model to compress, preserving architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs, then continuing with that compressed context plus the most recently accessed files (Effective context engineering for AI agents).

The design question is not whether to summarize. It is what the summary must preserve. Over-aggressive compaction loses the subtle detail whose importance only becomes obvious later. Anthropic's tuning advice is to maximize recall first, making sure the compaction prompt captures everything relevant, then improve precision by cutting the superfluous (Effective context engineering for AI agents).

Both major platforms now expose this. OpenAI documents server-side compaction on the Responses API, enabled by setting a compaction threshold, plus a standalone compaction endpoint (OpenAI compaction). Anthropic documents server-side compaction that summarizes the conversation as it approaches the window limit (Claude compaction).

Two design points are worth knowing before you build on either. First, how you carry conversation forward changes what you are allowed to prune: OpenAI's guidance is that with response-identifier chaining you should not prune manually, while with stateless input chaining you can drop items that came before the most recent compaction item (OpenAI conversation state, OpenAI context management). Second, compaction is not the same as truncation. Truncation drops the oldest material. Compaction tries to carry its meaning forward. Only one of those preserves the decision you made in turn three.

Context editing, or clearing what is already spent

The lightest-touch form of compaction is clearing old tool results. Once a tool has been called deep in the history, the raw result is rarely needed again. Anthropic's context editing feature clears specific tool results on the client, distinct from compaction, which summarizes the whole conversation server-side (Context editing).

Anthropic reports measured results on an internal agentic search evaluation: the memory tool combined with context editing improved performance 39 percent over baseline, and context editing alone improved it 29 percent. In a 100-turn web search evaluation, context editing let agents finish workflows that would otherwise have failed from context exhaustion, while reducing token consumption 84 percent (Managing context on the Claude Developer Platform).

Structured note-taking

The agent writes notes to persistent storage outside the context window and reads them back later. Anthropic describes this as agentic memory and gives the plain version: a to-do list, or a NOTES.md file that the agent maintains as it goes. The pattern lets an agent track progress across a long task and keep dependencies that would otherwise vanish across dozens of tool calls (Effective context engineering for AI agents).

Anthropic's memory tool implements this as a file directory the model can create, read, update, and delete, persisting between sessions. Their documentation recommends pairing memory with compaction for long-running agents: compaction keeps the active context small, and memory preserves the information that has to survive the summary. They also recommend setting up memory files deliberately for projects that span sessions, rather than writing them ad hoc as work proceeds (Memory tool).

That last recommendation is the one that transfers to everyone, including people who will never touch an API. A file you designed on purpose survives. A file the assistant improvised at turn 60 deserves inspection.

Sub-agents

Rather than one agent holding state for a whole project, specialized sub-agents handle focused tasks with clean context windows. The coordinating agent holds the plan. Each sub-agent may explore extensively, using tens of thousands of tokens, and returns only a condensed summary, often 1,000 to 2,000 tokens (Effective context engineering for AI agents).

The detailed search context stays isolated in the sub-agent. The lead agent gets the conclusion. Send someone to read the whole report and come back with a page, not the box it arrived in.

Choosing among the three

Anthropic's own guidance on when to use which:

Source: Effective context engineering for AI agents.

The subscription version of all of this

You do not need an API to use these ideas.

• Compaction is the continuation brief in Chapter 12.
• Structured note-taking is a decision log in a document you own.
• Sub-agents are a separate chat given only the deliverable and the acceptance criteria, asked to find defects.

The mechanisms differ. The discipline is identical.

Where this manual stops

Once AI begins operating tools, repositories, terminals, applications, and other agents, context optimization becomes part of agent architecture. Permissions, tool selection, state recovery, verification, and rollback deserve their own treatment. That is a different manual, and this one stops at the boundary.

If someone asked me what improved and by how much, could I answer with a number I did not make up?

In one line: If you cannot say what improved, you did not optimize. You rearranged.

The core metrics

Tokens per successful task
= total workload tokens / accepted tasks

Cost per successful task
= total workload cost / accepted tasks

First-pass success rate
= tasks accepted without retry / total tasks

Retry rate
= repeated attempts caused by defects / total attempts

Cache effectiveness

Cache hit rate
= requests with an eligible cache hit / eligible requests

Cached-token share
= cache-read tokens / total eligible input tokens

Both come from provider usage fields, not from your assumptions.

Context relevance ratio

This one is a sampled audit, not a billing field:

Context relevance ratio
= input tokens judged necessary or materially useful
  / total reviewed input tokens

Do not pretend this number is objective. Sample both failed and successful tasks. Label each piece of context as required, helpful, neutral, distracting, or conflicting. The labels are more useful than the ratio. A well-running workflow tolerates neutral context right up until growth turns neutral into distracting.

Quality-adjusted efficiency

Quality-adjusted efficiency
= accepted quality score × successful tasks
  / total cost

Define the quality rubric before you compare systems. Otherwise the metric improves every time someone lowers the bar, which is a thing that happens.

Minimum measurement set

Track, per task:

• task type
• model or mode
• input, cached input, reasoning, and output usage where available
• total cost
• latency
• accepted or rejected
• retry reason
• human correction minutes
• source or retrieval failure

Nine fields. A spreadsheet handles it. The reason to write them down is that memory is generous about which experiments worked.

I had none of these for my $200. I had a balance that went down and a vague sense that the work was getting done. When I finally looked, I could not have told you whether any given reload had bought a finished deliverable or a fourth attempt at the same one. That is the difference between a limit and a measurement.

What Is Durable and What Is Not

Three things in this manual should outlive the products described in it.

Context is a finite resource with diminishing returns. That follows from how the models are built, not from any vendor's roadmap.

Cost per accepted task is the only denominator that reflects the actual work. Everything else is a proxy.

The failure is usually in the framing, the grounding, or the verification, and only rarely in the model.

Everything else here, meaning the plan names, the caching parameters, the feature names, and the numbers, is a snapshot. It was accurate on September 4, 2026. Check the links before you spend anything on the strength of it.

AI models will keep improving. Good operating discipline does not become obsolete because the machinery gets stronger. More capable systems raise both the value of clear direction and the cost of vague authority.