I see two things coming up regularly in our internal IBM Bob Slack channel. People want a larger context window, and people are surprised how expensive their conversations become. Well, both make sense when you look at what the agent sends to the model with each request.
In this episode, I explain what a context window contains and how IBM Bob, Codex, Claude Code, and Gemini CLI decide what goes into it. We follow twenty model requests, starting at 10,000 input tokens and adding another 10,000 each time. The last request contains 200,000 tokens, but the total across those requests is 2.1 million. At the standard GPT-6.1 Sol rate used in the episode, that is $4.20 for input alone, with caching excluded and output or tool charges added separately. Then we look at how caching and compaction change the calculation.
I also share guidance from the Book of Bob, my collection of agentic development best practices on an internal IBM website. Unfortunately, I haven’t been able to open source it so far. We go through standing context, tool descriptions, focused task briefs, and short handover notes, using a Java email-validation example to make the advice concrete.
And, of course, there is the business model. The labs sell tokens, but serving those tokens costs them money too. I consider those incentives alongside the cost of reaching a result I can accept, including retries and review time. A larger window can help with a change across several modules; the capacity number alone doesn’t tell us what the task will cost or how reliably the model will connect the details.
Model limits and API prices are a snapshot from October 14, 2026. The Codex defaults mentioned came from my local installation and can differ across accounts and sessions.
The episode is narrated locally with Fish S2 Pro and my trained LoRA voice adapter.
Chapters
00:00 — Intro
00:12 — Wanting more context and wondering about the bill
01:52 — What goes into a context window
04:25 — Model limits and the applications around them
06:57 — How 200,000 tokens become 2.1 million across a conversation
09:45 — Caching, pricing tiers, and response time
13:01 — Long-context reasoning and what compaction can lose
15:11 — Context practices from the Book of Bob
20:24 — Selling tokens and the cost of an accepted result
22:52 — What to measure in your own coding sessions
24:45 — Outro
Links and references
Context windows and coding tools
IBM Bob: Context window management — Public documentation for the context breakdown, task limit, and compaction threshold.
OpenAI: GPT-6 Astra and Codex configuration reference — Model limits and application settings. The configuration reference is separate from the local catalog observation discussed in the episode.
Claude model overview and Claude Code extended context — Model capacities and the conditions that apply inside Claude Code.
Gemini 3.1 Pro, Gemini 4 Argon, and Gemini CLI model selection — Check the selected model and whether its limit describes input or a shared input/output window.
Conversation costs and caching
OpenAI API pricing — Source for the GPT-6.1 Sol example: $2 per million uncached input tokens at the standard short-context rate on the research date.
OpenAI conversation state — Continuing a conversation with a response ID, including how earlier input is billed.
OpenAI prompt caching — Prefix reuse and the conditions for cache hits.
Claude API pricing — Cache rates and model-specific long-context pricing.
Context practices and research
Anthropic: Effective context engineering for AI agents — Retrieving information when needed, compaction, notes, and separate agent contexts.
Lost in the Middle: How Language Models Use Long Contexts — Research on how information placement affected the models tested. These results describe earlier models.
RULER: What’s the Real Context Size of Your Long-Context Language Models? — Evaluating long-context behavior beyond finding a single fact.
