Before the cache expires

Bonsai’s chat has a new line of small text, right next to the context-usage ring: Cache: 42 min left.

It only shows up when the agent has stopped to wait for you, either because it finished a turn or because it asked you something. With ten minutes or less to go it turns red, and when time runs out it says Cache expired.

This post explains how that number is worked out, and why we didn’t ask the harness for a “real” expiry time.

Why the cache matters

Every time a harness like Claude Code or Codex calls the model, it usually sends the whole conversation again: the system prompt, the tool definitions, every message so far and every tool result. A conversation that has been working all afternoon can be hundreds of thousands of tokens per call.

The prompt cache saves the model provider from processing all of that from scratch every time. If the prefix matches the last call, it is read from the cache. Cache reads cost far less than normal input (about a tenth of the price at Anthropic) and they are much faster.

But the cache expires. If nobody uses it, it is dropped. The next message then has to process the whole context again, which is slower and costs more.

So whenever an agent stops to wait for you, an invisible countdown is running. Reply before it ends and the next turn keeps using the cache. Reply after, and the next turn pays full price to read the whole conversation again.

How Bonsai counts

The rule is short:

1. Start from the time of the last event in the session. Events come from the runtime: a piece of a reply, a tool call, the end of a turn, a request. Each one carries a millisecond timestamp from the runtime’s own clock.
2. Add one hour.
3. Subtract the current time and round up to the minute.

Why the last event? The cache’s lifetime restarts each time it is used, and the model’s last call happens just as the agent emits its last few events. The gap is usually a few seconds, which doesn’t matter for a reminder shown in minutes.

Why one hour? Anthropic’s API offers two cache durations: five minutes by default, and one hour at a higher write price. We count with the longer one. If your harness or provider uses a shorter window, the number will be optimistic. We would rather give one consistent rule that is easy to remember: reply within an hour.

The line is hidden when:

• The agent is running. The cache is in active use, so a countdown means nothing.
• No turn has finished yet and nothing is waiting for your answer.
• The conversation has ended and the cache has already expired. A permanent “expired” would just be noise when you reread a conversation from three days ago.

A conversation that has ended but can still be continued keeps counting down, because continuing it reuses the same context, so the cache still applies.

We don’t read the harness’s cache

You might ask why we don’t simply ask the harness when its cache expires.

Mostly, they can’t tell you. A harness knows which cache settings it put on each request, but when the provider actually evicts the cache is usually not in any response. The rules differ between harnesses, between providers, and between models from the same provider.

Bonsai has always treated harnesses the same way: it doesn’t know about any specific setting. Models, effort and permission modes are declared by the runtime when it connects, and Bonsai just draws what it was told. The cache is no different. A cache rule per harness would mean following every one of their updates, and one of them would always be out of date.

So we did the simpler and more honest thing. Bonsai looks only at what it actually knows, which is the time of the last event. The hint next to the number says so too: this is a reminder, not a bill.

What’s next

This is the first piece of Bonsai’s work on “your turn”. When an agent is stuck, whether the workflow planned that stop or it ran into something nobody foresaw, it’s the same thing for you: something is waiting for you, and it’s best to answer within the hour.

The countdown in the chat only helps once you open the conversation. We are also building Bonsai’s own notification system, so you can find out without opening it.