Context compression does not prevent context overflow errors

[dobesv]

When you have context compression enabled with a token limit lower than your model's limit, you can still get "stuck" with an error like:

litellm.ContextWindowExceededError: litellm.BadRequestError: AnthropicError - b'{"type":"error","error":{"type":"invalid_request_error","message":"prompt is too long: 217388 tokens > 200000 maximum"},"request_id":"req_011CYGQ9pNMT8pdQbGzZcPUs"}'

This is because the context compression logic runs after the event is sent to the LLM. The context compression should either happen before running the LLM or detect an error like the above and compress and retry after.

See also

• More Context Compaction Algorithms #4146
• a88e864
• Smart Context Pruning with Token-Aware Filtering #3829

[dobesv]

Working on a little workaround over here - kagent-dev/kagent@3d59cf2 . It would be nice to have some version of this logic built into the ADK.