← Back to PRs

#23639: fix(agents): stop re-resizing session history images on every turn (#23590)

by yinghaosang open 2026-02-22 14:50 View on GitHub →
agents size: S trusted-contributor
## Summary - Problem: every image in session history gets re-decoded and re-resized on every turn, even if it was already processed. A session with 10 screenshots and 50 messages triggers 500+ redundant resize operations. - Why it matters: adds latency per turn that scales linearly with image count × message count, and floods logs with identical resize entries. - What changed: added an in-memory LRU cache (max 64 entries) in `resizeImageBase64IfNeeded` keyed by a SHA-256 hash of the image content + resize limits. Second pass for the same image returns the cached result instantly. - What did NOT change: no changes to session persistence, transcript format, or the resize algorithm itself. The cache is purely additive — on miss, the existing path runs as before. lobster-biscuit ## Change Type (select all) - [x] Bug fix ## Scope (select all touched areas) - [x] Skills / tool execution ## Linked Issue/PR - Closes #23590 ## Root Cause `sanitizeSessionHistory` runs on every turn and calls `sanitizeSessionMessagesImages` → `sanitizeContentBlocksImages` → `resizeImageBase64IfNeeded` for every image block in the full message history. The resized output is only used in-memory for the API call via `agent.replaceMessages()` — it's never persisted back to the session transcript. So the original oversized images get loaded and re-processed from scratch on every subsequent turn. ## User-visible / Behavior Changes Repeated "Image resized to fit limits" log entries for the same images across turns are gone. Turns in image-heavy sessions should be noticeably faster. ## Security Impact (required) - New permissions/capabilities? No - Secrets/tokens handling changed? No - New/changed network calls? No - Command/tool execution surface changed? No - Data access scope changed? No ## Evidence - [x] Failing test/log before + passing after - `tool-images.test.ts` — new test "returns cached result when same oversized image is processed twice" calls `sanitizeContentBlocksImages` twice with the same oversized PNG and asserts the second call returns identical output (cache hit) - All 5 tests in `tool-images.test.ts` pass, plus 29 tests in related media test files ## Human Verification (required) - Verified: `pnpm build && pnpm check` pass, all existing + new tests pass - Edge cases: cache eviction at 64 entries, cache key uses content prefix + length + limits so different images or different limits produce different keys - Not verified: live gateway session with real Telegram images (local-only fix, deterministic code path) ## Compatibility / Migration - Backward compatible? Yes - Config/env changes? No - Migration needed? No ## Failure Recovery (if this breaks) - Revert the single commit. The cache is additive — removing it just goes back to re-processing every turn. - No config or data changes to undo. ## Risks and Mitigations - Risk: cache holds base64 strings in memory, could use significant RAM with many large images - Mitigation: capped at 64 entries with LRU eviction. Worst case ~320MB (64 × 5MB max), but typical images are much smaller. The gateway process already holds these images in the session message array anyway. <!-- greptile_comment --> <h3>Greptile Summary</h3> Adds an in-memory cache (max 64 entries, FIFO eviction) to `resizeImageBase64IfNeeded` so that session-history images are not re-decoded and re-resized on every agent turn. The cache key is a SHA-256 hash of the full base64 content plus the resize limits, ensuring correctness across different images and limit configurations. The change is additive — on cache miss, the existing resize path runs unmodified. - `src/agents/tool-images.ts`: Added module-level `Map` cache with `resizeCacheKey()`, `evictResizeCacheIfNeeded()`, and exported `clearImageResizeCache()`. Cache lookups are inserted at the top of `resizeImageBase64IfNeeded`; cache stores are added on the within-limits and successful-resize code paths. - `src/agents/tool-images.test.ts`: Added 5 new tests covering cache hit, within-limits caching, different-limits separation, different-image separation, and post-clear re-processing. `clearImageResizeCache()` is called in `beforeEach` to isolate tests. - `src/agents/tool-images.log.test.ts`: Added `clearImageResizeCache()` import and call in `beforeEach` so that log-assertion tests aren't affected by cross-test cache hits. <h3>Confidence Score: 5/5</h3> - This PR is safe to merge — it adds a purely additive, well-tested cache with no changes to existing resize logic or session persistence. - The change is narrowly scoped to a single module with no external API changes. The cache is additive (on miss, the existing path runs unchanged), the key uses full-content SHA-256 hashing eliminating collision risk, and the test suite thoroughly covers cache behavior including isolation between tests. Prior review feedback (prefix-only hashing, off-by-one eviction, cross-test cache interference) has been addressed. The only minor inaccuracy is that the comment says "LRU" but the Map-based implementation is FIFO, which has no practical impact given the use case. - No files require special attention. <sub>Last reviewed commit: 744ac2a</sub> <!-- greptile_other_comments_section --> <!-- /greptile_comment -->

Most Similar PRs