#23639: fix(agents): stop re-resizing session history images on every turn (#23590)
agents
size: S
trusted-contributor
## Summary
- Problem: every image in session history gets re-decoded and re-resized on every turn, even if it was already processed. A session with 10 screenshots and 50 messages triggers 500+ redundant resize operations.
- Why it matters: adds latency per turn that scales linearly with image count × message count, and floods logs with identical resize entries.
- What changed: added an in-memory LRU cache (max 64 entries) in `resizeImageBase64IfNeeded` keyed by a SHA-256 hash of the image content + resize limits. Second pass for the same image returns the cached result instantly.
- What did NOT change: no changes to session persistence, transcript format, or the resize algorithm itself. The cache is purely additive — on miss, the existing path runs as before.
lobster-biscuit
## Change Type (select all)
- [x] Bug fix
## Scope (select all touched areas)
- [x] Skills / tool execution
## Linked Issue/PR
- Closes #23590
## Root Cause
`sanitizeSessionHistory` runs on every turn and calls `sanitizeSessionMessagesImages` → `sanitizeContentBlocksImages` → `resizeImageBase64IfNeeded` for every image block in the full message history. The resized output is only used in-memory for the API call via `agent.replaceMessages()` — it's never persisted back to the session transcript. So the original oversized images get loaded and re-processed from scratch on every subsequent turn.
## User-visible / Behavior Changes
Repeated "Image resized to fit limits" log entries for the same images across turns are gone. Turns in image-heavy sessions should be noticeably faster.
## Security Impact (required)
- New permissions/capabilities? No
- Secrets/tokens handling changed? No
- New/changed network calls? No
- Command/tool execution surface changed? No
- Data access scope changed? No
## Evidence
- [x] Failing test/log before + passing after
- `tool-images.test.ts` — new test "returns cached result when same oversized image is processed twice" calls `sanitizeContentBlocksImages` twice with the same oversized PNG and asserts the second call returns identical output (cache hit)
- All 5 tests in `tool-images.test.ts` pass, plus 29 tests in related media test files
## Human Verification (required)
- Verified: `pnpm build && pnpm check` pass, all existing + new tests pass
- Edge cases: cache eviction at 64 entries, cache key uses content prefix + length + limits so different images or different limits produce different keys
- Not verified: live gateway session with real Telegram images (local-only fix, deterministic code path)
## Compatibility / Migration
- Backward compatible? Yes
- Config/env changes? No
- Migration needed? No
## Failure Recovery (if this breaks)
- Revert the single commit. The cache is additive — removing it just goes back to re-processing every turn.
- No config or data changes to undo.
## Risks and Mitigations
- Risk: cache holds base64 strings in memory, could use significant RAM with many large images
- Mitigation: capped at 64 entries with LRU eviction. Worst case ~320MB (64 × 5MB max), but typical images are much smaller. The gateway process already holds these images in the session message array anyway.
<!-- greptile_comment -->
<h3>Greptile Summary</h3>
Adds an in-memory cache (max 64 entries, FIFO eviction) to `resizeImageBase64IfNeeded` so that session-history images are not re-decoded and re-resized on every agent turn. The cache key is a SHA-256 hash of the full base64 content plus the resize limits, ensuring correctness across different images and limit configurations. The change is additive — on cache miss, the existing resize path runs unmodified.
- `src/agents/tool-images.ts`: Added module-level `Map` cache with `resizeCacheKey()`, `evictResizeCacheIfNeeded()`, and exported `clearImageResizeCache()`. Cache lookups are inserted at the top of `resizeImageBase64IfNeeded`; cache stores are added on the within-limits and successful-resize code paths.
- `src/agents/tool-images.test.ts`: Added 5 new tests covering cache hit, within-limits caching, different-limits separation, different-image separation, and post-clear re-processing. `clearImageResizeCache()` is called in `beforeEach` to isolate tests.
- `src/agents/tool-images.log.test.ts`: Added `clearImageResizeCache()` import and call in `beforeEach` so that log-assertion tests aren't affected by cross-test cache hits.
<h3>Confidence Score: 5/5</h3>
- This PR is safe to merge — it adds a purely additive, well-tested cache with no changes to existing resize logic or session persistence.
- The change is narrowly scoped to a single module with no external API changes. The cache is additive (on miss, the existing path runs unchanged), the key uses full-content SHA-256 hashing eliminating collision risk, and the test suite thoroughly covers cache behavior including isolation between tests. Prior review feedback (prefix-only hashing, off-by-one eviction, cross-test cache interference) has been addressed. The only minor inaccuracy is that the comment says "LRU" but the Map-based implementation is FIFO, which has no practical impact given the use case.
- No files require special attention.
<sub>Last reviewed commit: 744ac2a</sub>
<!-- greptile_other_comments_section -->
<!-- /greptile_comment -->
Most Similar PRs
#23706: perf: cache image resize results to avoid redundant processing (#23...
by echoVic · 2026-02-22
91.3%
#23662: fix: cache sanitized images to avoid redundant re-processing per turn
by davidemanuelDEV · 2026-02-22
90.7%
#9598: fix(agents): check base64 string length against 5MB API limit
by BlockBB · 2026-02-05
81.9%
#19135: fix(agents): prune excess images from conversation history (#19099)
by pierreeurope · 2026-02-17
79.7%
#8172: fix(sessions_list): strip base64 image data to prevent context over...
by Flamrru · 2026-02-03
78.1%
#2958: fix(media): wire tools.media.image.maxBytes config to image processin…
by shamsulalam1114 · 2026-01-27
78.1%
#22220: feat(bootstrap): cache session's bootstrap files so we don't invali...
by anisoptera · 2026-02-20
77.8%
#18219: fix: validate base64 image data before sending to LLM APIs
by Grynn · 2026-02-16
75.8%
#5817: fix: strip old images during compaction to prevent 413 session bloat
by jduartedj · 2026-02-01
75.6%
#12358: fix: Sessions format timestamps in sessions_history using userTimezone
by xialonglee · 2026-02-09
75.4%