#4269: fix: support tool result image format in Control UI
app: web-ui
Cluster:
Web UI Enhancements and Fixes
Add support for the backend tool result image format { type: "image", data: "base64...", mimeType: "image/png" } in the extractImages function.
Previously, Control UI only recognized the Anthropic standard format with a nested source object. This fix allows browser screenshots and other tool-generated images to display correctly in the chat interface.
<!-- greptile_comment -->
<h2>Greptile Overview</h2>
<h3>Greptile Summary</h3>
Updates `extractImages` in `ui/src/ui/chat/grouped-render.ts` to recognize an additional backend tool-result image block shape (`{ type: "image", data, mimeType }`) alongside the existing Anthropic `source: { type: "base64", data, media_type }` and URL-based formats. This ensures tool-generated screenshots and similar image outputs are surfaced in the Control UI chat rendering pipeline.
<h3>Confidence Score: 4/5</h3>
- This PR is largely safe to merge; it adds a small, localized format handler with minimal behavioral change.
- Change is confined to `extractImages` and follows the existing pattern for base64-to-data-URL conversion. Main remaining concern is unvalidated `mimeType` being embedded into `data:` URLs, which could lead to malformed or non-image data URLs if upstream data is unexpected.
- ui/src/ui/chat/grouped-render.ts
<!-- greptile_other_comments_section -->
<sub>(4/5) You can add custom instructions or style guidelines for the agent [here](https://app.greptile.com/review/github)!</sub>
<!-- /greptile_comment -->
Most Similar PRs
#8284: Fix: Webchat images now persist after sending
by vishaltandale00 · 2026-02-03
83.6%
#21042: fix(ui): render images in tool result messages
by Mellowambience · 2026-02-19
83.0%
#6819: fix(tui): handle unstructured tool results and errors in tool execu...
by TreyDong · 2026-02-02
81.3%
#2958: fix(media): wire tools.media.image.maxBytes config to image processin…
by shamsulalam1114 · 2026-01-27
76.4%
#8880: feat(ui): make URLs clickable in tool output
by jnvw · 2026-02-04
76.3%
#17448: ui: make tool cards collapsible with inline expansion
by karimStekelenburg · 2026-02-15
75.6%
#9248: Fix: Webchat UI goes grey/unresponsive after Slack message tool calls
by vishaltandale00 · 2026-02-05
75.1%
#12974: fix: intermittent (no output) reported by users
by vincentkoc · 2026-02-10
74.6%
#22333: Fix webchat inline image rendering and size handling
by AIflow-Labs · 2026-02-21
74.4%
#16733: fix(ui): avoid injected newlines when tool output is hidden
by jp117 · 2026-02-15
74.2%