← Back to PRs

#4269: fix: support tool result image format in Control UI

by bobcyw open 2026-01-30 00:27 View on GitHub →
app: web-ui
Add support for the backend tool result image format { type: "image", data: "base64...", mimeType: "image/png" } in the extractImages function. Previously, Control UI only recognized the Anthropic standard format with a nested source object. This fix allows browser screenshots and other tool-generated images to display correctly in the chat interface. <!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> Updates `extractImages` in `ui/src/ui/chat/grouped-render.ts` to recognize an additional backend tool-result image block shape (`{ type: "image", data, mimeType }`) alongside the existing Anthropic `source: { type: "base64", data, media_type }` and URL-based formats. This ensures tool-generated screenshots and similar image outputs are surfaced in the Control UI chat rendering pipeline. <h3>Confidence Score: 4/5</h3> - This PR is largely safe to merge; it adds a small, localized format handler with minimal behavioral change. - Change is confined to `extractImages` and follows the existing pattern for base64-to-data-URL conversion. Main remaining concern is unvalidated `mimeType` being embedded into `data:` URLs, which could lead to malformed or non-image data URLs if upstream data is unexpected. - ui/src/ui/chat/grouped-render.ts <!-- greptile_other_comments_section --> <sub>(4/5) You can add custom instructions or style guidelines for the agent [here](https://app.greptile.com/review/github)!</sub> <!-- /greptile_comment -->

Most Similar PRs