← Back to PRs

#4300: Gateway: prevent OpenAI-compatible client crash on SSE termination

by perryraskin open 2026-01-30 01:32 View on GitHub →
gateway
### What Some OpenAI-compatible clients crash or throw when the SSE stream ends without a terminal `chat.completion.chunk` containing `finish_reason`. ### Fix - Emit a final `chat.completion.chunk` with `finish_reason: "stop"` (and empty `delta`) before sending `data: [DONE]`. - Keep content-part parsing compatible with both `{ type: "input_text", input_text: "..." }` and older `{ type: "input_text", text: "..." }` shapes. ### Tests - `pnpm vitest run --config vitest.e2e.config.ts src/gateway/openai-http.e2e.test.ts` <!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> This PR adjusts the OpenAI-compatible `/v1/chat/completions` SSE streaming implementation to avoid client crashes on stream termination. - Adds a helper to emit a terminal `chat.completion.chunk` (with `finish_reason`) immediately before `data: [DONE]`. - Refactors stream shutdown into a shared `endStream()` path used for lifecycle end/error and the `finally` cleanup. - Updates content-part parsing to accept both `{type: "input_text", input_text: "..."}` and the older `{type: "input_text", text: "..."}` shapes. Overall this fits the existing gateway approach: it continues to proxy agent events into OpenAI-style chunks while keeping request parsing permissive for multiple client payload formats. <h3>Confidence Score: 4/5</h3> - This PR is generally safe to merge and addresses a real interoperability issue, with a couple of small cleanup/semantics nits. - The change is localized to SSE termination behavior and request content parsing, with minimal surface area. The new `endStream()` reduces duplicated shutdown logic and should prevent premature `[DONE]` without a terminal chunk. Remaining concerns are: listener cleanup (`req.on('close')`) and distinguishing normal vs error termination in the terminal chunk semantics. - src/gateway/openai-http.ts <!-- greptile_other_comments_section --> <sub>(2/5) Greptile learns from your feedback when you react with thumbs up/down!</sub> <!-- /greptile_comment -->

Most Similar PRs