← Back to PRs

#11124: feat(plugins): add before_llm_request hook for custom LLM headers

by johnlanni open 2026-02-07 12:36 View on GitHub →
agents stale
### Motivation fixes: #11137 Currently there is no way for plugins to inject custom HTTP headers into LLM API requests. This makes it impossible to forward sender identity information (e.g. `senderId`) to upstream LLM providers -- a requirement for scenarios like per-user audit logging, rate limiting, or routing at the provider gateway level. The infrastructure is almost entirely in place: - `senderId` already flows through the entire agent runner pipeline - The underlying `@mariozechner/pi-ai` library supports `headers?: Record<string, string>` on every stream call - A `streamFn` wrapper pattern for header injection already exists (used by OpenRouter attribution headers) The only missing piece is a plugin-accessible hook at the point where LLM request headers are assembled. ### Design A new **`before_llm_request`** plugin hook is added. It follows the established modifying-hook pattern (sequential execution, result merging) used by `before_agent_start` and `before_tool_call`. ``` Channel message received → senderId extracted → agent runner pipeline → applyExtraParamsToAgent() // existing: temperature, cache, OpenRouter headers → before_llm_request hook (NEW) // plugins inject custom headers here → cacheTrace / payload logger wrappers → streamFn → LLM API (with merged headers) ``` Key design decisions: - **Called once per agent turn**, not per HTTP request. The hook runs in the async context of `attempt.ts` right after `applyExtraParamsToAgent`. The resulting headers are captured in a `streamFn` closure, so all subsequent LLM calls within that turn carry them. This avoids the async-in-sync problem that would arise from calling hooks inside `streamFn` directly. - **Separate function** (`applyCustomHeaders`) rather than modifying `applyExtraParamsToAgent` signature, keeping the change additive and backward-compatible. - **Per-call headers take precedence** over plugin-injected headers (via spread order), preserving the existing behavior for provider-specific overrides. ### Plugin usage example ```typescript api.on("before_llm_request", (event, ctx) => { const headers: Record<string, string> = {}; if (ctx.senderId) { headers["X-Sender-Id"] = ctx.senderId; } if (ctx.channel) { headers["X-Source-Channel"] = ctx.channel; } return { headers }; }); ``` ### Hook context | Field | Type | Description | |---|---|---| | `event.provider` | `string` | LLM provider name (e.g. `"anthropic"`, `"openai"`) | | `event.modelId` | `string` | Model identifier (e.g. `"claude-sonnet-4-20250514"`) | | `event.headers` | `Record<string, string>` | Current headers (empty by default) | | `ctx.senderId` | `string?` | Sender identifier from the inbound message | | `ctx.senderName` | `string?` | Sender display name | | `ctx.channel` | `string?` | Message channel (e.g. `"telegram"`, `"discord"`) | | `ctx.agentId` | `string?` | Agent identifier | | `ctx.sessionKey` | `string?` | Session key | ### Files changed | File | Change | |---|---| | `src/plugins/types.ts` | Add `"before_llm_request"` to `PluginHookName`, define event/context/result types, add handler map entry | | `src/plugins/hooks.ts` | Add `runBeforeLlmRequest` runner (sequential modifying hook with header merging), add imports and re-exports | | `src/agents/pi-embedded-runner/extra-params.ts` | Add `applyCustomHeaders()` -- wraps `streamFn` to inject plugin-provided headers | | `src/agents/pi-embedded-runner.ts` | Add `applyCustomHeaders` to barrel export | | `src/agents/pi-embedded-runner/run/attempt.ts` | Wire hook call after `applyExtraParamsToAgent`, passing sender context | | `src/plugins/hooks.test.ts` | 5 tests: no hooks, single handler, multi-handler merge with priority, error catching, event passthrough | | `src/agents/pi-embedded-runner-extraparams.test.ts` | 3 tests: header injection, per-call precedence, empty headers no-op | ### Test plan - [x] `pnpm test` -- all existing tests pass (no regressions) - [x] `src/plugins/hooks.test.ts` -- new tests for `runBeforeLlmRequest` - [x] `src/agents/pi-embedded-runner-extraparams.test.ts` -- new tests for `applyCustomHeaders` <!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> This PR adds a new plugin modifying hook, `before_llm_request`, intended to let plugins inject custom HTTP headers into LLM API requests. It introduces new hook types (`PluginHookBeforeLlmRequestEvent/Context/Result`), a hook runner method (`runBeforeLlmRequest`) that executes handlers sequentially by priority and merges returned headers, and wires the hook into the embedded attempt runner to wrap the agent’s `streamFn` with the merged headers via `applyCustomHeaders`. The change fits into the existing plugin hook architecture by following the same “sequential modifying hook” pattern used for `before_agent_start` and `before_tool_call`, and integrates at the point where `streamFn` wrappers are assembled (after `applyExtraParamsToAgent`, before cache/payload logger wrappers). <h3>Confidence Score: 3/5</h3> - This PR is close to safe, but has a concrete runtime crash case in the new hook result merge that should be fixed before merging. - The overall design and wiring look consistent with existing hook patterns, but `runBeforeLlmRequest` will throw if any handler returns a result without `headers` (since `headers` is optional in the type). That’s a definite runtime failure path for plugin authors and should be addressed before merge. Tests also don’t cover this edge case. - src/plugins/hooks.ts <!-- greptile_other_comments_section --> <sub>(2/5) Greptile learns from your feedback when you react with thumbs up/down!</sub> <!-- /greptile_comment -->

Most Similar PRs