← Back to PRs

#10505: feat(compaction): add timeout, model override, and diagnostic logging

by thebtf open 2026-02-06 15:49 View on GitHub →
agents size: L
## Summary Add configurable timeout, optional model override, and structured diagnostic logging to session compaction. Also disable extended thinking on the compaction model to reduce latency and cost. lobster-biscuit ## Use Cases - **Timeout**: compaction can hang indefinitely on slow providers or large sessions. A configurable `timeoutMs` (default 120s) aborts the run and returns a clear `compaction_timeout` error so the caller can retry or skip. - **Model override**: use a cheaper/faster model for summarization (e.g. `openai/gpt-4o-mini`) while keeping the primary model for agent turns. Gated behind `overrideModel: true` so existing configs are unaffected. - **Disable reasoning**: the Pi SDK's `generateSummary()` hardcodes `reasoning: "high"`, which triggers extended thinking on reasoning-capable models. This adds 16K+ token budget and significant latency to compaction with marginal quality benefit. We now set `reasoning: false` on the model passed to `createAgentSession` for compaction. - **Diagnostic logging**: structured `log.info` on start/done with session ID, model, timeout, token counts (before/after), and duration. `log.warn` on timeout and errors. ## Behavior Changes - New config fields under `agents.defaults.compaction`: - `timeoutMs` (number, default 120000): max ms for compaction summarization - `model` (string): model ref for compaction (e.g. `"openai/gpt-4o-mini"`) - `overrideModel` (boolean, default false): enable the model override - Compaction now logs start/done/timeout/error events at `info`/`warn` level - Reasoning-capable models get `reasoning: false` during compaction sessions - No change to default behavior when config fields are omitted ## Existing Functionality Check I searched the codebase for existing compaction config — `AgentCompactionConfig` in `types.agent-defaults.ts` already has `mode`, `reserveTokensFloor`, `maxHistoryShare`, `memoryFlush`. The new fields extend this type cleanly. ## Tests All 15 tests pass (9 new in `compact.test.ts`, 6 existing in `run.overflow-compaction.test.ts`): ``` ✓ returns ok on successful compaction ✓ aborts and returns timeout error when compaction exceeds timeoutMs ✓ uses compaction model override when overrideModel is true ✓ ignores compaction model when overrideModel is false ✓ ignores compaction model when overrideModel is omitted ✓ returns error for unknown model override ✓ disables reasoning on model passed to createAgentSession ✓ passes model unchanged when reasoning is already falsy ✓ re-throws non-timeout errors from compact() ✓ retries after successful compaction on context overflow promptError ✓ returns error if compaction fails ✓ retries compaction up to 3 times before giving up ✓ succeeds after second compaction attempt ✓ treats compaction timeout as failed compaction and returns overflow error ✓ does not attempt compaction for compaction_failure errors ``` ## Evidence Config example: ```json5 { agents: { defaults: { compaction: { timeoutMs: 90000, model: "openai/gpt-4o-mini", overrideModel: true, } } } } ``` Log output (start/done): ``` compaction: start sessionId=agent:main:main model=openai/gpt-4o-mini timeoutMs=90000 compaction: done sessionId=agent:main:main model=openai/gpt-4o-mini tokensBefore=180432 tokensAfter=12500 durationMs=8432 ``` <!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> - Extends agent defaults compaction config with `timeoutMs`, a gated `overrideModel` flag, and optional `model` override. - Updates embedded Pi session compaction to support model override (when enabled), add structured start/done/timeout/error logging, and attempt to abort long-running compactions. - Forces `reasoning: false` on the model passed to the SDK compaction session to avoid extended thinking during summarization. - Adds unit tests covering timeout behavior, model override gating, and reasoning disabling, plus an overflow-run-loop test asserting timeout is treated as compaction failure. <h3>Confidence Score: 3/5</h3> - This PR is close to mergeable but has two correctness issues around model override consistency and enforceable timeout behavior. - Score reduced because the compaction model override currently only affects `resolveModel()`/`createAgentSession()` while other provider/model-dependent logic still uses the primary provider/model, which can break compaction when switching providers/APIs. Also, the timeout relies on `abortCompaction()` causing `compact()` to reject; if abort is best-effort, compaction can still hang indefinitely despite `timeoutMs`. Fixing these should make the change set safe. - src/agents/pi-embedded-runner/compact.ts <!-- greptile_other_comments_section --> <sub>(2/5) Greptile learns from your feedback when you react with thumbs up/down!</sub> <!-- /greptile_comment --> --- ## Validation - [x] `pnpm build` — passes - [x] `pnpm check` — passes - [x] `pnpm test` — 15 tests pass (9 new + 6 existing) ## Contribution checklist - [x] **Focused scope**: Compaction timeout, model override, diagnostic logging - [x] **What + why**: described above - [x] **AI-assisted**: Yes, Claude Code was used for implementation. Testing level: fully tested (15 tests)

Most Similar PRs