← Back to PRs

#15632: fix: use provider-qualified key in MODEL_CACHE for context window lookup

by linwebs open 2026-02-13 17:43 View on GitHub →
gateway commands agents stale size: S
#### Summary Fix incorrect context window display when the same model id exists across multiple providers with different context limits. `MODEL_CACHE` used bare model id as key, so the last-loaded provider overwrote earlier entries. lobster-biscuit #### Repro Steps 1. Configure two providers with the same model id but different context windows in `models.json`: ```json { "providers": { "provider-1": { "baseUrl": "http://provider-1/v1/", "apiKey": "", "api": "openai-completions", "models": [ { "id": "claude-4.6-opus", "name": "claude-4.6-opus", "reasoning": false, "input": ["text", "image"], "cost": { "input": 5, "output": 25, "cacheRead": 0.5, "cacheWrite": 6.25 }, "contextWindow": 200000, "maxTokens": 128000, "compat": { "supportsStore": false } } ] }, "provider-2": { "baseUrl": "http://provider-2/", "apiKey": "", "api": "anthropic-messages", "authHeader": true, "models": [ { "id": "claude-4.6-opus", "name": "claude-4.6-opus (1M)", "reasoning": true, "input": ["text", "image"], "cost": { "input": 5, "output": 25, "cacheRead": 0.5, "cacheWrite": 6.25 }, "contextWindow": 1000000, "maxTokens": 128000, "headers": { "anthropic-beta": "context-1m-2025-08-07" } } ] } } } ``` 2. `/model provider-1/claude-4.6-opus` → `/status` shows `Context: xx/1.0m` ❌ (should be 200k) 3. `/model provider-2/claude-4.6-opus` → `/status` shows `Context: xx/1.0m` ✅ #### Root Cause `MODEL_CACHE.set(m.id, m.contextWindow)` in `src/agents/context.ts` uses bare model id as key. When multiple providers define the same model id, last-loaded wins. Additionally, `entry.contextTokens` (cached in session store after each agent run) takes priority over fresh lookups in status display, persisting the wrong value across model switches. #### Behavior Changes - `MODEL_CACHE` now stores provider-qualified keys (e.g. `provider-1/claude-4.6-opus`) in addition to bare model id - All `lookupContextTokens` call sites prefer provider-qualified key when provider info is available, falling back to bare model id - `/status` context window lookup prioritizes provider-qualified match over cached session store value - `resolveContextTokens` and `resolveMemoryFlushContextWindowTokens` accept optional provider parameter - `SessionRow` type includes `modelProvider` for provider-aware context resolution in session list #### Codebase and GitHub Search - Searched `MODEL_CACHE`, `lookupContextTokens` — confirmed all call sites updated (14 files) - No existing tests for `lookupContextTokens` context window resolution #### Tests - No existing unit tests for `lookupContextTokens` — function is a best-effort cache lookup - Manual verification performed (see below) #### Manual Testing ##### Prerequisites - Two providers configured with same model id, different context windows (200k vs 1M) - One uses `openai-completions` API, other uses `anthropic-messages` API with `anthropic-beta: context-1m-2025-08-07` header ##### Steps 1. Start gateway with both providers configured 2. `/model provider-1/claude-4.6-opus` → `/status` 3. `/model provider-2/claude-4.6-opus` → `/status` 4. Verify context window displays correctly for each ##### Results - Before fix: Both show `Context: xx/1.0m` - After fix: provider-1 shows `Context: xx/200k` ✅, provider-2 shows `Context: xx/1.0m` ✅ #### Evidence CI `check` job lint failure (`no-control-regex` in `src/gateway/server/ws-connection.ts:22`) is a pre-existing issue in upstream main — not related to this PR. **Sign-Off** - Models used: claude-4.6-opus - Submitter effort: human-guided, AI-assisted (OpenClaw agent analyzed source, implemented fix, human verified) - Agent notes: Root cause identified by tracing `MODEL_CACHE` initialization in dist bundle, then confirmed in source. All 14 call sites updated with provider-qualified lookup + bare id fallback for backward compatibility. <!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> This PR fixes incorrect context-window displays when the same model id exists across multiple providers by: - Writing provider-qualified keys (e.g. `provider/model`) into `MODEL_CACHE` alongside the legacy bare-model-id key. - Updating context-window resolution call sites (status, sessions, agent runners, cron runs, memory flush) to prefer provider-qualified lookups when provider information is available. - Extending session listing rows to carry `modelProvider` so session views can resolve context windows provider-aware. Key behavior change: `/status` now prefers provider-qualified cache lookups over the cached session entry value to avoid persisting a wrong context window across model switches. <h3>Confidence Score: 3/5</h3> - This PR is close to safe to merge but has two logic issues that can persist incorrect provider/model context metadata. - Core provider-qualified cache approach is consistent across most call sites, but one followup persistence path stores the wrong provider while computing context tokens from a different provider, and one runner does an unguarded qualified lookup that can query `undefined/<model>`. - src/auto-reply/reply/followup-runner.ts, src/auto-reply/reply/agent-runner.ts <sub>Last reviewed commit: cc2113b</sub> <!-- greptile_other_comments_section --> <sub>(3/5) Reply to the agent's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!</sub> <!-- /greptile_comment -->

Most Similar PRs