← Back to PRs

#21503: feat(doctor): validate fallback model providers are defined (#20909)

by echoVic open 2026-02-20 01:12 View on GitHub →
commands size: M trusted-contributor
## Problem `openclaw doctor scan` does not validate that fallback models have their providers properly defined. This leads to silent configuration errors that only surface at runtime when the primary model is unavailable and the fallback chain is triggered. For example, configuring `kimi-coding/k2p5` as a fallback without defining `kimi-coding` in `models.providers` passes doctor validation but fails when Claude goes down. ## Fix Add a new `noteFallbackModelHealth()` check to `openclaw doctor` that: 1. **Collects all fallback model references** from: - `agents.defaults.model.fallbacks` - `agents.defaults.imageModel.fallbacks` - Per-agent `model.fallbacks` in `agents.list[]` - `agents.defaults.subagents.model.fallbacks` 2. **Validates each provider** against known sources: - Model catalog (built-in providers: anthropic, openai, google, etc.) - Explicit `models.providers` config entries - CLI backends (`agents.defaults.cliBackends`) 3. **Reports actionable warnings** with the exact config path and provider name. Example output: ``` ╭ Fallback models │ Fallback model chain references undefined providers. │ These fallbacks will fail at runtime when the primary model is unavailable. │ │ - "kimi-coding/k2p5" in agents.defaults.model.fallbacks: provider "kimi-coding" is not defined │ in models.providers and not found in the model catalog. │ │ Fix: add the missing provider to models.providers in your config, │ or run: openclaw auth add --provider <provider-name> ╰ ``` ## Changes - **`src/commands/doctor-fallback-models.ts`** — New validation module - **`src/commands/doctor-fallback-models.test.ts`** — 9 test cases covering: - No fallbacks configured (no-op) - All providers in catalog (pass) - Provider in explicit `models.providers` (pass) - Undefined provider (warning) - Multiple undefined providers - imageModel fallbacks - Per-agent fallbacks - Deduplication of identical refs - CLI backends recognized as valid - **`src/commands/doctor.ts`** — Wire up the new check after `noteMemorySearchHealth` ## Testing ``` ✓ emits no note when there are no fallbacks configured ✓ emits no note when all fallback providers are in the catalog ✓ emits no note when fallback provider is in models.providers ✓ emits a warning when fallback provider is undefined ✓ reports multiple undefined providers ✓ checks imageModel fallbacks ✓ checks per-agent fallbacks ✓ deduplicates identical fallback refs ✓ recognizes CLI backends as valid providers Test Files 1 passed (1) Tests 9 passed (9) ``` Closes #20909 <!-- greptile_comment --> <h3>Greptile Summary</h3> Adds validation for fallback model providers in `openclaw doctor scan`. The implementation correctly validates fallback models from multiple config locations and provides actionable error messages when providers are undefined. **Key changes:** - New `noteFallbackModelHealth()` function validates fallback providers against known sources (model catalog, explicit config, CLI backends) - Comprehensive test coverage (9 test cases covering all major scenarios) - Properly integrated into doctor command flow **Issue found:** - Missing validation for per-agent subagent model fallbacks (`agents.list[].subagents.model.fallbacks`). The code validates `agents.defaults.subagents.model.fallbacks` but doesn't check the per-agent override. This is a minor gap since per-agent subagent models are supported and used at runtime (confirmed in `src/agents/model-selection.ts:325`). <h3>Confidence Score: 4/5</h3> - Safe to merge with one minor gap in validation coverage - The implementation is well-structured with strong test coverage and correctly handles the main fallback validation scenarios. However, it's missing validation for per-agent subagent model fallbacks, which reduces the score from 5 to 4. The missing case is an edge scenario that's unlikely to be commonly used, but it should be included for completeness since the config schema supports it and it's used at runtime. - src/commands/doctor-fallback-models.ts requires a minor addition to validate per-agent subagent model fallbacks <sub>Last reviewed commit: d543412</sub> <!-- greptile_other_comments_section --> <!-- /greptile_comment -->

Most Similar PRs