#13370: Tools: rewrite Grok parser, add Tavily provider, multi-provider configure wizard

#### Summary Rewrites the Grok web search parser, adds Tavily as a new provider, refactors `runWebSearch` into a clean provider-dispatch architecture, and updates the configure wizard to support all four web search providers with a uniform config layout. Closes #13171. Supersedes #13263. lobster-biscuit #### Behavior Changes - **Grok web_search now works correctly** — previous parser failed to extract text/citations from the xAI Responses API format (`type: "message"` + `status: "completed"`) - **Tavily available as a new web_search provider** — configurable via `tools.web.search.provider: "tavily"` with API key via config or `TAVILY_API_KEY` env var; supports `basic`, `advanced`, `fast`, and `ultra-fast` search depths - **Configure wizard supports all 4 providers** — `openclaw configure --section web` now lets users pick Brave, Perplexity, Grok, or Tavily and enter per-provider API keys (previously Brave-only) - **Brave API key normalized** — moved from `tools.web.search.apiKey` to `tools.web.search.brave.apiKey` so all providers follow the same `[provider].apiKey` pattern; legacy migration auto-moves existing configs - **`GrokSearchResult.output` is now optional** — handles malformed/edge API responses without crashing - **Dead `inlineCitations` config field removed** from Grok config - **Onboarding summary is provider-agnostic** — no longer mentions only Brave Search No changes to Brave or Perplexity provider runtime behavior. #### Codebase and GitHub Search - Searched for existing Tavily integrations — none found; this is the first Tavily provider - Verified `performProviderSearch` dispatch covers all four providers with `default: throw` exhaustiveness guard - Confirmed `buildSearchCacheKey` produces unique keys per provider (covered by tests) - Reviewed Greptile feedback on superseded #13263 — both flagged issues (citation URL wrapping, dead `inlineCitations` config) resolved here - Searched all references to `search.apiKey` / `search?.apiKey` to ensure no stale paths remain after the Brave key normalization #### Tests - Split `web-search.test.ts` into focused per-provider files: - `web-search.test.ts` — shared (perplexity, freshness, cache key, provider dispatch) - `web-search.brave.test.ts` — Brave search tests - `web-search.grok.test.ts` — Grok config, fixtures, error handling - `web-search.tavily.test.ts` — Tavily config, fixtures, error handling - Fixture-driven tests for Grok and Tavily response parsing - Updated `config.env-vars.test.ts` to use new `brave.apiKey` path - All existing tests pass: - `pnpm vitest run src/agents/tools/web-search` — 81 tests pass (4 files) - `pnpm vitest run src/commands/configure.wizard.test.ts` — 1 test pass - `pnpm vitest run src/config/config.web-search-provider.test.ts` — 2 tests pass - `pnpm vitest run src/config/config.legacy-config-detection` — 57 tests pass - `pnpm vitest run src/config/config.env-vars.test.ts` — 4 tests pass - `pnpm build && pnpm check` — clean #### Manual Testing ##### Prerequisites - xAI API key (`XAI_API_KEY`) - Tavily API key (`TAVILY_API_KEY`) ##### Steps 1. Configure Grok provider (`tools.web.search.provider: "grok"`) and run a web search query — verify results contain text content and citation URLs 2. Configure Tavily provider (`tools.web.search.provider: "tavily"`) and run a web search query — verify results contain titles, URLs, and descriptions 3. Verify Brave search still works with no config changes (default provider) 4. Run `openclaw configure --section web` — verify provider selection (Brave/Perplexity/Grok/Tavily), per-provider API key prompt, and correct env var hints 5. Place `tools.web.search.apiKey: "BSA-test"` in config, reload — verify legacy migration moves it to `tools.web.search.brave.apiKey` 6. Run `openclaw onboard` — verify the web search summary note is provider-agnostic ##### Results - [x] Grok search via live xAI key — working - [x] Tavily search via live key — working - [x] Configure wizard: all 4 providers selectable, API key stored at correct path - [x] Legacy migration: `apiKey` → `brave.apiKey` auto-migrated on load - [x] Onboarding summary: provider-agnostic messaging - [ ] Brave search via live key — no key available (no regression in code paths) **Sign-Off** - Models used: Claude Opus 4.6 - Submitter effort: high — multi-session development across parser rewrite, new provider, refactor, test reorganization, configure wizard rewrite, and config normalization with legacy migration - Agent notes: Supersedes #13263 with expanded scope; all Greptile review findings from that PR are addressed here <!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> This PR refactors `web_search` into a provider-dispatch architecture, fixes Grok (xAI Responses API) parsing, adds a new Tavily provider, and updates config + configure wizard to support provider-specific API keys under `tools.web.search.<provider>.apiKey` (with a legacy migration for Brave’s old `tools.web.search.apiKey`). The changes fit the existing tool/config architecture by: - keeping `web_search` as a single tool with a stable input schema, while selecting the runtime provider from config - centralizing cache-key generation and provider execution in `src/agents/tools/web-search.ts` - extending the Zod runtime schema + config types and migrating legacy config keys on load. Main issue to address before merge: Grok citation extraction currently doesn’t filter/validate annotations, which can produce invalid citation objects if the API returns non-URL annotations or missing fields. <h3>Confidence Score: 4/5</h3> - This PR looks close to safe to merge, with one concrete runtime issue to fix in Grok citation parsing. - Most changes are additive/refactoring with solid test coverage and schema updates. The remaining concern is that Grok annotation-to-citation mapping doesn’t filter/validate fields, which can yield invalid citation objects and break consumers that expect valid URLs/indices. Also a minor text-file newline issue in CLAUDE.md will cause churn. - src/agents/tools/web-search.ts, CLAUDE.md _{Last reviewed commit: 4ded777} <!-- greptile_other_comments_section --> <!-- /greptile_comment -->