#21141: fix- Auto-reply: improve user-facing error messages

## Summary Describe the problem and fix in 2–5 bullets: - Problem: Error messages from the embedded agent exposed technical details (e.g., "sandbox is not defined\nLogs: openclaw logs --follow") instead of user-friendly messages.  - Why it matters: Users see confusing technical errors instead of actionable guidance. - What changed: Added error classification using existing helper functions (`isRateLimitErrorMessage`, `isAuthErrorMessage`, etc.) to show friendly messages for rate limit, auth, billing, timeout errors. - What did NOT change (scope boundary): Error handling logic for context overflow and role ordering errors remains unchanged. ## Change Type (select all) - [ x] Bug fix - [ ] Feature - [x ] Refactor - [ ] Docs - [ ] Security hardening - [ ] Chore/infra ## Scope (select all touched areas) - [ ] Gateway / orchestration - [ x] Skills / tool execution - [ ] Auth / tokens - [ ] Memory / storage - [ ] Integrations - [ ] API / contracts - [ ] UI / DX - [ ] CI/CD / infra ## Linked Issue/PR - Closes # - Related # ## User-visible / Behavior Changes Users now see friendly error messages: - "The AI service is busy. Please wait a moment and try again." (rate limit/overloaded) - "I couldn't connect to the AI service. Please verify your API key is configured correctly." (auth errors) - "I've reached my limit with the AI service. Please check your account balance and try again." (billing errors) - "The request timed out. Please try again, or start a fresh session with /new." (timeout errors) - "Something unexpected happened. Try /new to start a fresh conversation, or try again in a moment." (unknown errors) ## Security Impact (required) - New permissions/capabilities? (No) - Secrets/tokens handling changed? (No) - New/changed network calls? (No) - Command/tool execution surface changed? (No) - Data access scope changed? (No) ## Repro + Verification ### Environment - OS: macOS (Darwin) - Runtime/container: Node 22+ with pnpm - Model/provider: Any AI provider (OpenAI, Anthropic, etc.) - Integration/channel (if any): Auto-reply (embedded Pi agent) - Relevant config (redacted): None - error handling is internal ### Steps 1. Configure OpenClaw with an AI provider 2. Trigger a condition that causes the embedded agent to throw an error (e.g., invalid API key, rate limit, timeout) 3. Observe the error message returned to the user in the messaging channel ### Expected - User-friendly message based on error type (e.g., "The AI service is busy. Please wait a moment and try again.") ### Actual - Friendly messages displayed correctly - verified via unit and e2e tests ## Evidence Attach at least one: - [ x] Failing test/log before + passing after - Updated tests pass: - 24 unit tests: src/auto-reply/reply/agent-runner.runreplyagent.test.ts - 10 e2e tests: src/auto-reply/reply.triggers.trigger-handling.includes-error-cause-embedded-agent-throws.e2e.test.ts - [ ] Trace/log snippets - [ ] Screenshot/recording - [ ] Perf numbers (if relevant) ## Human Verification (required) What you personally verified (not just CI), and how: - Verified scenarios: All 5 error types (rate limit, auth, billing, timeout, unknown) return correct friendly messages - Edge cases checked: Context overflow and role ordering errors still work as before - What you did **not** verify: Live integration with real API errors ## Compatibility / Migration - Backward compatible? (Yes) - Config/env changes? (No) - Migration needed? (No) ## Failure Recovery (if this breaks) - How to disable/revert this change quickly: Revert commit 30a9bcac6 - Files/config to restore: src/auto-reply/reply/agent-runner-execution.ts - Known bad symptoms reviewers should watch for: Users seeing generic "something unexpected" message for known error types that should show specific friendly messages ## Risks and Mitigations List only real risks for this PR. Add/remove entries as needed. If none, write `None`. - Risk: Error classification helper functions may not detect all error formats - Mitigation: Functions are already battle-tested in other parts of the codebase (src/agents/pi-embedded-helpers.ts); fallback generic message ensures users always get a response **AI-assisted PR** (built with OpenCode and MiniMax M2.5) - Testing: Fully tested (24 unit + 10 e2e tests pass) ## Build prompt in Markdown file- # Execution Plan: Improve User-Friendly Error Messages > **IMPORTANT:** This plan file must NOT be committed to the repository. Keep it for reference and PR description. --- ## Phase 1: Preparation (Steps 0-3) - [ ] **Step 0: Verify Clean Working Tree** ```bash git status ``` Ensure working tree is clean (no uncommitted changes) before starting. - [ ] **Step 1: Run Baseline Tests** ```bash pnpm test src/auto-reply/reply/agent-runner.runreplyagent.test.ts pnpm vitest run --config vitest.e2e.config.ts src/auto-reply/reply.triggers.trigger-handling.includes-error-cause-embedded-agent-throws.e2e.test.ts ``` - [ ] **Step 2: Verify Branch** ```bash git branch ``` Should show: `refactor/improve-error-messages` - [ ] **Step 3: Verify Imports Available** Confirm these functions exist in `src/agents/pi-embedded-helpers.ts`: - `isRateLimitErrorMessage()` - `isAuthErrorMessage()` - `isBillingErrorMessage()` - `isTimeoutErrorMessage()` - `isOverloadedErrorMessage()` --- ## Phase 2: Code Changes (Steps 4-6) - [ ] **Step 4: Edit `src/auto-reply/reply/agent-runner-execution.ts`** #### 4.1 Add imports (replace lines 7-13): ```typescript import { isAuthErrorMessage, isBillingErrorMessage, isCompactionFailureError, isContextOverflowError, isLikelyContextOverflowError, isOverloadedErrorMessage, isRateLimitErrorMessage, isTimeoutErrorMessage, isTransientHttpError, sanitizeUserFacingText, } from "../../agents/pi-embedded-helpers.js"; ``` #### 4.2 Add error classification after line 428: ```typescript // Classify error type for user-friendly messaging const isRateLimit = isRateLimitErrorMessage(message); const isAuthError = isAuthErrorMessage(message); const isBillingError = isBillingErrorMessage(message); const isTimeoutError = isTimeoutErrorMessage(message); const isOverloaded = isOverloadedErrorMessage(message); ``` #### 4.3 Replace lines 515-524 with user-friendly messages: ```typescript let fallbackText: string; if (isContextOverflow) { fallbackText = "⚠️ Context overflow — prompt too large for this model. Try a shorter message or a larger-context model."; } else if (isRoleOrderingError) { fallbackText = "⚠️ Message ordering conflict - please try again. If this persists, use /new to start a fresh session."; } else if (isRateLimit || isOverloaded) { fallbackText = "The AI service is busy. Please wait a moment and try again."; } else if (isAuthError) { fallbackText = "I couldn't connect to the AI service. Please verify your API key is configured correctly."; } else if (isBillingError) { fallbackText = "I've reached my limit with the AI service. Please check your account balance and try again."; } else if (isTimeoutError) { fallbackText = "The request timed out. Please try again, or start a fresh session with /new."; } else { fallbackText = "Something unexpected happened. Try /new to start a fresh conversation, or try again in a moment."; } ``` #### 4.4 Remove lines 516-519 (old safeMessage/trimmedMessage variables - no longer needed) - [ ] **Step 5: Run Lint** ```bash pnpm check ``` - [ ] **Step 6: Run TypeScript** ```bash pnpm tsgo ``` --- ## Phase 3: Test Updates (Steps 7-10) - [ ] **Step 7: Edit Test File - Existing Test** Open: `src/auto-reply/reply.triggers.trigger-handling.includes-error-cause-embedded-agent-throws.e2e.test.ts` Update line 69-71: - From: `"⚠️ Agent failed before reply: sandbox is not defined.\nLogs: openclaw logs --follow"` - To: `"Something unexpected happened. Try /new to start a fresh conversation, or try again in a moment."` - [ ] **Step 8: Add New Tests** Add these 4 new tests to the same file (after the existing test): #### 8.1 Rate Limit Error Test ```typescript it("returns friendly message for rate limit errors", async () => { await withTempHome(async (home) => { const runEmbeddedPiAgentMock = getRunEmbeddedPiAgentMock(); runEmbeddedPiAgentMock.mockRejectedValue( new Error("rate_limit_exceeded: API rate limit exceeded"), ); const res = await getReplyFromConfig(BASE_MESSAGE, {}, makeCfg(home)); const text = Array.isArray(res) ? res[0]?.text : res?.text; expect(text).toBe("The AI service is busy. Please wait a moment and try again."); }); }); ``` #### 8.2 Auth Error Test ```typescript it("returns friendly message for auth errors", async () => { await withTempHome(async (home) => { const runEmbeddedPiAgentMock = getRunEmbeddedPiAgentMock(); runEmbeddedPiAgentMock.mockRejectedValue(new Error("401 Unauthorized: Invalid API key")); const res = await getReplyFromConfig(BASE_MESSAGE, {}, makeCfg(home)); const text = Array.isArray(res) ? res[0]?.text : res?.text; expect(text).toBe( "I couldn't connect to the AI service. Please verify your API key is configured correctly.", ); }); }); ``` #### 8.3 Billing Error Test ```typescript it("returns friendly message for billing errors", async () => { await withTempHome(async (home) => { const runEmbeddedPiAgentMock = getRunEm...