#19948: feat: add PostHog LLM Analytics extension

Howdy 👋 , Andy here from the PostHog LLM Analytics team. We have had a few users ask for more native support for openclaw in PostHog. So the idea here is for a PostHog extension for folks who want to use PostHog for observability on their openclaw stuff. ## Summary - **Problem:** OpenClaw has no native PostHog integration — users wanting LLM observability have to build custom instrumentation - **Why it matters:** PostHog LLM Analytics provides cost tracking, latency monitoring, and conversation tracing, but needs structured `$ai_*` events - **What changed:** New `extensions/posthog/` plugin that captures `$ai_generation`, `$ai_span`, and `$ai_trace` events from OpenClaw lifecycle hooks. Includes privacy mode, configurable trace grouping, session windowing, and Anthropic→OpenAI message format normalization. - **What did NOT change:** No changes to OpenClaw core, plugin SDK, or any existing extensions See [`extensions/posthog/README.md`](https://github.com/andrewm4894/openclaw/blob/feat/posthog-extension/extensions/posthog/README.md) for full configuration, tracked properties, and usage details. ## Change Type (select all) - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Docs - [ ] Security hardening - [ ] Chore/infra ## Scope (select all touched areas) - [ ] Gateway / orchestration - [ ] Skills / tool execution - [ ] Auth / tokens - [ ] Memory / storage - [x] Integrations - [ ] API / contracts - [ ] UI / DX - [ ] CI/CD / infra ## Linked Issue/PR - Related https://github.com/openclaw/openclaw/discussions/22592 - Docs pr here - https://github.com/PostHog/posthog.com/pull/15196 ## User-visible / Behavior Changes - New `posthog` plugin available under `plugins.entries.posthog` in `openclaw.json` - When enabled, LLM generations, tool calls, and traces are sent to PostHog as `$ai_*` events - Privacy mode (opt-in) redacts all I/O content while preserving metrics - Key config: `apiKey` (required), `traceGrouping` (`"message"` | `"session"`), `sessionWindowMinutes` (default: 60), `privacyMode` (default: false) - `$ai_session_id` uses windowed IDs (`"{sessionKey}:{windowId}"`) that rotate after inactivity, preventing shared keys like WebChat's `agent:main:main` from collapsing into one eternal session ## Security Impact (required) - New permissions/capabilities? `No` - Secrets/tokens handling changed? `Yes` — accepts a PostHog `apiKey` in plugin config (stored in user's `openclaw.json`, same as other plugin secrets) - New/changed network calls? `Yes` — sends `$ai_*` events to PostHog via `posthog-node` SDK (fire-and-forget, async batched) - Command/tool execution surface changed? `No` - Data access scope changed? `No` — only reads data already in hook payloads - **Mitigation:** Privacy mode available (`privacyMode: true`) to redact all I/O content. Even without it, data only goes to the user's own PostHog project. ## Repro + Verification ### Environment - OS: macOS - Runtime/container: Node.js 22, pnpm - Model/provider: OpenRouter / Anthropic Claude Sonnet 4 - Integration/channel: WebChat ### Steps 1. Configure `posthog` plugin in `openclaw.json` with API key 2. Start gateway: `node openclaw.mjs gateway --force --allow-unconfigured` 3. Open WebChat and send messages 4. Check PostHog LLM Analytics dashboard ### Expected - `$ai_generation` events appear with model, tokens, cost, latency - `$ai_session_id` is windowed (e.g., `agent:main:main:a1b2c3d4`) ### Actual - Verified in PostHog project ## Evidence - [x] Failing test/log before + passing after - [x] Trace/log snippets - [x] Screenshot/recording - [ ] Perf numbers (if relevant) 47 tests passing (`npx vitest run --config vitest.extensions.config.ts extensions/posthog`): 29 event builder tests + 18 plugin integration tests. Trace and generations end up in PostHog like this: example final trace in PostHog - https://gist.github.com/andrewm4894/d2aeb465cd669e39f41e2267a62f1b66 <img width="2584" height="1698" alt="image" src="https://github.com/user-attachments/assets/7b54abe3-ad6c-4c1e-8d90-ceabfca62c6e" /> ## Human Verification (required) - Verified scenarios: Manual WebChat testing — `$ai_generation` events with cost, stop reason, tool_calls format, windowed session IDs - Edge cases checked: Privacy mode redaction, missing usage data, orphan llm_output, stale run cleanup - What you did **not** verify: `$ai_trace` for non-WebChat channels, `$ai_span` in live environment (unit tested only), `traceGrouping: "session"` with real chat channel ## Compatibility / Migration - Backward compatible? `Yes` — new extension, no changes to existing code - Config/env changes? `Yes` — new opt-in `posthog` plugin entry in `openclaw.json` - Migration needed? `No` ## Failure Recovery (if this breaks) - How to disable/revert: Set `"enabled": false` in plugin config or remove the entry - Files/config to restore: Only `~/.openclaw/openclaw.json` - Known bad symptoms: PostHog client init failures (network), memory growth from run state maps (mitigated by 5-min stale cleanup) ## Risks and Mitigations - Risk: Memory growth from in-flight run state maps under heavy load - Mitigation: Stale run cleanup on every `llm_input` with 5-minute TTL - Risk: PostHog SDK network failures - Mitigation: Fire-and-forget with async batching (flushAt: 20, flushInterval: 10s), doesn't block gateway <!-- greptile_comment --> <h3>Greptile Summary</h3> New PostHog LLM Analytics extension that captures `$ai_generation`, `$ai_span`, and `$ai_trace` events from OpenClaw lifecycle hooks. The extension is self-contained under `extensions/posthog/` with no changes to existing functionality — only 3 type exports were added to `src/plugin-sdk/index.ts`. - Hooks into `llm_input`, `llm_output`, `after_tool_call`, and `message.processed` diagnostic events to build structured PostHog analytics events - Includes Anthropic-to-OpenAI message format normalization for PostHog LLM Analytics compatibility - Supports privacy mode (redacts I/O content), configurable trace grouping (`"message"` or `"session"`), and session windowing with timeout-based rotation - Well-tested with 47 tests covering event builders, plugin lifecycle, trace grouping modes, session windowing, and edge cases - Follows repo conventions: `openclaw` in `devDependencies` with `workspace:*`, production dep in `dependencies`, standard plugin entry point pattern - The `lastActiveSessionKey` fallback for `after_tool_call` (where upstream does not pass `sessionKey` in context) could misattribute tool spans under concurrent multi-session load — see inline comment - Session-keyed Maps (`lastRunId`, `lastOutputAt`, `sessionWindows`) grow unboundedly for long-running gateways — worth adding periodic eviction <h3>Confidence Score: 4/5</h3> - This PR is safe to merge — it adds a new self-contained extension with no changes to core behavior, and the identified issues are edge cases under concurrent load - Score reflects clean architecture, thorough testing, and adherence to repo conventions. Deducted one point for the `lastActiveSessionKey` race condition that can cause misattribution of tool spans under concurrent multi-session load, and the unbounded growth of session-keyed Maps in long-running gateways. Both are non-critical for typical single-user or low-concurrency deployments. - `extensions/posthog/src/plugin.ts` — review the `lastActiveSessionKey` race condition and session-keyed Map eviction strategy _{Last reviewed commit: 614d2bb} <!-- greptile_other_comments_section --> <!-- /greptile_comment -->