← Back to PRs

#19920: fix(memory): populate FTS index in FTS-only mode so search returns results

by forketyfork open 2026-02-18 09:22 View on GitHub →
size: M
## Summary - **Problem:** When no embedding provider is available (missing API key, network issue, etc.), the builtin memory backend falls into FTS-only mode. `syncMemoryFiles`, `syncSessionFiles`, and `indexFile` all returned early without indexing anything, leaving the FTS table empty. Additionally, the `search()` method fired sync as fire-and-forget, so even if sync were attempted, the search would race ahead and query the still-empty index. Every `memory_search` call returned zero results. - **Why it matters:** The agent tool can end up with `provider: "none"` and hit this empty-index path, making memory search silently useless. The CLI wasn't affected because it typically runs with a working embedding provider. - **What changed:** Removed the early returns so files are still chunked and written to the `chunks` and `chunks_fts` tables with an `"fts-only"` model sentinel and empty embeddings. Embedding generation and vector insertion are skipped as before. In `search()`, FTS-only mode now awaits any in-flight sync (or triggers one if needed) before querying, so the index is guaranteed to be populated. Two `this.provider.model` references in stale-file cleanup were null-unsafe and are now guarded. - **What did NOT change:** The vector/hybrid search path is untouched. When an embedding provider is available, behavior is identical to before. The fire-and-forget sync in the normal hybrid path is unchanged. ## Change Type (select all) - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Docs - [ ] Security hardening - [ ] Chore/infra ## Scope (select all touched areas) - [ ] Gateway / orchestration - [ ] Skills / tool execution - [ ] Auth / tokens - [x] Memory / storage - [ ] Integrations - [ ] API / contracts - [ ] UI / DX - [ ] CI/CD / infra ## Linked Issue/PR - Related: none (discovered during manual testing) ## User-visible / Behavior Changes - `memory_search` tool now returns FTS results when no embedding provider is available, instead of silently returning zero hits. ## Security Impact (required) - New permissions/capabilities? `No` - Secrets/tokens handling changed? `No` - New/changed network calls? `No` - Command/tool execution surface changed? `No` - Data access scope changed? `No` ## Repro + Verification ### Environment - OS: macOS - Runtime/container: Node.js (pnpm) - Model/provider: Any (issue triggers when embedding provider is unavailable) - Integration/channel: Agent tool (`memory_search`) - Relevant config: `memorySearch.provider: "openai"` with missing or invalid API key ### Steps 1. Remove the `remote.apiKey` under `memorySearch` in `~/.openclaw/openclaw.json` 2. Build openclaw from source (`pnpm build`) 3. Start the agent and send a message that triggers `memory_search` ### Expected - FTS results returned from indexed memory files with `provider: "none"`, `mode: "fts-only"` ### Actual (before fix) - Zero results, FTS table is empty because sync methods bail out early and search doesn't wait for sync ## Evidence - [x] Failing test/log before + passing after - [ ] Trace/log snippets - [ ] Screenshot/recording - [ ] Perf numbers (if relevant) New test file `manager.fts-only.test.ts` with 5 tests covering FTS-only indexing, auto-sync on first search, status reporting, and stale file cleanup. ## Human Verification (required) - Verified scenarios: Built openclaw from this branch and ran the agent without an embedding provider API key. Before the fix, the agent reported `provider: none, mode: fts-only, no results for "Sergei"`. After the fix, the agent reports `provider: none, mode: fts-only, results present (keyword matches), no semantic embeddings active`. Also ran `pnpm test` (5592 tests pass) and `pnpm tsgo` (no type errors). - Edge cases checked: Stale file removal in FTS-only mode (dedicated test), status reporting with `searchMode: "fts-only"`, auto-sync on first search without explicit sync call. - What you did **not** verify: Behavior when switching between provider-available and provider-unavailable states within the same process lifetime. ## Compatibility / Migration - Backward compatible? `Yes` - Config/env changes? `No` - Migration needed? `No` ## Failure Recovery (if this breaks) - How to disable/revert this change quickly: Revert the commit; the early-return guards are restored. - Files/config to restore: None - Known bad symptoms reviewers should watch for: FTS results appearing with unexpected content, or `chunks` table growing with `"fts-only"` model entries when an embedding provider is actually available. ## Risks and Mitigations - **Risk:** The `"fts-only"` model sentinel is a new convention; stale cleanup queries filter by model, so mixing real model names with `"fts-only"` could leave orphan rows if a user switches between provider-available and provider-unavailable states. - **Mitigation:** The stale cleanup now uses `this.provider?.model ?? "fts-only"`, which matches the model used at index time, so the cleanup stays consistent within a given mode. - **Risk:** The awaited pre-search sync in FTS-only mode adds latency to the first search call. - **Mitigation:** This only affects the first search when the index is dirty. Subsequent searches skip the sync. The alternative (zero results) is strictly worse. <!-- greptile_comment --> <h3>Greptile Summary</h3> This PR fixes a bug where `memory_search` returned zero results when no embedding provider was available (FTS-only mode). The root causes were: (1) early returns in `syncMemoryFiles`, `syncSessionFiles`, and `indexFile` that skipped all indexing when `!this.provider`, leaving the FTS table empty; (2) fire-and-forget sync in `search()` that raced ahead of the still-empty index. **Key changes:** - Removed early returns in sync/index methods so files are chunked and written to `chunks` and `chunks_fts` tables with an `"fts-only"` model sentinel and empty embeddings, while embedding generation and vector insertion remain skipped - Added awaited pre-search sync in FTS-only mode (`manager.ts:238-244`) to guarantee the index is populated before querying - Fixed null-unsafe `this.provider.model` references in stale-file cleanup with `this.provider?.model ?? "fts-only"` - Added `manager.fts-only.test.ts` with 4 tests covering FTS-only indexing, auto-sync, status reporting, and stale file cleanup The hybrid/vector search path is untouched — when an embedding provider is available, behavior is identical to before. <h3>Confidence Score: 4/5</h3> - This PR is safe to merge — it fixes a clear bug with well-scoped changes and good test coverage, with only a minor style suggestion. - The changes are focused and correct. The FTS-only code path is well-isolated from the normal hybrid/vector path. The pre-search sync logic correctly handles all timing scenarios. The null-safety fixes for `this.provider.model` prevent runtime crashes. New tests cover the main scenarios. One minor style concern: the `"fts-only"` sentinel is defined as a constant in one file but used as raw string literals in another. - `src/memory/manager-sync-ops.ts` has `"fts-only"` string literals that should ideally reference the `FTS_ONLY_MODEL` constant for consistency. <sub>Last reviewed commit: f2fec12</sub> <!-- greptile_other_comments_section --> <sub>(3/5) Reply to the agent's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!</sub> <!-- /greptile_comment -->

Most Similar PRs