#4231: fix(memory): use sqlite-vec knn (MATCH+k) for vector search
## Summary
This speeds up `memory_search` vector retrieval when sqlite-vec is enabled by switching from a brute-force `ORDER BY vec_distance_cosine(...)` scan over the full vec0 table to sqlite-vec KNN preselection (`MATCH` + `k`).
## Root cause
The sqlite-vec path was still effectively doing an O(N) scan + global sort:
- compute `vec_distance_cosine(v.embedding, ?)` for every row
- `ORDER BY dist ASC LIMIT ?`
On larger stores this can be tens of seconds per query even on decent CPU/RAM.
## Fix
- Use sqlite-vec KNN (`embedding MATCH ? AND k = ?`) to select a small candidate set.
- Compute cosine distance only for the candidates, then sort/limit.
JS fallback behavior is unchanged when sqlite-vec is unavailable.
## Repro environment (my setup)
Configuration (redacted, but precise):
- `memorySearch.enabled = true`
- `memorySearch.provider = openai`
- `memorySearch.model = text-embedding-3-small` (1536 dims)
- `memorySearch.sources = ["memory", "sessions"]` (session memory enabled)
- `memorySearch.extraPaths` indexes an additional large Markdown vault (path omitted)
- Hybrid search enabled (BM25 + vector): `vectorWeight=0.7`, `textWeight=0.3`, `maxResults=15`, `minScore=0.4` (candidate multiplier left at default)
- Embedding cache enabled (`maxEntries=100000`)
- sqlite-vec enabled and loads successfully
Scale:
- Indexed Markdown files: 5,198 total
- `memory`: 5,057 files / 165,798 chunks
- `sessions`: 141 files / 934 chunks
- Chunks: 166,732
- SQLite store size: ~8.0 GB
Hardware:
- CPU: AMD EPYC 4244P (6 cores / 12 threads)
- RAM: 30 GiB
- Storage: ~892 GB usable NVMe (RAID1)
- GPU: none for compute (server has only a basic display controller)
Software:
- OS: Linux x86_64
- Node: v22.21.1
- sqlite-vec: 0.1.7-alpha.2
## Performance notes
- Direct SQL benchmark on the above store (top-20): ~75s (brute-force) → ~0.35s (KNN)
- End-to-end CLI (includes embedding API latency): `moltbot memory search --max-results 5 --json "moltbot"` went from >60s to ~3.5s
## Testing
- `pnpm lint`
- `pnpm test`
- `pnpm build`
- Manual spot-check against a real ~166k-chunk store
## AI-assisted disclosure
This PR was AI-assisted (OpenAI Codex CLI).
- Testing level: fully tested (lint + full unit tests + build) + manual runtime check.
- I reviewed/understand the change and am happy to iterate if you’d prefer a different query shape.
<!-- greptile_comment -->
<h2>Greptile Overview</h2>
<h3>Greptile Summary</h3>
This PR speeds up sqlite-vec-backed `memory_search` by switching from a full-table brute-force `ORDER BY vec_distance_cosine(...)` scan to a two-phase approach: use sqlite-vec KNN (`embedding MATCH ? AND k = ?`) to preselect candidates, then compute cosine distance and sort/limit those results. It also adds a focused unit test to assert the KNN query shape is used when the vector extension is available.
<h3>Confidence Score: 3/5</h3>
- Generally safe to merge, but the new KNN query shape may reduce recall when source filters are active.
- The change is localized and test-covered for the SQL shape, but KNN candidate selection currently happens before applying source filters, which can exclude relevant results for the requested sources depending on data distribution.
- src/memory/manager-search.ts
<!-- greptile_other_comments_section -->
<sub>(2/5) Greptile learns from your feedback when you react with thumbs up/down!</sub>
<!-- /greptile_comment -->
Most Similar PRs
#19967: feat(memory): add semantic clustering and enhanced MMR
by alihassan6520 · 2026-02-18
73.6%
#19920: fix(memory): populate FTS index in FTS-only mode so search returns ...
by forketyfork · 2026-02-18
73.4%
#3364: refactor(memory): extract vector management to dedicated module
by tianrking · 2026-01-28
72.5%
#18595: feat: native PostgreSQL + pgvector memory backend
by IrriVisionTechnologies · 2026-02-16
71.6%
#11179: fix(memory): replace confusing "No API key" errors in memory tools ...
by liuxiaopai-ai · 2026-02-07
70.1%
#9624: fix(memory): resolve QMD search returning empty results [AI-assisted]
by kowshik24 · 2026-02-05
70.1%
#15620: fix(memory): await sync in search to prevent database closure race
by superlowburn · 2026-02-13
70.0%
#15339: fix: BM25 score normalization and FTS5 query join operator
by echoVic · 2026-02-13
69.9%
#8706: fix(memory): fall back to better-sqlite3 when node:sqlite lacks FTS5
by ElmerProject · 2026-02-04
69.8%
#8795: feat(memory): add Redis-backed long-term memory plugin
by tf-gmail · 2026-02-04
69.4%