#20148: fix(memory): persist session dirty state and fix reindex gate

🤖 AI-assisted PR (Claude/OpenClaw agents). Fully tested, all changes reviewed and understood by human submitter. Fixes two related issues causing session indexing to stop after gateway restart: 1. Move needsFullReindex check before reason gate in shouldSyncSessions() 2. Persist sessionsDirty flag to database metadata and rebuild dirty file set on startup ## Summary - **Problem:** Session memory indexing silently stops after gateway restart — no errors, no warnings - **Why it matters:** Users lose session search permanently until manual DB intervention; no workaround survives restarts - **What changed:** Reordered logic gate in `shouldSyncSessions()`, added `sessionsDirty` persistence to DB meta, added `rebuildSessionsDirtyFiles()` on startup - **What did NOT change:** Indexing logic, chunking, embedding pipeline, config schema — purely a state management fix ## Change Type (select all) - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Docs - [ ] Security hardening - [ ] Chore/infra ## Scope (select all touched areas) - [ ] Gateway / orchestration - [ ] Skills / tool execution - [ ] Auth / tokens - [x] Memory / storage - [ ] Integrations - [ ] API / contracts - [ ] UI / DX - [ ] CI/CD / infra ## Linked Issue/PR - Related #6600 - Related #3558 ## User-visible / Behavior Changes - Session memory indexing now resumes correctly after gateway restart (previously silently stopped) - No config changes, no new options, no migration needed ## Security Impact (required) - New permissions/capabilities? `No` - Secrets/tokens handling changed? `No` - New/changed network calls? `No` - Command/tool execution surface changed? `No` - Data access scope changed? `No` ## Repro + Verification ### Environment - OS: Ubuntu 22.04 - Runtime/container: Node.js v22 - Model/provider: Local embeddings (nomic-embed-text-v1.5 GGUF) - Relevant config: ```json { "memorySearch": { "sources": ["memory", "sessions"], "experimental": { "sessionMemory": true }, "sync": { "intervalMinutes": 5 } } } ``` ### Steps 1. Enable session memory with interval sync (config above) 2. Let interval sync index some sessions — verify with `openclaw memory status` 3. Restart gateway: `openclaw gateway restart` 4. Check `openclaw memory status` over the next interval cycle ### Expected - Session file count continues increasing after restart ### Actual - Session count freezes, shows `Dirty: yes` but no indexing activity ## Evidence - [x] Failing test/log before + passing after Two new tests in `src/memory/index.test.ts`: 1. **Reindex gate order** — calls `shouldSyncSessions({ reason: "session-start" }, true)` and asserts `true`. Before the fix this returned `false`. Also verifies negative case. 2. **Dirty state persistence** — writes meta with `sessionsDirty: true`, closes manager, reopens, asserts dirty state is restored. All 10 tests passing, linter clean. ## Human Verification (required) - Verified: `shouldSyncSessions()` gate order — `needsFullReindex` now checked before `reason` - Verified: `sessionsDirty` added to `MemoryIndexMeta` type - Verified: Persistence write in `runSafeReindex` and restore in constructor - Verified: `rebuildSessionsDirtyFiles()` queries existing `files` table with correct schema - Edge cases: Empty sessions dir (empty Set, no-op), missing meta field (optional, defaults falsy) - **Not verified:** Full end-to-end reindex cycle on live system (unit tests only) ## Compatibility / Migration - Backward compatible? `Yes` - Config/env changes? `No` - Migration needed? `No` ## Failure Recovery (if this breaks) - How to revert: Revert commit, restart gateway - Files to restore: `src/memory/manager-sync-ops.ts`, `src/memory/manager.ts` - Bad symptoms: If `rebuildSessionsDirtyFiles()` fails, logs warning and continues — does not block startup ## Risks and Mitigations - Risk: `rebuildSessionsDirtyFiles()` scans all session files on startup, could be slow with thousands of files - Mitigation: Runs async (`void this.rebuildSessionsDirtyFiles()`), does not block constructor or first sync <!-- greptile_comment --> <h3>Greptile Summary</h3> This PR fixes two related bugs that caused session memory indexing to silently stop after gateway restart: - **Reindex gate order fix** (`manager-sync-ops.ts`): Moves the `needsFullReindex` check before the `reason` gate in `shouldSyncSessions()`. Previously, `reason === "session-start"` or `"watch"` would short-circuit and return `false` even when a full reindex was needed, preventing session indexing from ever completing after a restart. - **Dirty state persistence** (`manager.ts`, `manager-sync-ops.ts`): Adds `sessionsDirty` to the `MemoryIndexMeta` type and persists it to DB metadata during `runSafeReindex`. On startup, the constructor restores this flag and calls `rebuildSessionsDirtyFiles()` to rediscover unindexed session files by comparing the file system against the `files` table. - Two new tests cover both fixes with positive and negative assertions. The changes are well-scoped and backward-compatible. One minor inconsistency: `runUnsafeReindex` (test-only path) does not persist `sessionsDirty` to meta, unlike `runSafeReindex`. <h3>Confidence Score: 4/5</h3> - This PR is safe to merge — it fixes a clear logic ordering bug and adds state persistence with graceful fallbacks. - The gate reorder fix is straightforward and correct. The persistence mechanism is sound — `rebuildSessionsDirtyFiles` is wrapped in try/catch and runs as fire-and-forget, so startup isn't blocked if it fails. Tests cover both changes. Minor deduction for the `runUnsafeReindex` inconsistency (test-only) and the fact that `sessionsDirty` is only persisted during full reindex, not during regular sync cycles (mitigated by the rebuild-on-startup approach). - `src/memory/manager-sync-ops.ts` — `runUnsafeReindex` should be updated for consistency with the new `sessionsDirty` persistence in `runSafeReindex`. _{Last reviewed commit: eca6eca} <!-- greptile_other_comments_section --> _{(3/5) Reply to the agent's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!} **Context used:** - Context from `dashboard` - CLAUDE.md ([source](https://app.greptile.com/review/custom-context?memory=fd949e91-5c3a-4ab5-90a1-cbe184fd6ce8)) <!-- /greptile_comment -->