← Back to PRs

#12997: feat(infra): Add query caching layer with TTL and LRU eviction

by trevorgordon981 open 2026-02-10 01:24 View on GitHub →
channel: slack app: web-ui gateway agents stale
## Summary Adds in-memory query cache to reduce redundant API calls. ## Features - TTL-based expiration (1d news, 7d docs) - LRU eviction when cache is full - cachedQuery() wrapper for transparent cache-through - Stats tracking: hit rate, entries by category, evictions ## Expected Impact - 5-10% reduction in repeated queries ## Files - src/infra/query-cache.ts - src/infra/query-cache.test.ts (16 tests) <!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> This PR adds two new in-memory caching utilities (a general `QueryCache` with TTL/eviction and a `SlackChannelCache` for Slack channel listings), plus related tests. It also includes maintenance/memory-leak related changes around agent event sequencing (`seqByRun`) and some gateway/session file tail-reading for large transcripts. Key behavior changes: - `src/infra/query-cache.ts` introduces a global query cache and a `cachedQuery()` wrapper intended to reduce repeated expensive operations. - `src/slack/channel-cache.ts` adds a singleton cache for Slack channel list results keyed by account. - Gateway maintenance now prunes additional in-memory maps and adds safety-net pruning of orphaned agent event sequence state. Before merging, the cache implementation needs a few correctness fixes (stable keying, proper eviction semantics, and size accounting) and Slack cache keying should avoid token-prefix collisions. <h3>Confidence Score: 3/5</h3> - This PR has a few correctness issues in the new caching layers that should be fixed before merging. - The new caches are largely self-contained and tested, but key generation can be unstable, eviction is not actually LRU as described, overwrites can break max size guarantees, and Slack cache keying can collide across tokens leading to incorrect results. - src/infra/query-cache.ts, src/slack/channel-cache.ts <!-- greptile_other_comments_section --> <!-- /greptile_comment -->

Most Similar PRs