← Back to PRs

#21587: Control UI: show approximate token count for core files in Agents …

by marinusdebeer open 2026-02-20 04:19 View on GitHub →
app: web-ui gateway size: S
# Control UI: show approximate token count for core files in Agents → Files ## Summary - **Problem:** Core Files in Control UI (Agents → Files) showed only size and timestamp; no sense of context size for prompts. - **Why it matters:** Users can see approximate token load per file when judging workspace context. - **What changed:** Backend counts words per file and returns `wordCount`; UI shows "· ~N tokens" (words × 1.3) next to size. Schema and types updated; one new test. - **What did NOT change:** No tokenizer, no change to file read/write behavior or to other UI surfaces. ## Change Type (select all) - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Docs - [ ] Security hardening - [ ] Chore/infra ## Scope (select all touched areas) - [x] Gateway / orchestration - [ ] Skills / tool execution - [ ] Auth / tokens - [ ] Memory / storage - [ ] Integrations - [x] API / contracts - [x] UI / DX - [ ] CI/CD / infra ## Linked Issue/PR - Closes # - Related # ## User-visible / Behavior Changes Core Files list (Agents → Files) now shows approximate token count next to file size and time, e.g. "7.7 KB · ~1610 tokens · 13m ago". Missing files still show "Missing" only. ## Security Impact (required) - New permissions/capabilities? **No** - Secrets/tokens handling changed? **No** - New/changed network calls? **No** - Command/tool execution surface changed? **No** - Data access scope changed? **No** (same files already read for list/content; we only count words in content we already read.) - If any Yes, explain risk + mitigation: N/A ## Repro + Verification ### Environment - OS: Any (e.g. Linux/macOS) - Runtime/container: Node 22+, gateway + Control UI - Model/provider: N/A - Integration/channel: N/A - Relevant config: Default; agent workspace with core files (e.g. AGENTS.md, SOUL.md). ### Steps 1. Open Control UI → Agents → Files. 2. Select an agent with core files present. 3. Check Core Files list. ### Expected Each existing file shows size, "· ~N tokens", and relative time. Missing files show "Missing". ### Actual Matches expected. ## Evidence - [x] Failing test/log before + passing after: New test "returns wordCount for existing files" in `agents-mutate.test.ts`. - [ ] Trace/log snippets - [x] Screenshot/recording: Optional — screenshot of Core Files list with token counts. - [ ] Perf numbers (if relevant) ## Human Verification (required) - **Verified scenarios:** Opened Agents → Files, saw token count next to size for existing files; missing files unchanged. - **Edge cases checked:** Missing files (no token count); existing files with content (wordCount and display correct). - **What you did not verify:** Other agents, very large files, non-English content (heuristic is English-oriented). ## Compatibility / Migration - Backward compatible? **Yes** - Config/env changes? **No** - Migration needed? **No** - If yes, exact upgrade steps: N/A ## Failure Recovery (if this breaks) - **How to disable/revert:** Revert this PR or disable by not using the Control UI Agents → Files view. - **Files/config to restore:** N/A - **Known bad symptoms:** If wordCount is wrong or missing, UI may show "· ~0 tokens" or omit the token part; list still works. ## Risks and Mitigations - **Risk:** Extra file reads for word count when listing files (one read per existing file). - **Mitigation:** Same files are already read for `get`/content; list is small (core files only). Acceptable for this feature. <!-- greptile_comment --> <h3>Greptile Summary</h3> Adds approximate token count display to Control UI's Agents → Files view by implementing word counting on backend and multiplying by 1.3 for token estimation. The implementation is straightforward: backend reads file content to count words using whitespace splitting, UI displays "~N tokens" next to file size. - New `countWords` and `readWordCount` functions handle word counting with proper error handling - Schema updated to include optional `wordCount` field in `AgentsFileEntrySchema` - UI multiplies word count by 1.3 heuristic to approximate tokens - Test coverage added for word counting functionality - One unrelated vite config change included (HMR host configuration) <h3>Confidence Score: 4/5</h3> - This PR is safe to merge with low risk - The implementation is simple and well-tested with proper error handling. The word counting logic correctly handles edge cases (empty files, whitespace). Performance impact is minimal as files are small and already read for display. The only minor issue is an unrelated vite config change that could have been in a separate PR for cleaner history. - No files require special attention <sub>Last reviewed commit: 45bac31</sub> <!-- greptile_other_comments_section --> <sub>(2/5) Greptile learns from your feedback when you react with thumbs up/down!</sub> <!-- /greptile_comment -->

Most Similar PRs