← Back to PRs

#17502: feat: normalize skill scanner reason codes and trust messaging

by ArthurzKV open 2026-02-15 20:41 View on GitHub →
docs security agents size: M
## Summary Improve skill trust transparency by standardizing scanner reason codes and surfacing structured reasons/evidence in install and audit flows. ## Why Current messaging is generic (`suspicious pattern(s)`) and hard to verify. This change makes trust decisions explainable and consistent across scanner, install, plugin install, and audit output. ## Focused scope This PR is scoped to one theme: **OpenClaw skill/plugin trust scanner contract + trust messaging**. ## What changed - Standardized scanner rule IDs/severity semantics in `src/security/skill-scanner.ts`. - Improved scanner precision with markdown/manifest-aware rules and scannable extensions. - Added structured reason/evidence messaging in: - `src/agents/skills-install.ts` - `src/plugins/install.ts` - `src/security/audit-extra.async.ts` - Fixed severity filtering regression so critical warnings only include critical findings. - Renamed network rule ID to `suspicious.nonstandard_network` for naming consistency. - Updated tests/docs for the new contract. - CI follow-up in this branch: made the informational `formal-conformance` comment step fork-safe (skip PR comment on forked PRs) so the check no longer fails with GitHub 403. ## Local validation I could not run `pnpm build && pnpm check && pnpm test` in this local environment because `pnpm` is not installed here. What I ran locally instead: - `bun run lint -- src/agents/skills-install.ts src/plugins/install.ts` - `bun run lint -- src/security/skill-scanner.ts src/security/skill-scanner.test.ts` - `bunx vitest run src/security/skill-scanner.test.ts -c vitest.unit.config.ts` - `bunx vitest run -c vitest.e2e.config.ts src/plugins/install.e2e.test.ts src/agents/skills-install.e2e.test.ts` ## AI assistance transparency - AI-assisted: **Yes** (implemented with Codex assistance) - Testing level: **Lightly to moderately tested locally** (targeted lint/unit/e2e on touched areas; full matrix covered by CI) - Prompts/session logs: available in this Codex thread if maintainers want them linked/exported. - I reviewed the final diffs and understand the behavior changes. <!-- greptile_comment --> <h3>Greptile Summary</h3> This PR standardizes skill/plugin scanner rule IDs from flat kebab-case (`dangerous-exec`) to a `namespace.descriptor` format (`suspicious.dangerous_exec`, `malicious.env_harvesting`) and surfaces structured reason codes plus evidence snippets in install and audit warning messages. - **Scanner contract**: Rule IDs now use `suspicious.*` / `malicious.*` namespaces with consistent naming. New `filePattern` field scopes rules to specific file types. - **Expanded scan surface**: Scannable extensions now include `.md`, `.mdx`, `.json`, `.yaml`, `.yml`, `.toml` — enabling new rules for prompt injection detection in markdown and untrusted install sources in manifests. - **New detection rules**: `suspicious.prompt_injection_instructions`, `suspicious.privileged_always`, `suspicious.install_untrusted_source`, `malicious.known_blocked_signature`. - **Structured messaging**: `skills-install.ts`, `plugins/install.ts`, and `audit-extra.async.ts` now include `[reason_codes]` and top evidence snippets in warning messages. Previous review feedback about unfiltered severity in `skills-install.ts` and `plugins/install.ts` has been addressed with `reasonCodesBySeverity`/`topEvidenceBySeverity` helper closures. - **CI fix**: Formal conformance workflow's negative suite step no longer uses `continue-on-error` (properly handles expected failures via if/else), and the PR comment step is now fork-safe. - **Docs**: `docs/cli/security.md` documents the new reason code list (missing `suspicious.nonstandard_network`). <h3>Confidence Score: 4/5</h3> - This PR is safe to merge — it improves scanner precision and messaging with no behavioral regressions. - The changes are well-structured: rule ID renames are mechanical with consistent test updates, new rules use appropriate filePattern scoping, and the severity filtering issue from previous review rounds has been addressed. The only gaps are a missing doc entry and a minor evaluation order inconsistency in source rules, neither of which affect correctness. - docs/cli/security.md (missing `suspicious.nonstandard_network` from documented reason codes) <sub>Last reviewed commit: 40092ef</sub> <!-- greptile_other_comments_section --> <!-- /greptile_comment -->

Most Similar PRs