← Back to PRs

#7044: feat: Add local model tool calling support

by jokelord open 2026-02-02 10:14 View on GitHub →
extensions: memory-lancedb agents
## Summary This PR adds support for local inference models (sglang/vLLM) to use tool calling by allowing them to declare supported API parameters. ## Changes - Added `supportedParameters` field to model compatibility configuration - Added `modelSupportsTools()` function for tool capability detection - Added tool support check in agent runner before tool creation - Updated schema and types to support local model parameter declarations ## Testing - Verified with Qwen3-Coder-30B-A3B-Instruct-FP8 model - Tested with sglang server using `--tool-call-parser qwen3_coder` - Confirmed tools are properly created and executed ## Usage Local models can now declare tool support in configuration: ```json { "compat": { "supportedParameters": ["tools", "tool_choice"] } } <!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> This PR introduces a `supportedParameters` declaration on model compat config and adds a `modelSupportsTools()` helper used by the embedded runner to skip tool creation for models that don’t advertise `tools` support. While the core idea is clear, the diff also includes several changes that look accidental/unrelated to the stated goal (a non-TS `src/agents/attempt.test.ts` file, major schema strictness changes for model definitions, and removal of the `edge` TTS provider/schema). Additionally, the runner now contains multiple `/tmp` debug file writes which add side effects and can fail/unhandled-reject in some environments. <h3>Confidence Score: 1/5</h3> - Not safe to merge as-is due to likely accidental breaking changes and a malformed test file. - The PR contains a non-TypeScript test file that will fail CI, plus schema changes that make previously-optional config required and remove the `edge` TTS provider despite existing references. The tool-support gating itself looks plausible, but the unrelated breakages and debug side effects significantly increase merge risk. - src/agents/attempt.test.ts, src/config/zod-schema.core.ts, src/agents/pi-embedded-runner/run/attempt.ts <!-- greptile_other_comments_section --> <sub>(2/5) Greptile learns from your feedback when you react with thumbs up/down!</sub> <!-- /greptile_comment -->

Most Similar PRs