← Back to PRs

#7278: feat(ollama): optimize local LLM support with auto-discovery and timeouts

by alltomatos open 2026-02-02 16:40 View on GitHub →
docs docker agents
## Summary Optimizes the Ollama provider integration to better support local LLM workflows without requiring manual configuration or API keys. ## Changes - **Auto-discovery**: Automatically detects local Ollama instances running on default ports (`11434`) without requiring `OLLAMA_API_KEY`. - **Performance**: Eliminated double network calls during model discovery. - **Reliability**: Increased discovery timeout from 5s to 10s to accommodate slower local model loading. - **Configuration**: Added support for `OLLAMA_HOST` and `OLLAMA_BASE_URL` environment variables to override defaults. - **UX**: Added silent failure mode for unconfigured instances to reduce console noise. ## Testing - Added unit tests for auto-discovery logic (with and without API keys). - Validated `OLLAMA_HOST` environment variable overrides. - Verified that existing tests pass. ## Checklist - [x] Code matches the existing style. - [x] Tests added/updated. - [x] Documentation updated. <!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> This PR improves the Ollama provider experience by enabling implicit provider auto-discovery against a local Ollama instance (defaulting to `127.0.0.1:11434`), adding env overrides (`OLLAMA_HOST`, `OLLAMA_BASE_URL`), increasing discovery timeouts, and avoiding duplicate discovery calls when building the provider config. It also adds unit tests for the new discovery behavior and updates docs and docker examples to better support local-model workflows. Key interactions: `src/agents/models-config.providers.ts` now probes Ollama during `resolveImplicitProviders()` and conditionally adds the `ollama` provider when either an API key is present or models are discovered; docs and docker config changes aim to make local Ollama use easier out of the box. <h3>Confidence Score: 3/5</h3> - This PR is likely safe to merge, but it has a few correctness/documentation mismatches and a misleading runtime version message that should be fixed first. - Core Ollama changes are straightforward and covered by new tests, but there are a couple of user-facing correctness issues (Node version guard message mismatch, docs contradicting new behavior) and some brittleness/noise in auto-discovery and tests that could lead to confusing behavior or flaky CI. - src/infra/runtime-guard.ts, src/agents/models-config.providers.ts, docs/providers/ollama.md, src/agents/models-config.providers.ollama.test.ts <!-- greptile_other_comments_section --> <sub>(2/5) Greptile learns from your feedback when you react with thumbs up/down!</sub> <!-- /greptile_comment -->

Most Similar PRs