#20867: fix: allow 'video' and 'audio' in models.input config
agents
size: XS
The zod schema for model input modalities only accepted 'text' and 'image', but the codebase already supports video and audio through MediaUnderstandingCapabilitiesSchema and related constants. Trying to configure a model with video or audio input would fail validation.
This adds 'video' and 'audio' as valid values in both the zod schema and the TypeScript type. Also updates the Bedrock discovery module to pass through video/audio modalities from the AWS API instead of dropping them.
Fixes #20721
<!-- greptile_comment -->
<h3>Greptile Summary</h3>
Extended model input modalities to support `video` and `audio` in addition to `text` and `image`. The codebase already supported these modalities through `MediaUnderstandingCapabilitiesSchema` (src/config/zod-schema.core.ts:408-410), but the `ModelDefinitionSchema.input` field was limited to `text` and `image`, causing validation failures when configuring models with video or audio capabilities. This change aligns the model configuration schema with existing media understanding capabilities throughout the codebase.
<h3>Confidence Score: 5/5</h3>
- This PR is safe to merge with minimal risk
- The changes are straightforward type additions that align the model input schema with existing media capabilities. The modifications are consistent across all three affected files (TypeScript type, Zod schema, and Bedrock discovery), with no breaking changes to existing functionality. The code already handles video and audio throughout the media understanding system.
- No files require special attention
<sub>Last reviewed commit: 4906ac0</sub>
<!-- greptile_other_comments_section -->
<!-- /greptile_comment -->
Most Similar PRs
#21499: fix #20721: add video and audio to models.input type union
by neipor · 2026-02-20
90.2%
#20738: Fix model input schema to accept audio and video modalities
by Clawborn · 2026-02-19
89.4%
#20878: fix: Widen models.input to accept "video" and "audio" modalities
by marcodelpin · 2026-02-19
87.8%
#21298: fix(config): extend model input schema for video/audio modalities
by Alfa-ai-ccvs-tech · 2026-02-19
83.3%
#23211: fix: include modelByChannel in allowed channels validator
by westerbamos · 2026-02-22
74.3%
#23155: fix: add modelByChannel to allowed channel config keys
by tiagocampo · 2026-02-22
73.8%
#10943: fix(config): resolve Control UI "Unsupported schema node" for confi...
by kraftbj · 2026-02-07
72.8%
#22998: fix(config): add modelByChannel to allowed channels keys
by bbekdemir · 2026-02-21
72.5%
#14640: feat(agents): support per-agent temperature and maxTokens in agents...
by lailoo · 2026-02-12
71.3%
#16412: fix(config): align tools.web.fetch schema with firecrawl/readabilit...
by ciberponk · 2026-02-14
70.4%