#14996: feat(matrix): pass asVoice flag through message tool and add waveform generation
channel: matrix
size: S
Cluster:
Voice Message Enhancements
The message tool's asVoice parameter was accepted by the schema but never passed through the Matrix plugin's action handler chain. This meant org.matrix.msc3245.voice was never set on outgoing audio events, causing Element clients to render voice messages as file attachments instead of inline voice bubbles.
Changes:
- Pass asVoice from action handler → tool-actions → messages → send
- Add generateWaveform() using ffmpeg to extract PCM amplitude peaks
- Include waveform data in org.matrix.msc1767.audio for Element's voice message visualization (progress bar with amplitude wave)
Fixes voice messages sent via the message tool not rendering as native voice bubbles in Element (Desktop, iOS, Android).
(AI Assisted, fully tested, fully understood the change)
<!-- greptile_comment -->
<h2>Greptile Overview</h2>
<h3>Greptile Summary</h3>
This PR threads an `asVoice`/`audioAsVoice` boolean through the Matrix message tool/action chain so outgoing audio uploads can be marked as voice messages (adds `org.matrix.msc3245.voice` and `org.matrix.msc1767.audio`).
It also adds optional waveform generation for voice messages by calling `ffmpeg` to downmix/resample audio and compute 100 peak buckets (0–1024), attaching the waveform to `org.matrix.msc1767.audio.waveform` so Element can render the voice-message waveform UI.
Changes are localized to the Matrix extension send pipeline (`actions.ts` → `tool-actions.ts` → `matrix/actions/messages.ts` → `matrix/send.ts` → `matrix/send/media.ts`) and don’t alter non-media message sending behavior.
<h3>Confidence Score: 4/5</h3>
- This PR is reasonably safe to merge and is scoped to Matrix voice-message metadata and optional waveform extraction.
- I didn’t find any definite runtime/type errors introduced in the changed code after verifying the only suspicious pattern (`encoding: "buffer"`) is already used elsewhere in the repo. The ffmpeg path is best-effort and falls back cleanly when unavailable/failing, limiting blast radius. The main remaining risk is environment-specific ffmpeg behavior (stderr volume, input probing), but that isn’t provably broken from the diff alone.
- extensions/matrix/src/matrix/send/media.ts (ffmpeg waveform generation path)
<sub>Last reviewed commit: 35e597b</sub>
<!-- greptile_other_comments_section -->
<sub>(2/5) Greptile learns from your feedback when you react with thumbs up/down!</sub>
<!-- /greptile_comment -->
Most Similar PRs
#16940: fix(matrix): send voice messages as native voice bubbles
by zerone0x · 2026-02-15
86.0%
#14227: fix(matrix): pass asVoice to audioAsVoice for native voice messages
by tfm-neo-ai · 2026-02-11
85.8%
#10721: fix for matrix media: destructure downloadContent return value in m...
by mklasen · 2026-02-06
73.0%
#19439: fix(tts): pass audioAsVoice flag through tool result pipeline
by brandonwise · 2026-02-17
72.9%
#15394: telegram: treat mp3 as voice-compatible when asVoice=true
by SnugMorg · 2026-02-13
72.6%
#13026: matrix: optionally include media metadata in message.read
by yamoroc · 2026-02-10
72.1%
#21193: fix(tts): send voice messages as Opus bubbles on Telegram
by aris-katkova · 2026-02-19
71.1%
#13389: feat(telegram): support native voice notes with automatic OGG/Opus ...
by leavingme · 2026-02-10
70.7%
#23333: fix(matrix): add accountId routing for multi-account message sending 🤖
by BadTurki · 2026-02-22
70.4%
#10313: fix(matrix): fallback to authenticated media download (Matrix v1.11+)
by iter-next · 2026-02-06
70.1%