VoiceBlender 语音控制平台

VoiceBlender

A Go service that bridges SIP and WebRTC voice calls with multi-party audio mixing, a REST API, and real-time webhooks.

Features

• SIP inbound & outbound -- receive and originate SIP calls with codec negotiation (PCMU, PCMA, G.722, Opus, AMR-WB, AMR-NB), digest auth, session timers (RFC 4028)
• SIP over TLS -- optional TLS transport on a second port alongside UDP, reusable by classic SIP trunks and required by WhatsApp
• Early media -- SIP 183 Session Progress with SDP for pre-answer audio (custom ringback, IVR)
• Hold/unhold -- SIP re-INVITE with sendonly/sendrecv direction
• WebRTC -- browser-based voice via SDP offer/answer with trickle ICE
• WhatsApp Business Calling -- inbound and outbound calls over SIP-TLS + ICE/DTLS-SRTP + Opus
• WebSocket legs -- inbound (HTTP upgrade) and outbound (dial) PCM-over-WebSocket legs with binary or json_base64 framing, configurable sample rate (8/16/24/48 kHz), bidirectional text, and caller-supplied X-/P- headers — designed to also back a future generic Agent API
• MoQ legs (experimental, PoC) -- inbound Media-over-QUIC legs over WebTransport/HTTP/3 with Opus framed one frame per MoQ Object (LOC-style). Tracks mengelbart/moqtransport (IETF draft-11); browser interop with draft-16 clients (moqtail, moq.dev) is not expected to work out of the box. Disabled by default; enable with MOQ_ENABLED=true + MOQ_TLS_CERT_FILE / MOQ_TLS_KEY_FILE
• Multi-party rooms -- mix N participants with mixed-minus-self audio at a configurable sample rate (8 kHz, 16 kHz, or 48 kHz per room; default 16 kHz)
• Room bridging -- join two rooms' mixers (same sample rate) with live-configurable direction (bidirectional, one-way each way, or parked); echo-free via mixed-minus-self
• Audio routing matrix -- per-room role-based routing for asymmetric audio (barge-in / whisper / supervisor monitor). Tag legs with a free-form role and declare a matrix of who-hears-whom by role. Applied atomically at leg-join time so a supervisor cannot momentarily bleed into the customer's audio. See API.md.
• Multi-stream SIP calls -- several m=audio sections in one dialog (RFC 3264), each with its own RTP port, direction, language and mixer room; built for live translation, where the original audio and a translated feed are mixed separately. Follows the SIPREC (RFC 7866) wire profile for interoperability. See API.md.
• SIPREC session recording server (RFC 7865 / RFC 7866) -- accept recording sessions forked from an SBC or PBX: multipart SDP + rs-metadata INVITEs are answered receive-only on every m=audio section, and each section is bound to the participant the metadata names. The received audio is an ordinary set of leg streams, so it can be recorded to file and attached to rooms for live STT/agents. Disabled by default; enable with SIPREC_ENABLED=true (needs SIP_TCP_ENABLED=true). The metadata is checked against the SDP it arrived with, so a client that binds a participant to the wrong a=label -- a document that is otherwise valid and would silently attribute audio to the wrong party -- is reported in warnings on GET /v1/legs/{id}/siprec instead of being recorded as if it were correct. VoiceBlender can also act as the recording client, forking a room's participants to an external recording server one stream each (POST /v1/rooms/{id}/siprec, SIPREC_SRC_ENABLED=true). See API.md.
• WebSocket room access -- join rooms from any client over a WebSocket with base64 PCM frames
• DTMF -- send and receive RFC 4733 telephone-events
• Real-Time Text (RTT) -- ITU-T T.140 over RTP per RFC 4103 with RFC 2198 redundancy;
• Recording -- stereo WAV recording per-leg or per-room, multi-channel per-participant tracks, pause/resume (writes silence to preserve timeline while sensitive data is exchanged), optional S3 or Google Cloud Storage upload
• Playback -- stream WAV/MP3 audio or built-in telephone tones into legs or rooms
• TTS -- text-to-speech into legs or rooms (ElevenLabs, Google Cloud, AWS Polly, Deepgram, Azure), with optional preflight staging: synthesize a speculative reply off the critical path, then commit it for instant playback or discard it
• STT -- real-time speech-to-text with partial transcripts (ElevenLabs, Deepgram, Deepgram Flux, Azure, Speechmatics). Deepgram Flux adds conversational turn detection (stt.turn), including eager end-of-turn signals for speculative generation; Speechmatics reports end-of-turn from server-side silence detection and supports mid-stream finalize
• AI Agent -- attach a conversational AI agent to a leg or room (ElevenLabs, VAPI, Pipecat, Deepgram) with mid-session context injection
• Answering Machine Detection (AMD) -- per-call analysis of outbound call audio to classify the answerer as human, machine, no-speech, or not-sure; optional voicemail beep detection via Goertzel frequency analysis
• Webhooks -- real-time event delivery with HMAC-SHA256 signing and retry; a stable per-event event_id (also sent as X-Event-Id) for receiver-side deduplication; typed event data with CDR-style leg.disconnected (disposition, timing, quality)
• WebSocket event stream (VSI) -- GET /v1/vsi streams all events and accepts commands (mute, hold, DTMF, room management) over a single persistent WebSocket; filter by app_id regex for multi-tenant isolation
• Prometheus metrics -- operational metrics exposed at GET /metrics (active legs/rooms, call durations, disconnect reasons, event-egress drop/delivery counters, Go runtime). See API.md for the full metric reference. Profiling via go tool pprof is available at /debug/pprof/ when built with -tags pprof.

Capabilities

• Inbound — Meta-originated INVITEs are auto-routed to a WhatsApp handler when the From URI host ends in meta.vc and the offer still carries DTLS-SRTP (a=fingerprint). A fronting SIP proxy that terminates encryption into plain RTP keeps the Meta From but takes the classic SIP path; one that re-encrypts with SDES-SRTP is rejected with 488 Not Acceptable Here rather than answered with media nothing can decrypt. On the WhatsApp path the leg comes up in ringing, fires leg.ringing (leg_type: "whatsapp_in"), and waits for POST /v1/legs/{id}/answer. The 200 OK then carries the pre-gathered ICE/DTLS-SRTP answer.
• Outbound — POST /v1/legs {"type":"whatsapp", ...} returns 201 immediately with the leg in ringing. ICE gathering, the digest 401/407 round-trip, and the SDP-answer apply happen asynchronously; outcome is signalled via leg.connected or leg.disconnected.
• Audio — full-duplex Opus at 48 kHz with mixed-minus-self room participation, recording, TTS, STT, agent attachment, speaking detection, playback. The mixer auto-resamples between WhatsApp's 48 kHz and your room's configured rate.
• DTMF — inbound RFC 4733 telephone-events are decoded and emitted as dtmf.received plus the standard cross-leg broadcast.
• Webhooks + WebSocket events — leg.ringing / leg.connected / leg.disconnected / dtmf.received / speaking.started / speaking.stopped all carry leg_type set to whatsapp_in or whatsapp_out so multi-tenant filtering works as it does for SIP and WebRTC legs.