butlr: live-listen now broadcasts BOTH call tracks (inbound + outbound) so the owner hears the full conversation, not just the called party

5ed99e35beec15388de9738026e4f42f27aac02d · 2026-05-26 10:05:23 -0700 · SteveStudio2

Fixes "can't hear anything on live-listen". The bridge only called
broadcastPcm for the inbound track, so the AI (outbound) and on-hold/silent
business produced no audio. Now every decoded 20ms frame is broadcast as it
arrives; the browser's nextPlayTime queue interleaves them in order. Phone
calls are mostly half-duplex so this sounds clean with no per-sample summing,
no clipping, no buffering latency, and zero client change (still mono 8kHz
Int16). Stereo MP3 archive (archiveAppend both tracks) untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files touched

M lib/listen-bridge.js

Diff

commit 5ed99e35beec15388de9738026e4f42f27aac02d
Author: SteveStudio2 <steve@designerwallcoverings.com>
Date:   Tue May 26 10:05:23 2026 -0700

    butlr: live-listen now broadcasts BOTH call tracks (inbound + outbound) so the owner hears the full conversation, not just the called party
    
    Fixes "can't hear anything on live-listen". The bridge only called
    broadcastPcm for the inbound track, so the AI (outbound) and on-hold/silent
    business produced no audio. Now every decoded 20ms frame is broadcast as it
    arrives; the browser's nextPlayTime queue interleaves them in order. Phone
    calls are mostly half-duplex so this sounds clean with no per-sample summing,
    no clipping, no buffering latency, and zero client change (still mono 8kHz
    Int16). Stereo MP3 archive (archiveAppend both tracks) untouched.
    
    Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---
 lib/listen-bridge.js | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/lib/listen-bridge.js b/lib/listen-bridge.js
index c893d79..561155b 100644
--- a/lib/listen-bridge.js
+++ b/lib/listen-bridge.js
@@ -8,8 +8,10 @@
 // SEPARATE frames distinguished by media.track — they are NOT pre-mixed.
 // We archive each track into its own .pcm file and ffmpeg-merge to a
 // STEREO MP3 at call end (inbound=left, outbound=right). Live broadcast
-// to /listen-ws is inbound-only for now (single-channel, matches the
-// browser's existing playback path; no client changes needed).
+// to /listen-ws sends BOTH tracks (each 20ms frame as it arrives) so a
+// browser listener hears the full conversation; the half-duplex nature
+// of a phone call makes arrival-order interleave sound clean (no client
+// change needed — still mono 8kHz Int16 frames).
 //
 // Two WebSocket "lanes" on the same upgrade handler — Twilio Streams on
 // /stream/:call_id, browser listeners on /listen-ws/:call_id.
@@ -241,7 +243,13 @@ function attachListenBridge(httpServer) {
         const mulaw = Buffer.from(msg.media.payload, 'base64');
         const pcm = mulawDecodeBuffer(mulaw);
         archiveAppend(callId, track, pcm);                        // both tracks → separate .pcm files
-        if (track === 'inbound') broadcastPcm(callId, pcm);       // live listen = inbound only (mic)
+        // Live listen broadcasts BOTH tracks (not just inbound) so Steve hears the full
+        // conversation — the AI speaks on outbound, the business on inbound. We send each
+        // 20ms frame as it arrives rather than per-sample summing: a phone call is mostly
+        // half-duplex (one party at a time), so arrival-order interleave in the browser's
+        // nextPlayTime queue reproduces the conversation cleanly with no clipping/buffering
+        // latency, and needs zero client change (it already expects mono 8kHz Int16 frames).
+        broadcastPcm(callId, pcm);
       });
       ws.on('close', () => { finalizeArchive(callId); });
       ws.on('error', () => { finalizeArchive(callId); });

← 4ef812b live page: chime on every new call (two-tone Web Audio beep, · back to Butlr · butlr: add voicemail-IVR language guard to auto-hangup when 40322db →