DIA
discord.js - Imagine an app•4mo ago
jmprzz

How to properly play audio/webm;codecs=opus buffers to a voice channel sent via WebSocket?

Trying to broadcast a stream of audio buffers being recorded as audio/webm;codecs=opus using MediaRecorder to a voice channel via a WebSocket, I know that the recording works because i can save the recording to a file and it plays in the browser. Here's my code: 
// join channel if name contains radio
    if (newVC?.name.toLocaleLowerCase().includes("radio")) {
      if (newVC.members.size >= 1) {
        // defining the voice connection
        const connection = joinVoiceChannel({
          channelId: newVC.id,
          guildId: newVC.guild.id,
          adapterCreator: newVC.guild.voiceAdapterCreator,
          selfMute: false,
          debug: true,
        });

        const audioPlayer = createAudioPlayer();
        // websocket receiveing buffer as audio/webm;codecs=opus
        // listener is a just custom subprotocol to broadcast audio to this client.
        const ws = new WebSocket("ws://localhost:3000", "listener"); 


        ws.addEventListener("open", () => {
          console.log("Connection opened");
        });

        connection.subscribe(audioPlayer);

        let stream = new Readable({
          objectMode: true,
          highWaterMark: 1024 * 1024 * 20,
          read() {},
        });

        connection.on(VoiceConnectionStatus.Ready, () => {
          ws.addEventListener("message", (event) => {
            // pushing buffer to the stream  
            stream.push(event.data); 

          });
          // creating resouce with stream
          const audio = createAudioResource(stream); 

          // playing the buffers pushed to the stream
          audioPlayer.play(audio); 

          audioPlayer.on("error", console.error);
          audioPlayer.on("debug", console.log);

          connection.on(VoiceConnectionStatus.Destroyed, async () => {
            stream.destroy();
            ws.close();
          });
        });
      } // endif
    } // endif
// join channel if name contains radio
    if (newVC?.name.toLocaleLowerCase().includes("radio")) {
      if (newVC.members.size >= 1) {
        // defining the voice connection
        const connection = joinVoiceChannel({
          channelId: newVC.id,
          guildId: newVC.guild.id,
          adapterCreator: newVC.guild.voiceAdapterCreator,
          selfMute: false,
          debug: true,
        });

        const audioPlayer = createAudioPlayer();
        // websocket receiveing buffer as audio/webm;codecs=opus
        // listener is a just custom subprotocol to broadcast audio to this client.
        const ws = new WebSocket("ws://localhost:3000", "listener"); 


        ws.addEventListener("open", () => {
          console.log("Connection opened");
        });

        connection.subscribe(audioPlayer);

        let stream = new Readable({
          objectMode: true,
          highWaterMark: 1024 * 1024 * 20,
          read() {},
        });

        connection.on(VoiceConnectionStatus.Ready, () => {
          ws.addEventListener("message", (event) => {
            // pushing buffer to the stream  
            stream.push(event.data); 

          });
          // creating resouce with stream
          const audio = createAudioResource(stream); 

          // playing the buffers pushed to the stream
          audioPlayer.play(audio); 

          audioPlayer.on("error", console.error);
          audioPlayer.on("debug", console.log);

          connection.on(VoiceConnectionStatus.Destroyed, async () => {
            stream.destroy();
            ws.close();
          });
        });
      } // endif
    } // endif
4 Replies
d.js toolkit
d.js toolkit•4mo ago
- What's your exact discord.js npm list discord.js and node node -v version? - Not a discord.js issue? Check out #other-js-ts. - Consider reading #how-to-get-help to improve your question! - Explain what exactly your issue is. - Post the full error stack trace, not just the top part! - Show your code! - Issue solved? Press the button!
duck
duck•4mo ago
I imagine you'll at least want to specify inputType: StreamType.WebmOpus for your resource beyond this if it's still not properly playing audio, please share the logs from debugging your AudioPlayer and VoiceConnection and just to make absolutely sure, might as well generate a dependency report
d.js docs
d.js docs•4mo ago
:guide: Getting Started: Introduction - Debugging Dependencies read more To debug your voice connection and player: - Use debug: true when creating your VoiceConnection and AudioPlayer - Add an event listener to the <VoiceConnection> and the <AudioPlayer>: 
// Add one for each class if applicable
<AudioPlayer | VoiceConnection>
    .on('debug', console.log)
    .on('error', console.error)
// Add one for each class if applicable
<AudioPlayer | VoiceConnection>
    .on('debug', console.log)
    .on('error', console.error)
- Add an error listener to the stream you are passing to the resource: 
<Stream>.on('error', console.error)
<Stream>.on('error', console.error)
Note: The <> represents classes that need to be adapted to their respective name in your code
jmprzz
jmprzzOP•4mo ago
Hi @duck thanks for replying, here's what happened: After I set inputType:StreamType.WebmOpus it gave me the following error which i've encountered before, the audioPlayer debug info is included in this log. 
44 |     let result;
45 |     while (result !== TOO_SHORT) {
46 |       try {
47 |         result = this._readTag(chunk, offset);
48 |       } catch (error) {
49 |         done(error);
     ^
error: Did not find the EBML tag at the start of the stream
      at onwrite (native:1:1)
      at native:1:1
      at _transform (/Users/Main/Desktop/weaver/node_modules/prism-media/src/core/WebmBase.js:49:9)
      at native:1:1
      at writeOrBuffer (native:1:1)
      at native:1:1
      at ondata (native:1:1)
      at emit (native:1:1)
      at addChunk (native:1:1)
      at readableAddChunk (native:1:1)

state change:
from {"status":"buffering","resource":true,"stepTimeout":false}
to {"status":"idle","resource":false,"stepTimeout":false}
44 |     let result;
45 |     while (result !== TOO_SHORT) {
46 |       try {
47 |         result = this._readTag(chunk, offset);
48 |       } catch (error) {
49 |         done(error);
     ^
error: Did not find the EBML tag at the start of the stream
      at onwrite (native:1:1)
      at native:1:1
      at _transform (/Users/Main/Desktop/weaver/node_modules/prism-media/src/core/WebmBase.js:49:9)
      at native:1:1
      at writeOrBuffer (native:1:1)
      at native:1:1
      at ondata (native:1:1)
      at emit (native:1:1)
      at addChunk (native:1:1)
      at readableAddChunk (native:1:1)

state change:
from {"status":"buffering","resource":true,"stepTimeout":false}
to {"status":"idle","resource":false,"stepTimeout":false}
Dependency report 
--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.17.0
- prism-media: 1.3.5

Opus Libraries
- @discordjs/opus: 0.9.0
- opusscript: not found

Encryption Libraries
- sodium-native: not found
- sodium: 3.0.2
- libsodium-wrappers: 0.7.14
- tweetnacl: not found

FFmpeg
- version: 6.0
- libopus: yes
--------------------------------------------------
--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.17.0
- prism-media: 1.3.5

Opus Libraries
- @discordjs/opus: 0.9.0
- opusscript: not found

Encryption Libraries
- sodium-native: not found
- sodium: 3.0.2
- libsodium-wrappers: 0.7.14
- tweetnacl: not found

FFmpeg
- version: 6.0
- libopus: yes
--------------------------------------------------
One more thing, when i save my recording as an audio.webm file and i play it through discord, using createAudioResource("path/to/audio.webm") it plays normally... WebSocket server code Basically it receives the recorded audio from a WS client (called insider) and then forwards it to other WS clients (the listeners) like in the first snippet. 
// websocket server
const server = Bun.serve({
  fetch(req, server) {

    // checking for "listener" in  headers
    const headers = Object.fromEntries(req.headers.entries())
    const isListener = headers["sec-websocket-protocol"] === "listener"

    // upgrade the request to a WebSocket
    // add data to the client so it can be referenced
    if (server.upgrade(req, {
      data: {
        headers,
        listener: isListener,
        id: isListener ? listeners.length + 1 : 0
      }
    })) {
      return; 
    }
    return new Response("Upgrade failed", { status: 500 });
  },

  websocket: {
    // if client is a listener, subscribe it to "listeners" topic, else is the insider client.
    open: (ws) => {
      if (ws.data.listener) {
        listeners.push(ws.data)
        ws.subscribe('listeners')
      } else {
        insider = ws.data
      }
      updateConsole()
    },
    close: (ws, code, reason) => {
      // a listener disconnected
      if (ws.isSubscribed("listeners") || ws.data.listener) {
        ws.unsubscribe('listeners')
        listeners = listeners.filter(l => l.id !== ws.data.id)
      }
      else {
       // terminate script
        end("Insider disconnected")
      }
    },
    async message(ws, data) {
      try {
        // if this option is enabled, write recording to a file
        if (writer) { 
          writer.write(data)
          writer.flush()
        }

        if (listeners.length > 0) {
          // broadcast recording
          server.publish("listeners", data)
        }
        
        // for debugging
        buffer = data
        totalBytes += data.length

        updateConsole()
      }
      catch (e) {
        console.error(e)
        end("Error in server message")
      }
    }
  }
});
// websocket server
const server = Bun.serve({
  fetch(req, server) {

    // checking for "listener" in  headers
    const headers = Object.fromEntries(req.headers.entries())
    const isListener = headers["sec-websocket-protocol"] === "listener"

    // upgrade the request to a WebSocket
    // add data to the client so it can be referenced
    if (server.upgrade(req, {
      data: {
        headers,
        listener: isListener,
        id: isListener ? listeners.length + 1 : 0
      }
    })) {
      return; 
    }
    return new Response("Upgrade failed", { status: 500 });
  },

  websocket: {
    // if client is a listener, subscribe it to "listeners" topic, else is the insider client.
    open: (ws) => {
      if (ws.data.listener) {
        listeners.push(ws.data)
        ws.subscribe('listeners')
      } else {
        insider = ws.data
      }
      updateConsole()
    },
    close: (ws, code, reason) => {
      // a listener disconnected
      if (ws.isSubscribed("listeners") || ws.data.listener) {
        ws.unsubscribe('listeners')
        listeners = listeners.filter(l => l.id !== ws.data.id)
      }
      else {
       // terminate script
        end("Insider disconnected")
      }
    },
    async message(ws, data) {
      try {
        // if this option is enabled, write recording to a file
        if (writer) { 
          writer.write(data)
          writer.flush()
        }

        if (listeners.length > 0) {
          // broadcast recording
          server.publish("listeners", data)
        }
        
        // for debugging
        buffer = data
        totalBytes += data.length

        updateConsole()
      }
      catch (e) {
        console.error(e)
        end("Error in server message")
      }
    }
  }
});
im pushing data everytime the socket receives from server, but the read() is required, however i don't see what code to put there since im directly calling push() on ws.addEventListener("message") How can i fix this particular error? 
error: Did not find the EBML tag at the start of the stream
      at onwrite (native:1:1)
      at native:1:1
      at _transform (/Users/Main/Desktop/weaver/node_modules/prism-media/src/core/WebmBase.js:49:9)
      at native:1:1
      at writeOrBuffer (native:1:1)
      at native:1:1
      at ondata (native:1:1)
      at emit (native:1:1)
      at addChunk (native:1:1)
      at readableAddChunk (native:1:1)
error: Did not find the EBML tag at the start of the stream
      at onwrite (native:1:1)
      at native:1:1
      at _transform (/Users/Main/Desktop/weaver/node_modules/prism-media/src/core/WebmBase.js:49:9)
      at native:1:1
      at writeOrBuffer (native:1:1)
      at native:1:1
      at ondata (native:1:1)
      at emit (native:1:1)
      at addChunk (native:1:1)
      at readableAddChunk (native:1:1)
Want results from more Discord servers?
Add your server