discord.js - Imagine an app•14mo ago

Help with audio stream.

Im trying to make a bot which would capture the incoming audio from a voice channel and play it back in realtime, how can i do this?, i tried doing this but it didnt work

  receiver.speaking.on("start", (userId) => {
    const opusStream = receiver.subscribe(userId, {
      end: {
        behavior: EndBehaviorType.AfterSilence,
        duration: 1000,
      },
    });

    const resource = createAudioResource(
      createReadStream(opusStream, {
        inputType: StreamType.OggOpus,
      })
    );

    const player = createAudioPlayer();
    player.play(resource);
    connection.subscribe(player);
  });

  receiver.speaking.on("start", (userId) => {
    const opusStream = receiver.subscribe(userId, {
      end: {
        behavior: EndBehaviorType.AfterSilence,
        duration: 1000,
      },
    });

    const resource = createAudioResource(
      createReadStream(opusStream, {
        inputType: StreamType.OggOpus,
      })
    );

    const player = createAudioPlayer();
    player.play(resource);
    connection.subscribe(player);
  });

12 Replies

d.js toolkit•14mo ago

- What's your exact discord.js npm list discord.js and node node -v version? - Not a discord.js issue? Check out #other-js-ts. - Consider reading #how-to-get-help to improve your question! - Explain what exactly your issue is. - Post the full error stack trace, not just the top part! - Show your code! - Issue solved? Press the button!

sixOP•14mo ago

nothing happens, and yes i did pass in selfDeaf as false

sixOP•14mo ago

sixOP•14mo ago

i tried to console.log the opusStream variable, and yea its working but its not playing any audio into the vc and the event does trigger when i speak into the vc but thats all that happens

d.js docs•14mo ago

To debug your voice connection and player: - Use debug: true when creating your VoiceConnection and AudioPlayer - Add an event listener to the <VoiceConnection> and the <AudioPlayer>:

// Add one for each class if applicable
<AudioPlayer | VoiceConnection>
    .on('debug', console.log)
    .on('error', console.error)

// Add one for each class if applicable
<AudioPlayer | VoiceConnection>
    .on('debug', console.log)
    .on('error', console.error)

- Add an error listener to the stream you are passing to the resource:

<Stream>.on('error', console.error)

<Stream>.on('error', console.error)

Note: The <> represents classes that need to be adapted to their respective name in your code

sixOP•14mo ago

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string or an instance of Buffer or URL. Received an instance of AudioReceiveStream

im guessing opusStream is a AudioReceiveStream, so to fix the problem how do i get the buffer from the AudioReceiveStream ?

sixOP•14mo ago

here is the full ss of the error

sixOP•14mo ago

i tried using

async function handleCall(callers, globe) {
  const connection = callers[0].connection;
  const receiver = connection.receiver;

  const player = createAudioPlayer();
  connection.subscribe(player);

  receiver.speaking.on("start", (userId) => {
    const opusStream = receiver.subscribe(userId, {
      end: {
        behavior: EndBehaviorType.AfterSilence,
        duration: 1000,
      },
    });

    const resource = createAudioResource(opusStream, {
      inputType: StreamType.OggOpus,
    });


    player.play(resource);
  });
}

async function handleCall(callers, globe) {
  const connection = callers[0].connection;
  const receiver = connection.receiver;

  const player = createAudioPlayer();
  connection.subscribe(player);

  receiver.speaking.on("start", (userId) => {
    const opusStream = receiver.subscribe(userId, {
      end: {
        behavior: EndBehaviorType.AfterSilence,
        duration: 1000,
      },
    });

    const resource = createAudioResource(opusStream, {
      inputType: StreamType.OggOpus,
    });


    player.play(resource);
  });
}

and it still doesnt work. im getting this error from the connection.on("debug")

AudioPlayerError: capture_pattern is not OggS
    at OggDemuxer._readPage (C:\Users\ywxim\OneDrive\Desktop\csc\node_modules\prism-media\src\opus\OggDemuxer.js:61:13)
    at OggDemuxer._transform (C:\Users\ywxim\OneDrive\Desktop\csc\node_modules\prism-media\src\opus\OggDemuxer.js:37:29)
    at Transform._write (node:internal/streams/transform:175:8)
    at writeOrBuffer (node:internal/streams/writable:399:12)
    at _write (node:internal/streams/writable:340:10)
    at Writable.write (node:internal/streams/writable:344:10)
    at AudioReceiveStream.ondata (node:internal/streams/readable:785:22)
    at AudioReceiveStream.emit (node:events:514:28)
    at Readable.read (node:internal/streams/readable:558:10)
    at flow (node:internal/streams/readable:1040:34) {
  resource: AudioResource {
    playStream: OggDemuxer {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 6,
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: true,
      _remainder: null,
      _head: null,
      _bitstream: null,
      [Symbol(kCapture)]: false,
      [Symbol(kCallback)]: null
    },
    edges: [ [Object] ],
    metadata: null,
    volume: undefined,
    encoder: undefined,
    audioPlayer: AudioPlayer {
      _events: [Object: null prototype] {},
      _eventsCount: 0,
      _maxListeners: undefined,
      _state: [Object],
      subscribers: [Array],
      behaviors: [Object],
      debug: [Function (anonymous)],
      [Symbol(kCapture)]: false
    },
    playbackDuration: 0,
    started: false,
    silencePaddingFrames: 5,
    silenceRemaining: -1
  }
} uncaughtException

AudioPlayerError: capture_pattern is not OggS
    at OggDemuxer._readPage (C:\Users\ywxim\OneDrive\Desktop\csc\node_modules\prism-media\src\opus\OggDemuxer.js:61:13)
    at OggDemuxer._transform (C:\Users\ywxim\OneDrive\Desktop\csc\node_modules\prism-media\src\opus\OggDemuxer.js:37:29)
    at Transform._write (node:internal/streams/transform:175:8)
    at writeOrBuffer (node:internal/streams/writable:399:12)
    at _write (node:internal/streams/writable:340:10)
    at Writable.write (node:internal/streams/writable:344:10)
    at AudioReceiveStream.ondata (node:internal/streams/readable:785:22)
    at AudioReceiveStream.emit (node:events:514:28)
    at Readable.read (node:internal/streams/readable:558:10)
    at flow (node:internal/streams/readable:1040:34) {
  resource: AudioResource {
    playStream: OggDemuxer {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 6,
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: true,
      _remainder: null,
      _head: null,
      _bitstream: null,
      [Symbol(kCapture)]: false,
      [Symbol(kCallback)]: null
    },
    edges: [ [Object] ],
    metadata: null,
    volume: undefined,
    encoder: undefined,
    audioPlayer: AudioPlayer {
      _events: [Object: null prototype] {},
      _eventsCount: 0,
      _maxListeners: undefined,
      _state: [Object],
      subscribers: [Array],
      behaviors: [Object],
      debug: [Function (anonymous)],
      [Symbol(kCapture)]: false
    },
    playbackDuration: 0,
    started: false,
    silencePaddingFrames: 5,
    silenceRemaining: -1
  }
} uncaughtException

Update: im trying to make my bot capture the audio coming from a voice channel and then play it back into the same vc in realtime, but i keep getting error AudioPlayerError: capture_pattern is not OggS from the connection.on("debug", console.log); here is my code

  const player = createAudioPlayer();
  connection.subscribe(player);

  connection.on("debug", console.log);

  receiver.speaking.on("start", (userId) => {
    const opusStream = receiver.subscribe(userId, {
      end: {
        behavior: EndBehaviorType.AfterSilence,
        duration: 1000,
      },
    });

    const resource = createAudioResource(opusStream, {
      inputType: StreamType.OggOpus,
    });

    player.play(resource);
  });

  const player = createAudioPlayer();
  connection.subscribe(player);

  connection.on("debug", console.log);

  receiver.speaking.on("start", (userId) => {
    const opusStream = receiver.subscribe(userId, {
      end: {
        behavior: EndBehaviorType.AfterSilence,
        duration: 1000,
      },
    });

    const resource = createAudioResource(opusStream, {
      inputType: StreamType.OggOpus,
    });

    player.play(resource);
  });

Another update. now what im trying to do is to make a bot who will capture audio from one voice channel and play that into another voice channel, its basically cross server call heres the code i have right now

const { createReadStream } = require("node:fs");
const { PassThrough } = require("stream");

const {
  demuxProbe,
  createAudioResource,
  createAudioPlayer,
  EndBehaviorType,

  VoiceReceiver,
  StreamType,
} = require("@discordjs/voice");

async function handleCall(callers, globe) {
  for (const caller of callers) {
    const connection = caller.connection;
    const connectionId = caller.connectionId;
    audioTransfer(callers, connectionId, connection, globe);
  }
}

async function audioTransfer(callers, connectionId, connection, globe) {
  const receiver = connection.receiver;
  const players = [];

  for (const caller of callers) {
    if (caller.connectionId === connectionId) continue;

    const player = createAudioPlayer();
    caller.connection.subscribe(player);

    players.push(player);
  }

  receiver.speaking.on("start", async (userId) => {
    const opusStream = receiver.subscribe(userId, {
      end: {
        behavior: EndBehaviorType.AfterSilence,
        duration: 1000,
      },
    });

    const nonObjectModeStream = new PassThrough();
    opusStream.pipe(nonObjectModeStream);

    for (const player of players) {
      const resource = await probeAndCreateResource(nonObjectModeStream);
      player.play(resource);
    }
  });
}

async function probeAndCreateResource(readableStream) {
  const { stream, type } = await demuxProbe(readableStream);
  console.log(type);
  return createAudioResource(stream, { inputType: type });
}

module.exports = { handleCall };

const { createReadStream } = require("node:fs");
const { PassThrough } = require("stream");

const {
  demuxProbe,
  createAudioResource,
  createAudioPlayer,
  EndBehaviorType,

  VoiceReceiver,
  StreamType,
} = require("@discordjs/voice");

async function handleCall(callers, globe) {
  for (const caller of callers) {
    const connection = caller.connection;
    const connectionId = caller.connectionId;
    audioTransfer(callers, connectionId, connection, globe);
  }
}

async function audioTransfer(callers, connectionId, connection, globe) {
  const receiver = connection.receiver;
  const players = [];

  for (const caller of callers) {
    if (caller.connectionId === connectionId) continue;

    const player = createAudioPlayer();
    caller.connection.subscribe(player);

    players.push(player);
  }

  receiver.speaking.on("start", async (userId) => {
    const opusStream = receiver.subscribe(userId, {
      end: {
        behavior: EndBehaviorType.AfterSilence,
        duration: 1000,
      },
    });

    const nonObjectModeStream = new PassThrough();
    opusStream.pipe(nonObjectModeStream);

    for (const player of players) {
      const resource = await probeAndCreateResource(nonObjectModeStream);
      player.play(resource);
    }
  });
}

async function probeAndCreateResource(readableStream) {
  const { stream, type } = await demuxProbe(readableStream);
  console.log(type);
  return createAudioResource(stream, { inputType: type });
}

module.exports = { handleCall };

the player status changes to "buffering" everytime i use the player.play but it never changes to "playing" and the resource is valid i checked what could be the problem here. also debuggers dont throw anything. here is what i get when i log resource if that may help

AudioResource {
  playStream: OggDemuxer {
    _readableState: ReadableState {
      objectMode: true,
      highWaterMark: 16,
      buffer: BufferList { head: null, tail: null, length:
 0 },
      length: 0,
      pipes: [],
      flowing: false,
      ended: false,
      endEmitted: false,
      reading: false,
      constructed: true,
      sync: false,
      needReadable: true,
      emittedReadable: false,
      readableListening: true,
      resumeScheduled: false,
      errorEmitted: false,
      emitClose: true,
      autoDestroy: true,
      destroyed: false,
      errored: null,
      closed: false,
      closeEmitted: false,
      defaultEncoding: 'utf8',
      awaitDrainWriters: null,
      multiAwaitDrain: false,
      readingMore: false,
      dataEmitted: false,
      decoder: null,
      encoding: null,
      [Symbol(kPaused)]: null
    },
    _events: [Object: null prototype] {
      prefinish: [Function: prefinish],
      close: [Array],
      end: [Array],
      finish: [Array],
      error: [Array],
      unpipe: [Function: onunpipe],
      readable: [Function]
    },
    _eventsCount: 7,
    _maxListeners: undefined,
    _writableState: WritableState {
      objectMode: false,
      highWaterMark: 16384,
      finalCalled: false,
      needDrain: false,
      ending: false,
      ended: false,
      finished: false,
      destroyed: false,
      decodeStrings: true,
      defaultEncoding: 'utf8',
      length: 0,
      writing: false,
      corked: 0,
      sync: true,
      bufferProcessing: false,
      onwrite: [Function: bound onwrite],
      writecb: null,
      writelen: 0,
      afterWriteTickInfo: null,
      buffered: [],
      bufferedIndex: 0,
      allBuffers: true,
      allNoop: true,
      pendingcb: 0,
      constructed: true,
      prefinished: false,
      errorEmitted: false,
      emitClose: true,
      autoDestroy: true,
      errored: null,
      closed: false,
      closeEmitted: false,
      [Symbol(kOnFinished)]: []
    },
    allowHalfOpen: true,
    _remainder: null,
    _head: null,
    _bitstream: null,
    [Symbol(kCapture)]: false,
    [Symbol(kCallback)]: null
  },
  edges: [
    {
      type: 'ffmpeg ogg',
      to: [Node],
      cost: 2,
      transformer: [Function: transformer],
      from: [Node]
    },
    {
      type: 'ogg/opus demuxer',
      to: [Node],
      cost: 1,
      transformer: [Function: transformer],
      from: [Node]
    }
  ],
  metadata: null,
  volume: undefined,
  encoder: undefined,
  audioPlayer: undefined,
  playbackDuration: 0,
  started: false,
  silencePaddingFrames: 5,
  silenceRemaining: -1
}

AudioResource {
  playStream: OggDemuxer {
    _readableState: ReadableState {
      objectMode: true,
      highWaterMark: 16,
      buffer: BufferList { head: null, tail: null, length:
 0 },
      length: 0,
      pipes: [],
      flowing: false,
      ended: false,
      endEmitted: false,
      reading: false,
      constructed: true,
      sync: false,
      needReadable: true,
      emittedReadable: false,
      readableListening: true,
      resumeScheduled: false,
      errorEmitted: false,
      emitClose: true,
      autoDestroy: true,
      destroyed: false,
      errored: null,
      closed: false,
      closeEmitted: false,
      defaultEncoding: 'utf8',
      awaitDrainWriters: null,
      multiAwaitDrain: false,
      readingMore: false,
      dataEmitted: false,
      decoder: null,
      encoding: null,
      [Symbol(kPaused)]: null
    },
    _events: [Object: null prototype] {
      prefinish: [Function: prefinish],
      close: [Array],
      end: [Array],
      finish: [Array],
      error: [Array],
      unpipe: [Function: onunpipe],
      readable: [Function]
    },
    _eventsCount: 7,
    _maxListeners: undefined,
    _writableState: WritableState {
      objectMode: false,
      highWaterMark: 16384,
      finalCalled: false,
      needDrain: false,
      ending: false,
      ended: false,
      finished: false,
      destroyed: false,
      decodeStrings: true,
      defaultEncoding: 'utf8',
      length: 0,
      writing: false,
      corked: 0,
      sync: true,
      bufferProcessing: false,
      onwrite: [Function: bound onwrite],
      writecb: null,
      writelen: 0,
      afterWriteTickInfo: null,
      buffered: [],
      bufferedIndex: 0,
      allBuffers: true,
      allNoop: true,
      pendingcb: 0,
      constructed: true,
      prefinished: false,
      errorEmitted: false,
      emitClose: true,
      autoDestroy: true,
      errored: null,
      closed: false,
      closeEmitted: false,
      [Symbol(kOnFinished)]: []
    },
    allowHalfOpen: true,
    _remainder: null,
    _head: null,
    _bitstream: null,
    [Symbol(kCapture)]: false,
    [Symbol(kCallback)]: null
  },
  edges: [
    {
      type: 'ffmpeg ogg',
      to: [Node],
      cost: 2,
      transformer: [Function: transformer],
      from: [Node]
    },
    {
      type: 'ogg/opus demuxer',
      to: [Node],
      cost: 1,
      transformer: [Function: transformer],
      from: [Node]
    }
  ],
  metadata: null,
  volume: undefined,
  encoder: undefined,
  audioPlayer: undefined,
  playbackDuration: 0,
  started: false,
  silencePaddingFrames: 5,
  silenceRemaining: -1
}

anyone coming into this thread please read from here

duck•14mo ago

I suppose while waiting on a response to https://discord.com/channels/222078108977594368/1167109032687968348/1167183879954767883, here's a side note your latest code seems to be overly complicated to address this error https://discord.com/channels/222078108977594368/1167132377156419584/1167146956716519534, you'd want to use StreamType.Opus for an object mode opus stream it's unnecessary to use demuxProbe as there's nothing to demux it's already an opus stream

sixOP•14mo ago

i decided to just send the whole code because i thought it will be better than just sending in parts as i have been facing this error for long i thought i might have overlooked on something in simple currently the callers array only has 2 instances of a custom class, the class includes the client, connection, and some other variables like connnectionId, etc which the program uses, my main goal is to make something like omegle but for audio and on discord, the bot is simply supposed to listen to the audio in vc1 and then play it in vc2 in realtime (thats why there is a array of players) and as you can see im stuck at the very end of coding this where i just need to take audio from one vc and play it in another in realtime the documentation about the discord/voice is very vague mostly so i was having a hard time understanding what the problem was when it kept saying object mode is not supported or something so i just simply decided to make it non object mode by using PassThrough and demuxProbe im pretty sure it shouldn't cause any issues right..

duck•14mo ago

it kept saying object mode is not supported or something

yes, that is due to specifying the wrong stream type as documented, the Opus type is expected to be in object mode unlike any of the other stream types https://discord.js.org/docs/packages/voice/main/StreamType:Enum#Opus the issue of the player being stuck in the buffering state may have been fixed after you added the GuildVoiceStates intent, as the connection would need to be ready to actually send audio but if the issue persists, the issue may be due to improper mutation from piping through a PassThrough to use demuxProbe

sixOP•14mo ago

nope it still continued after I added the intent i will try that first thing in the morning

Gaming

Programming

Help with audio stream.