Live playback of audio coming from discord causes crackling issue

I want to play incoming discord audio directly in the speakers. Here is my code so far:

const connection = joinVoiceChannel({
    channelId: channel.id,
    guildId: guild.id,
    // @ts-ignore
    adapterCreator: guild.voiceAdapterCreator,
    selfDeaf: false,
    selfMute: false,
});

const receivers: string[] = [];

connection.receiver.speaking.on("start", userId => {
    if (receivers.includes(userId)) return;
    receivers.push(userId);

    const opusStream = connection.receiver.subscribe(userId, {
        end: {
            behavior: EndBehaviorType.Manual
        },
    });

    opusStream.on("data", data => {
        const decoded = opusDecoder.decode(data);
        speaker.write(decoded);
    });
});

const connection = joinVoiceChannel({
    channelId: channel.id,
    guildId: guild.id,
    // @ts-ignore
    adapterCreator: guild.voiceAdapterCreator,
    selfDeaf: false,
    selfMute: false,
});

const receivers: string[] = [];

connection.receiver.speaking.on("start", userId => {
    if (receivers.includes(userId)) return;
    receivers.push(userId);

    const opusStream = connection.receiver.subscribe(userId, {
        end: {
            behavior: EndBehaviorType.Manual
        },
    });

    opusStream.on("data", data => {
        const decoded = opusDecoder.decode(data);
        speaker.write(decoded);
    });
});

Where speaker is:

import Speaker from "speaker";
const speaker = new Speaker({
    channels: 2,
    bitDepth: 16,
    sampleRate: 48000,
});

import Speaker from "speaker";
const speaker = new Speaker({
    channels: 2,
    bitDepth: 16,
    sampleRate: 48000,
});

The problem is that by doing this, the audio that plays is periodically interrupted by very tiny audio glitches which i can only describe as "crackling". I think I even know why this is happening - OpusEncoder returns a 3840 length buffer (1 frame) regardless of the size of the incoming opus data. The length of the incoming opus data is variable, around 160 bytes, however sometimes it even sits around 80 bytes around the time that I start/stop speaking. (I logged a few frames, here's the length variation: 160, 165, 180, 177, 165, 164, 152, 156, 164, 163, 161) I tried passing it two frames of opus data expecting it to give back a buffer containing two pcm frames, but instead it squished the data up into one frame and sped it up. Here's my assumption: opus frames longer than 20ms are squished into 20ms of pcm data, and frames that are shorter than 20ms have tiny silences added, which causes the crackling issue. Is there no better way of doing this? If I knew what Opus buffer length the encoder was expecting, I could give it that amount exactly instead of the variable amount coming from djs.

4 Replies

d.js toolkit•2mo ago

- What are your intents? GuildVoiceStates is required to receive voice data! - Show what dependencies you are using -- generateDependencyReport() is exported from @discordjs/voice. - Try looking at common examples: https://github.com/discordjs/voice-examples. - Consider reading #how-to-get-help to improve your question! - Explain what exactly your issue is. - Post the full error stack trace, not just the top part! - Show your code! - Issue solved? Press the button! - ✅ Marked as resolved by OP

AlexejheroOP•2mo ago

Core Dependencies
- @discordjs/voice: 0.17.0
- prism-media: 1.3.5

Opus Libraries
- @discordjs/opus: 0.9.0
- opusscript: not found

Encryption Libraries
- sodium-native: 4.2.1
- sodium: not found
- libsodium-wrappers: not found
- tweetnacl: not found

FFmpeg
- version: N-112807-ge56d91f8a8-20231120
- libopus: yes

Core Dependencies
- @discordjs/voice: 0.17.0
- prism-media: 1.3.5

Opus Libraries
- @discordjs/opus: 0.9.0
- opusscript: not found

Encryption Libraries
- sodium-native: 4.2.1
- sodium: not found
- libsodium-wrappers: not found
- tweetnacl: not found

FFmpeg
- version: N-112807-ge56d91f8a8-20231120
- libopus: yes

AlexejheroOP•2mo ago

Here's also what it sounds like

crackling-audio-samp...

AlexejheroOP•2mo ago

After doing a bit more digging, I've found out that I was wrong - the opus data returned is 20ms even though there are differences in the length, trying to splice frames together to achieve a specific byte length does not make sense and also throws exceptions about corrupted data I don't know what the problem is then Seems like this was indeed a buffer underrun issue - bot not from djs, rather from node-speaker library. It doesn't properly handle live incoming audio like that. I was able to fix it by replacing it with naudiodon2

Gaming

Programming

Live playback of audio coming from discord causes crackling issue