Alexejhero Posts - Answer Overflow

Alexejhero

•Created by Alexejhero on 10/16/2024 in #djs-voice

Live playback of audio coming from discord causes crackling issue

I want to play incoming discord audio directly in the speakers. Here is my code so far:

const connection = joinVoiceChannel({
    channelId: channel.id,
    guildId: guild.id,
    // @ts-ignore
    adapterCreator: guild.voiceAdapterCreator,
    selfDeaf: false,
    selfMute: false,
});

const receivers: string[] = [];

connection.receiver.speaking.on("start", userId => {
    if (receivers.includes(userId)) return;
    receivers.push(userId);

    const opusStream = connection.receiver.subscribe(userId, {
        end: {
            behavior: EndBehaviorType.Manual
        },
    });

    opusStream.on("data", data => {
        const decoded = opusDecoder.decode(data);
        speaker.write(decoded);
    });
});

const connection = joinVoiceChannel({
    channelId: channel.id,
    guildId: guild.id,
    // @ts-ignore
    adapterCreator: guild.voiceAdapterCreator,
    selfDeaf: false,
    selfMute: false,
});

const receivers: string[] = [];

connection.receiver.speaking.on("start", userId => {
    if (receivers.includes(userId)) return;
    receivers.push(userId);

    const opusStream = connection.receiver.subscribe(userId, {
        end: {
            behavior: EndBehaviorType.Manual
        },
    });

    opusStream.on("data", data => {
        const decoded = opusDecoder.decode(data);
        speaker.write(decoded);
    });
});

Where speaker is:

import Speaker from "speaker";
const speaker = new Speaker({
    channels: 2,
    bitDepth: 16,
    sampleRate: 48000,
});

import Speaker from "speaker";
const speaker = new Speaker({
    channels: 2,
    bitDepth: 16,
    sampleRate: 48000,
});

The problem is that by doing this, the audio that plays is periodically interrupted by very tiny audio glitches which i can only describe as "crackling". I think I even know why this is happening - OpusEncoder returns a 3840 length buffer (1 frame) regardless of the size of the incoming opus data. The length of the incoming opus data is variable, around 160 bytes, however sometimes it even sits around 80 bytes around the time that I start/stop speaking. (I logged a few frames, here's the length variation: 160, 165, 180, 177, 165, 164, 152, 156, 164, 163, 161) I tried passing it two frames of opus data expecting it to give back a buffer containing two pcm frames, but instead it squished the data up into one frame and sped it up. Here's my assumption: opus frames longer than 20ms are squished into 20ms of pcm data, and frames that are shorter than 20ms have tiny silences added, which causes the crackling issue. Is there no better way of doing this? If I knew what Opus buffer length the encoder was expecting, I could give it that amount exactly instead of the variable amount coming from djs.

8 replies

Gaming

Programming