discord.js - Imagine an app•3w ago

Issue with @discordjs/ws sharding

I am trying to use worker threads for sharding just like shown in the docs just in common js instead ts https://discordjs.dev/docs/packages/ws/main This is the code:

require("dotenv").config();

const { WebSocketManager, WebSocketShardEvents, WorkerShardingStrategy } = require('@discordjs/ws');
const { REST } = require('@discordjs/rest');
const { Routes } = require('discord-api-types/v10');

const rest = new REST({ version: "10" }).setToken(process.env.TOKEN);
// This example will spawn Discord's recommended shard count, all under the current process.
const manager = new WebSocketManager({
    token: process.env.TOKEN,
    intents: 0, // for no intents
    fetchGatewayInformation() {
        return rest.get(Routes.gatewayBot());
    },
    buildStrategy: (manager) =>
        new WorkerShardingStrategy(manager, {
            shardsPerWorker: 2,
            workerPath: './worker.js',
            // Optionally, if you have custom messaging, like for analytic collection, you can use this:
            async unknownPayloadHandler(data) {
                // handle data here :3
            },
        }),
    // uncomment if you have zlib-sync installed and want to use compression
    // compression: CompressionMethod.ZlibSync,

    // alternatively, we support compression using node's native `node:zlib` module:
    // compression: CompressionMethod.ZlibNative,
});

manager.on(WebSocketShardEvents.Dispatch, (event) => {
    // Process gateway events here.
});

manager.connect().catch();

require("dotenv").config();

const { WebSocketManager, WebSocketShardEvents, WorkerShardingStrategy } = require('@discordjs/ws');
const { REST } = require('@discordjs/rest');
const { Routes } = require('discord-api-types/v10');

const rest = new REST({ version: "10" }).setToken(process.env.TOKEN);
// This example will spawn Discord's recommended shard count, all under the current process.
const manager = new WebSocketManager({
    token: process.env.TOKEN,
    intents: 0, // for no intents
    fetchGatewayInformation() {
        return rest.get(Routes.gatewayBot());
    },
    buildStrategy: (manager) =>
        new WorkerShardingStrategy(manager, {
            shardsPerWorker: 2,
            workerPath: './worker.js',
            // Optionally, if you have custom messaging, like for analytic collection, you can use this:
            async unknownPayloadHandler(data) {
                // handle data here :3
            },
        }),
    // uncomment if you have zlib-sync installed and want to use compression
    // compression: CompressionMethod.ZlibSync,

    // alternatively, we support compression using node's native `node:zlib` module:
    // compression: CompressionMethod.ZlibNative,
});

manager.on(WebSocketShardEvents.Dispatch, (event) => {
    // Process gateway events here.
});

manager.connect().catch();

Why do I get this error?

 const data = await this.options.rest.get(import_v103.Routes.gatewayBot());
                                         ^

TypeError: Cannot read properties of undefined (reading 'get')
    at WebSocketManager.fetchGatewayInformation (C:\Users\Jonah\Github\CoreVoice\node_modules\@discordjs\ws\dist\index.js:1502:42)        
    at WebSocketManager.getShardIds

...

 const data = await this.options.rest.get(import_v103.Routes.gatewayBot());
                                         ^

TypeError: Cannot read properties of undefined (reading 'get')
    at WebSocketManager.fetchGatewayInformation (C:\Users\Jonah\Github\CoreVoice\node_modules\@discordjs\ws\dist\index.js:1502:42)        
    at WebSocketManager.getShardIds

...

52 Replies

d.js toolkit•3w ago

- What's your exact discord.js npm list discord.js and node node -v version? - Not a discord.js issue? Check out #other-js-ts. - Consider reading #how-to-get-help to improve your question! - Explain what exactly your issue is. - Post the full error stack trace, not just the top part! - Show your code! - Issue solved? Press the button! - ✅ Marked as resolved by OP

JonahOP•2w ago

CoreVoice@ C:\Users\Jonah\Github\CoreVoice ├── @discordjs/[email protected] ├── [email protected] ├── [email protected] ├── [email protected] ├── [email protected] ├── [email protected] └── [email protected]

aris•2w ago

from what i can see, they did not pass rest on the constructor try passing rest and retry

JonahOP•2w ago

How to do this in that case? The docs didnt showed it

aris•2w ago

simply add rest after token

require("dotenv").config();

const { WebSocketManager, WebSocketShardEvents, WorkerShardingStrategy } = require('@discordjs/ws');
const { REST } = require('@discordjs/rest');
const { Routes } = require('discord-api-types/v10');

const rest = new REST({ version: "10" }).setToken(process.env.TOKEN);
// This example will spawn Discord's recommended shard count, all under the current process.
const manager = new WebSocketManager({
    token: process.env.TOKEN,
    rest,
    intents: 0, // for no intents
    fetchGatewayInformation() {
        return rest.get(Routes.gatewayBot());
    },
    buildStrategy: (manager) =>
        new WorkerShardingStrategy(manager, {
            shardsPerWorker: 2,
            workerPath: './worker.js',
            // Optionally, if you have custom messaging, like for analytic collection, you can use this:
            async unknownPayloadHandler(data) {
                // handle data here :3
            },
        }),
    // uncomment if you have zlib-sync installed and want to use compression
    // compression: CompressionMethod.ZlibSync,

    // alternatively, we support compression using node's native `node:zlib` module:
    // compression: CompressionMethod.ZlibNative,
});

manager.on(WebSocketShardEvents.Dispatch, (event) => {
    // Process gateway events here.
});

manager.connect().catch();

require("dotenv").config();

const { WebSocketManager, WebSocketShardEvents, WorkerShardingStrategy } = require('@discordjs/ws');
const { REST } = require('@discordjs/rest');
const { Routes } = require('discord-api-types/v10');

const rest = new REST({ version: "10" }).setToken(process.env.TOKEN);
// This example will spawn Discord's recommended shard count, all under the current process.
const manager = new WebSocketManager({
    token: process.env.TOKEN,
    rest,
    intents: 0, // for no intents
    fetchGatewayInformation() {
        return rest.get(Routes.gatewayBot());
    },
    buildStrategy: (manager) =>
        new WorkerShardingStrategy(manager, {
            shardsPerWorker: 2,
            workerPath: './worker.js',
            // Optionally, if you have custom messaging, like for analytic collection, you can use this:
            async unknownPayloadHandler(data) {
                // handle data here :3
            },
        }),
    // uncomment if you have zlib-sync installed and want to use compression
    // compression: CompressionMethod.ZlibSync,

    // alternatively, we support compression using node's native `node:zlib` module:
    // compression: CompressionMethod.ZlibNative,
});

manager.on(WebSocketShardEvents.Dispatch, (event) => {
    // Process gateway events here.
});

manager.connect().catch();

you should have no need for the unknown payload handler so you can safely omit it, the same goes for workerpath

JonahOP•2w ago

node:internal/per_context/domexception:53
    ErrorCaptureStackTrace(this);
    ^
DOMException [DataCloneError]: fetchGatewayInformation() {
                return rest.get(Routes.gatewayBot());
        } could not be cloned.

node:internal/per_context/domexception:53
    ErrorCaptureStackTrace(this);
    ^
DOMException [DataCloneError]: fetchGatewayInformation() {
                return rest.get(Routes.gatewayBot());
        } could not be cloned.

aris•2w ago

im sorry what version of /ws and /rest do you have?

JonahOP•2w ago

aris•2w ago

can you try omitting fetchGatewayInformation() and retry this? /ws /rest is not listed there do you have it seperately installed?

JonahOP•2w ago

Oh I see

aris•2w ago

fetchGatewayInformation is in no stable release

JonahOP•2w ago

I use them from discordjs/core

aris•2w ago

so there is no way you can access it from a stable release so you must have a dev version installed can you omit fetchGatewayInformation() ? and retry running it

JonahOP•2w ago

sec If I omit it there seems to be no error in console

aris•2w ago

that is expected behaviour fetchGatewayInformation is in no stable release of /ws did you omit workerPath as well? unless you have a custom bootstrapper, you should not need it

JonahOP•2w ago

workerPath is still inside alright I will omit it too

aris•2w ago

are you trying to build a microservice with this? or are you trying to use it with discordjs/core?

JonahOP•2w ago

Trying to use it with discordjs/core to implement sharding instead of using a package called discord-hybrid-sharing

aris•2w ago

https://discord.js.org/docs/packages/core/main provides an example on how to use /ws with it

discord.js

discord.js is a powerful Node.js module that allows you to interact with the Discord API very easily. It takes a much more object-oriented approach than most other JS Discord libraries, making your bot's code significantly tidier and easier to comprehend.

JonahOP•2w ago

discord-hybrid-sharing shards via internal shards and worker threads to I thought using /ws would be the more native way

aris•2w ago

you can achieve sharding with workers even with mainlib

JonahOP•2w ago

Where is the example? 😅

aris•2w ago

scroll down to example usage

JonahOP•2w ago

Yeah that in that way I already use it but it doesnt show how to use sharding from /ws Its for a kinda bigger bot in 160k guilds

aris•2w ago

did you pass buildStrategy?

JonahOP•2w ago

No, do I just have to do this?

aris•2w ago

 buildStrategy: (manager) =>
        new WorkerShardingStrategy(manager, {
            shardsPerWorker: 2,
        }),

 buildStrategy: (manager) =>
        new WorkerShardingStrategy(manager, {
            shardsPerWorker: 2,
        }),

passing this should be sufficient yes

JonahOP•2w ago

oh I see

aris•2w ago

or you can cram all shards in one thread by passing "all" to shardsperworker if that is what you want :shruggiesrainbowcat: it is all up to the consumer Ah, yes. this should not be something you need and by general, it is considered bad practice, please excuse me for the unnecessary suggestion :bow: shards per worker being 2 is a sweet spot for that guild count

JonahOP•2w ago

Thats how discord-hyprid-sharding works

const manager = new ClusterManager(`${__dirname}/client.js`, {
  totalShards: "auto",
  shardsPerClusters: 16,
  mode: "process",
  token: process.env.TOKEN,
});

const manager = new ClusterManager(`${__dirname}/client.js`, {
  totalShards: "auto",
  shardsPerClusters: 16,
  mode: "process",
  token: process.env.TOKEN,
});

So I guess that the way i have to go

const gateway = new WebSocketManager({
  rest,
  token: process.env.TOKEN,
  compression: CompressionMethod.ZlibNative,
  buildStrategy: (manager) =>
    new WorkerShardingStrategy(manager, {
      shardsPerWorker: 16,
    }),
  intents:
    GatewayIntentBits.Guilds |
    GatewayIntentBits.GuildMembers |
    GatewayIntentBits.GuildVoiceStates,
  shardCount: // need to get that info somehow before
});

const gateway = new WebSocketManager({
  rest,
  token: process.env.TOKEN,
  compression: CompressionMethod.ZlibNative,
  buildStrategy: (manager) =>
    new WorkerShardingStrategy(manager, {
      shardsPerWorker: 16,
    }),
  intents:
    GatewayIntentBits.Guilds |
    GatewayIntentBits.GuildMembers |
    GatewayIntentBits.GuildVoiceStates,
  shardCount: // need to get that info somehow before
});

aris•2w ago

that is how it would work yes shardCount can be set to null if you want discord to handle it

JonahOP•2w ago

I will try it out

aris•2w ago

how many shards do you have?

JonahOP•2w ago

I am currently rewriting from discord.js to fully @discordjs/core but the shardcount will be the same So I will need around 176 because of large bot sharding I tought that 16 shards per worker would be fine

aris•2w ago

👍 have you acquired a max concurrency of 16?

JonahOP•2w ago

The rewrite has been fun so far. Learned a lot new stuff and noticed how much work the mainlib took over for me Yes

aris•2w ago

:LGJ_clap:

JonahOP•2w ago

I didnt noticed any difference when Discord send me an DM that the bot reached 150k and will now use large sharding

aris•2w ago

the process is automatic

JonahOP•2w ago

yea

aris•2w ago

max concurrency means how many shards can initialize every 5 seconds which in this case, is 16 for you

JonahOP•2w ago

ohhhh Thanks for telling me

aris•2w ago

so your bot should boot up faster now of course :nodnod:

JonahOP•2w ago

Well I didnt messured the time before currently it takes at least 15min to fully start

aris•2w ago

with mainlib?

JonahOP•2w ago

Yes and discord-hybrid-sharding

aris•2w ago

i see how many shards did you have again?

JonahOP•2w ago

176

DD•2w ago

re the docs, pay attention to the version you're looking at main is the default and we just merged a PR that makes it so ws doesn't take a REST instance anymore

JonahOP•2w ago

Another question from my side There wont be no broadCastEval so how could I get infos from every shard? Yeah exacly but currently its still using something like client.guilds = new Map() per shard and not a central location like a redis Ah I assumed its similar to the sharding of the mainlib which inizializes a client per shards and thats by broadcasteval is needed to run code on all shards

DD•2w ago

no, broadcastEval is a pretty bad pattern overall that will hopefully be getting phased out in the future ideally you should find your own way of centralizing data

JonahOP•2w ago

Yeah I am definitely smarter now then before I started the thread. I will try around to see how far I come now with the sharding of ws Do I have to note anything when using this typ of sharding? I am still not really sure how this process works and I am only familiar with the mainlib sharding and the documentation isn’t telling me much 😅 Unfortunately no one has made a guide for it to explain it further

Gaming

Programming

Issue with @discordjs/ws sharding

Did you find this page helpful?