discord.js - Imagine an app•2y ago

D.js bot using 8gb ram

A few months ago I'm rewriting my old bot in python for discord.js I made the sharding system according to the guide, always using all functions to search for data between shards. I tried to migrate the bot today and my server crashed because the bot exceeded 8gb of memory. I believe that a bot with 48 shards, 50.000 guilds should not be using so much memory. If you have any ideas where I went wrong? please let me know

25 Replies

d.js toolkit•2y ago

• What's your exact discord.js npm list discord.js and node node -v version? • Post the full error stack trace, not just the top part! • Show your code! • Explain what exactly your issue is. • Not a discord.js issue? Check out #useful-servers.

Gui•2y ago

Using discord.js@14.8.0 and node v16.17.1

Gui•2y ago

monbrey•2y ago

By default discord.js utilises quite a lot of cache, but you have the ability to control this

d.js docs•2y ago

Miscellaneous: Cache customization read more

Gui•2y ago

Can you tell me what exactly I should limit? Messages, guilds? What is making each shard to use 500mb, where can see that?

monbrey•2y ago

monbrey•2y ago

Not Guilds. Messages yes, members, users, presences Node has memory profiling tools

Gui•2y ago

I limited several managners and still got a bad result (7.5g), any idea why?

const client = new Client({ 
    makeCache: Options.cacheWithLimits({
        ReactionManager: 250,
        UserManager: 250,
        GuildMemberManager: 250,
        MessageManager: 250,
        MessageContent: 250,
        GuildEmojiManager: 250,
        ThreadManager: 0,
        PresenceManager: 0,
        VoiceStateManager: 0,
        AutoModerationRuleManager: 0,
        BaseGuildEmojiManager: 0,
        GuildBanManager: 0,
        GuildInviteManager: 0,
        GuildTextThreadManager: 0,
        ReactionUserManager: 0
    }),
    intents: [GatewayIntentBits.Guilds, GatewayIntentBits.GuildMessages, GatewayIntentBits.MessageContent],
    allowedMentions: { parse: ['users', 'roles'] }
});

const client = new Client({ 
    makeCache: Options.cacheWithLimits({
        ReactionManager: 250,
        UserManager: 250,
        GuildMemberManager: 250,
        MessageManager: 250,
        MessageContent: 250,
        GuildEmojiManager: 250,
        ThreadManager: 0,
        PresenceManager: 0,
        VoiceStateManager: 0,
        AutoModerationRuleManager: 0,
        BaseGuildEmojiManager: 0,
        GuildBanManager: 0,
        GuildInviteManager: 0,
        GuildTextThreadManager: 0,
        ReactionUserManager: 0
    }),
    intents: [GatewayIntentBits.Guilds, GatewayIntentBits.GuildMessages, GatewayIntentBits.MessageContent],
    allowedMentions: { parse: ['users', 'roles'] }
});

monbrey•2y ago

No, I don't really deal with any large bots. I recommend doing the memory profiling GuildMemberManager would be 250 per guild, that could be it Reactions are per message

Gui•2y ago

Changed all 250 to 50 and get same result Tomorrow will try make memory profiling I had some free time to development the bot again, I didn't get a solution Today I tried to apply a heavier cache and I didn't get good results Seems to have no effect, to be honest New cache policy:

const client = new Client({ 
    intents: [GatewayIntentBits.Guilds, GatewayIntentBits.GuildMessages, GatewayIntentBits.MessageContent],
    allowedMentions: { parse: ['users', 'roles'] },
    messageEditHistoryMaxSize: 0,
    messageCacheMaxSize: 0,
    messageCacheLifetime: 21600,
    messageSweepInterval: 21600,
    makeCache: Options.cacheWithLimits({
        ReactionManager: 25,
        UserManager: 25,
        GuildMemberManager: 25,
        MessageManager: 25,
        MessageContent: 25,
        GuildEmojiManager: 25,
        ThreadManager: 0,
        PresenceManager: 0,
        VoiceStateManager: 0,
        AutoModerationRuleManager: 0,
        BaseGuildEmojiManager: 0,
        GuildBanManager: 0,
        GuildInviteManager: 0,
        GuildTextThreadManager: 0,
        ReactionUserManager: 0
    }),
});

const client = new Client({ 
    intents: [GatewayIntentBits.Guilds, GatewayIntentBits.GuildMessages, GatewayIntentBits.MessageContent],
    allowedMentions: { parse: ['users', 'roles'] },
    messageEditHistoryMaxSize: 0,
    messageCacheMaxSize: 0,
    messageCacheLifetime: 21600,
    messageSweepInterval: 21600,
    makeCache: Options.cacheWithLimits({
        ReactionManager: 25,
        UserManager: 25,
        GuildMemberManager: 25,
        MessageManager: 25,
        MessageContent: 25,
        GuildEmojiManager: 25,
        ThreadManager: 0,
        PresenceManager: 0,
        VoiceStateManager: 0,
        AutoModerationRuleManager: 0,
        BaseGuildEmojiManager: 0,
        GuildBanManager: 0,
        GuildInviteManager: 0,
        GuildTextThreadManager: 0,
        ReactionUserManager: 0
    }),
});

I wonder if I'm putting the configuration in the right place, this is the file that creates the client If anyone has any ideas beyond the cache, it would be most welcome.

probablyraging•2y ago

That looks fine, probably stricter than needed. If that isn't solving the issue with RAM, then it might be coming from somewhere else in your code? Are you storing a lot of data in variables, storing duplicate data, or repeating operations frequently maybe? I would start dissecting your code to trouble shoot Then again, I have a bot running on a single server using 150MB memory on average. So 8GB for a bot your size might not be too unrealistic

Gui•2y ago

I may be confusing things in discord.py, I'm migrating from there due to the end of discord.py In d.py with 50k servers and with heavy usage I get 4.4gb of ram I'm scared to get double in discord.js (50k with idle bot (no players uses), aprox 7gb and goes up quickly) Do you recommend any specific tools to help me with this? there doesn't seem to be anything wrong with the code i tryed heapdump, but it seems obsolete and not work

probablyraging•2y ago

Doubling the memory usage does seem excessive for sure. But hard to say as there are so many differences in the way the libraries are implemented and the programming languages they are written in. I've not really used heapdump so I can't give advice there. You could try the --inspect flag in node.js? A good way to test if it is a memory leak would be to temporaily increase your server's memory capacity substantially, maybe 32gb and see if the process still has issues. If you see it flatten at around 8-12gb or something then you can sort of rule out a memory leak But yeah, without doing a proper deep dive into your entire project, would almost be impossible to diagnose it

Gui•2y ago

i have results after run with inspect, I left it running for a few moments, with no user interaction, and saw usage go up from 6.8gb to 7.2gb in around 20 minutes (test locally in my computer)

Gaming

Programming

D.js bot using 8gb ram