Cache growing too huge too fast for interacting with members in a role
I have a bot that sends money to a specified role at a certain time of the day. I'd like to send money to all the individuals inside a role.
Currently I'm using the following fetches per guild:
This unfortunatly loads all the thousands of users into the bot, making it crash. Is there a more efficient way to get the members of a role? I only need the userIDs of people inside a role without caching all the members on a guild.
Thank you for the help!
9 Replies
- What's your exact discord.js
npm list discord.js
and node node -v
version?
- Not a discord.js issue? Check out #other-js-ts.
- Consider reading #how-to-get-help to improve your question!
- Explain what exactly your issue is.
- Post the full error stack trace, not just the top part!
- Show your code!
- Issue solved? Press the button!No, you are currently doing the most efficient route
Role#members
is just a getter that returns which cached members have a role, so fetching is needed regardless
loads all the thousands of users into the bot, making it crashThat is not normal, though. Make sure you have the
GuildMembers
intent if you don't alreadyI do have the intent, still don't know why the heap size jumps from 100mb to 500mb when the cron for it is executed. It states that its unable to garbage collect.
What is a normal instance size for a bot with 800 servers? is 500mb max RAM too little, should I upgrade my cloud server?
I'm not going to pretend to be an expert about this, but yesterday someone was being berated for only having 1gb ram for a bot that's in more than just one server
500mb is definitely on the low-low side, even compared to most servers' free tiers, so I would absolutely suggest upgrading ASAP
Yeah, but this bot doesn't store any messages, I think the AWS instance is 1GB, but the bot maxes out at 500mb. Doesn't fetching the users of a server store them all into the cache? I see a cache = false option but thats only for single member fetches
I figured out another solution
That makes it only save 30 to cache, so I have to use stack cache members = guild.members.fetch()
to temporarily store 30+
No the cache was growing too big tho
I think
Yeah, that's true
It's supposed to send out the payment in 25 mins, I'll see if it crashes
If so, I'll implement your solution on top
thank you!
Didn't work
I didn't try your solution, my solution was wrong, seems like its one server that has a massive amount of members
I just upgraded the RAM to 2 GB from 1, maybe it will help
But since it's costly I'll implement your solution and try that
I don't have messages intent
Only guild and guild member intents
Gotcha
Wonder why discord makes it so hard just to get members that have a particular role
like I just want a list of IDs, not their entire life history
Can discord.js not optimize this by filtering on the backend?
well by backend I mean as the members come in
Yeah but thats only for the cached members right?
Yeah, makes sense
What's the typical RAM requirement for 1k server bot, or does it vary significantly?
Makes sense
I see, makes sense
Welp, time to ban this feature for servers over 1k people
or implement your solution
that works too
actually it won't
because I'l still have to fetch from that large server
and it's literally crashing the entire bot before it even completes that transaction
I guess that would only be a one-time cost
Woah
Maybe it's not the members then
I have the stack trace
I see
wonder what could be causing it then
unless it's a server with like 100k people
Wait, JsonStringifier?
huh
8k+ (that I know of)
Must be that one, actually
Wish I could only temporarily allocate more ram
Wait, how does the swap work again?
Maybe its the disk size?
it's only 8gb
hmm okay
huh?
Wait
I'm doing members = guild.members.fetch()
So the bot sends money hourly and daily, this only happens when the daily action triggers at 7PM cst
when I restart the bot wouldn't that be empty?
wait nvm
memberCount, yea
lemme try this
running above code
75,157
The stack trace gives no clues
lemme try and AI that
will this be beneficial: https://www.npmjs.com/package/heapdump
npm
heapdump
Make a dump of the V8 heap for later inspection.. Latest version: 0.3.15, last published: 4 years ago. Start using heapdump in your project by running
npm i heapdump
. There are 180 other projects in the npm registry using heapdump.I'm looking
I'm using JSON.parse
I'll paste it here
I deleted the first function because it's not relevant
subtracts and adds money
+ logs
where is what stored?
the code?
or the database type?
I'm using quick.db
Don't think that's the probem tho
It only happens with one server
The bot jumps from 128mb to 540mb while fetching
Thank you for the help anyways, I've doubled the RAM, lets see if it happens again
If it doesn't I'm banning the feature for servers over 1000 and downgrading
I'll try this
I think I know why, they have 70+ roles for many of the users on that server
I increased it RAM up to 2GB, it worked
the bot jumped from 151mb to 1GB when it was fetching that one server, then immediatly back to 162mb
Yes, I do
So if I hadn't had that 30 member thing, it would have been in there forever?
So this is what users not having a lifetime means?
cache lifetimes
read something about that in the wiki
If the type of cache has a lifetime associated with it, such as invites, messages, or threads, then you can set the lifetime option to sweep items older than specified. Otherwise, you can set the filter option for any type of cache, which will select the items to sweep.
![No description](https://cdn.answeroverflow.com/1153844843022405703/image.png)
Hmm, so the cache just simply grows and doesn't replace players who haven't interacted with the bot in a while with newer players?
Gotcha
Gotcha, that makes sense. Thank you so much for your help! I really appreciate it! I like your solution for the database storage of all users with a role, but at this stage of the bot I'm unwilling to do that. I'll simply ban big servers from using this feature until increased revenue justifies a special bot for them or an increase in RAM. Hopefully this will help a lot of people who have a similar issue.
Wish there was a way to only do segemented fetching with the API, but probably never going to happen
Damn wtf, that's an insane idea
The problem is removal, if someone is removed
from a role
what if the bot misses it?
is there documentation on this?
Gotcha, don't know what those are but will read into them 👍