Worker getting rate limited
I believe my worker is getting rate limited; my frontend client opens up a websocket connection with my worker and sends 2366 messages but my worker only receives 998. Is there some way to increase the limit?
60 Replies
How are you measuring how many messages the Worker receives?
I created a binding with a D1 database that writes every message to a table. Then I log on the client how many requests it makes (2366) and compare that the the # of rows in the database (1944), so I'm missing 372 messages.
Just realized the database initially records 988 records, but eventually stabilizes at 1944.
Can you share a repro?
Sure I can pull out a minimal repro after work today
I have a working version if you wanna see
Oh wait
Let me check one thing
This should be a working example: https://wscount.goalastair.com/
Sort of
The WebSocket itself isn't getting rate-limited anyway
It’s not actually writing to D1 on every request
At the speed it is running D1 was actually borking itself
So if you are sending messages that fast, that might be why
Ah interesting, thanks for trying that out. I wonder if I should be checking D1 write responses and retrying with a backoff or something
If possible, I would also try to batch inserts
As D1 can only handle so many at a time
I'm seeing similar issues try to write to R2 actually, which was what I was originally attempting to use as a backing store
Seems I eventually get more successful writes with D1 though, but both aren't able to capture all 2366 write requests
What I'm doing here is incrementing a counter in a DO, then pushing to D1 once the WebSocket disconnects
You could probably also make it push to D1 every n-seconds, if that is better
Hmm yeah, though I'm not sure I can rely on workers holding all the requested data in memory before a batch write
You could also push based on the number of stored messages? Like pushing every 1k messages, instead of one push per message?
Hmm yeah that might work, so you mean querying the count of rows in D1 to check whether my websocket client should push more messages?
I meant the other way around. The DO takes every message it receives, and pushes it to an array, then it checks if there is 1k messages in the array. If there is, convert the array into a batch query, then push to D1
Ah I see yeah that sounds like a fruitful avenue, I'll check it out, much appreciated @HardlyWorkin' !
I don't know about the underlying application here, but if your client generates messages that can be ignored, it may also help to filter those in the DO, before you ever try pushing them to D1
That should reduce load at least somewhat
Yeah, every message is critical so unfortunately I can't do any filtering in this case
One thing I did try as well was to push each message received by the websocket server worker to a worker queue, but based off your recommendation (and the fact that queues are still in beta status) that DOs are more reliable
That, and iirc each queue runs through a single DO, so using a DO per client might be more scalable
Interesting, this is me still prototyping something, so everything I've tried has been a single client
So I'm seeing the same thing with DOs, if my websocket worker just blindly writes to the DO, it will cap out at 1994 messages written, when I've sent 2366.
It does mention in the docs: "An individual Object has a soft limit of 1,000 requests per second. You can have an unlimited number of individual objects per namespace."
https://developers.cloudflare.com/durable-objects/platform/limits/
Cloudflare Docs
Limits · Cloudflare Durable Objects docs
Durable Objects are only available on the Workers Paid plan. Durable Objects limits are the same as Workers Limits, as well as the following limits …
The soft limit there is where someone tried throwing requests at a Worker, and it could handle ~1k before erroring. WebSocket messages are a lot less overhead, so they should be a tad faster. I was able to get are ~2.6k messages into a DO in 4 ms.
Hmm wonder if there's something wrong with my impl, do you have that code sample I can look at?
Uh... I may have deleted it. Give me a little bit to recreate it...
No worries, much appreciated
Do you want it with the D1 stuff? It shouldn't take too long to build out just the DO counter, but adding D1 may take a bit longer
Oh just the DO stuff, but now that you mention counter, I wonder if it's my payloads that are hitting limits. My payload is 2366 string URLs.
Maybe? The amount of messages the DO can handle depends on the amount of work you are making it do per message. If you are doing heavy parsing on each, you would get substantially lower throughput
Do you see any errors from the browser/in the terminal?
No errors, my frontend client successfully sends each one. Also I'm just storing them once received on the server side, no processing.
Hm... And no errors on the Worker itself either?
Nope
Hm...
Let me see if I can get a minimal repro with the payload going
Here's a repro on my end: https://github.com/helloimalastair/wscount
GitHub
GitHub - helloimalastair/wscount
Contribute to helloimalastair/wscount development by creating an account on GitHub.
Awesome thanks, here's mine: https://github.com/conbrad/cf-limit-repro
the csv there that you click to upload will push 2.6k or so links to the websocket server that just stores them. The
/count endpoint will give you a count of the number of links in the DO. I'm seeing a smaller count than what is sent.GitHub
GitHub - conbrad/cf-limit-repro
Contribute to conbrad/cf-limit-repro development by creating an account on GitHub.
Uh, I see a few of these in the console? Might be why it is erroring?
Oh browser console? I haven't seen those
No, in the Worker
Whatever you are passing into
addLink appears to be too large to be passed over RPC
Oh waitInteresting, I'm not seeing those errors in the worker. Are you looking at the real time logs?
The array is too large
I'm in
wrangler devAh I see
wrangler dev doesn't work for me in my wsl environment, seems like I'm missing some context
I'll see if I can get it running on my macbookYou appear to be hitting the 128 KiB limit for Transactional Storage values
I would probably recommending pushing each URL into its own key, instead of storing it all on one key
I see, by key do you mean the DO id?
No, I mean this part instead of pushing all your links into an array and storing them on
links, try storing each url in links/1, links/2, links/3, etc.Oh I see. Would I be able to retrieve all links at once from the key still?
No, you would have to list them out, then retrieve them one by one. You could also split them into multiple arrays, but that is more finnicky
Ah ok, thanks for identifying that. I'll need some way to partition things I guess. Wonder if this is the same issue that was happening originally when I was queuing the links
That one was just pushing to D1 right? No DOs?
Or...
Originally it was just queues actually, then I tried D1, then I tried DO
I would also recommend handling the WebSocket within the DO, using the WebSocket Hibernation API
Cloudflare Docs
WebSockets · Cloudflare Durable Objects docs
WebSockets are long-lived TCP connections that enable bi-directional, real-time communication between client and server.
You can see my repo as an example if you need
Awesome thanks so much!
Have you been working awhile with cloudflare workers?
this is my first foray so I'm just trying to get my first project off the ground with it
A little while... 😅
Though I'm still learning too
Feel free to come back if you need any more help!
Thanks @HardlyWorkin' , again much appreciated!
Alright I altered the test to instead just write directly to D1, and the same payload results in only 1000 records written. I guess I'm hitting the "Queries per worker invocation" limit described here?
https://developers.cloudflare.com/d1/platform/limits/
Cloudflare Docs
Limits · Cloudflare D1 docs
If you would like to explore other storage solutions for your application, Cloudflare also offers Workers KV, Durable Objects, and R2.
Sounds like it. Should be pretty easy to get around though if you use WebSocket Hibernation, since each message should count as a new "invocation"
Oh interesting, ok I'll give it a shot
So I'm trying to get the websocket hibernation in an DO and I'm not able to get the type of the env in typescript correct.
I'm trying to access a D1 binding to write the messages there.
Add
Environment/Env to the Durable Object Type like so: Thank you again!
Oh wow looks like I got everything after migrating to websocket hibernation! Thanks agains soooo much!
Happy to help!