Latency issues

How many Workflows do you have running? We are aware of the some performance issue under heavy load (per account) and we are working on fixes for it
7 Replies
Sam
Sam4d ago
So what happened was I was trying to queue all of my workflows (I had 900,000 to do) And what happened was initially I got response times of 200ms or so then as more were added my response time got slower and slower (up to 40 seconds to queue a workflow instance) The best I could get was 2 requests per second So I moved to cloudflare queues in the end as it wasn’t viable for me to leave this script running (from my local machine sending the workflow start instructions) for 5 days which is what I would have needed to do with being capped at this rps (Each workflow takes about 3 seconds)
Diogo Ferreira
Diogo FerreiraOP4d ago
Thank you for reporting Sam! I are looking into it
Sam
Sam4d ago
When I moved to queues i got the workload done but it did take 10 hours But at least I could queue all the requests up then switch my laptop off So I’m also curious how worker concurrency can scale beyond 250 workers in queues as a side note if you have a connection to queues team also @Diogo Ferreira Workflows is the ideal product for me (for this use case not queues) but I do understand it’s in beta. So I’m open minded to using either.
Diogo Ferreira
Diogo FerreiraOP4d ago
Sorry Sam, I do not get your question
Sam
Sam4d ago
I queued 900,000 requests (to cf queues) last night because of the issues with workflows It took 10 hours to finish even with the worker concurrency maxed out at 250 workers
Diogo Ferreira
Diogo FerreiraOP4d ago
Oh, I see. You can trigger/create multiple workflows from the same worker. This is a way to scale it.
Sam
Sam3d ago
So effectively create multiple workflows that do the same thing then randomly distribute to a pool of workflows @Diogo Ferreira ? Do i need to care about worker concurrency also or is that handled by the workflow itself? Basically what Im asking is if I do what you just said will it scale the worker compute as much as is needed automaticaly (providing I make sure my non cloudflare resources are setup for it e.g. postgres db) and this approach basically means if I have 5 workflows for example i can have 500 requests running at once instead of 100
Want results from more Discord servers?
Add your server