makaku6232
CDCloudflare Developers
•Created by makaku6232 on 3/4/2024 in #workers-help
Performance: Sequential queue processing instead of parallel (horizontal scaling)
The average wall time of my worker is 10 seconds.
So i would expect the total processing time to be significantly lower then the amount of messages x 10.
9 replies
CDCloudflare Developers
•Created by makaku6232 on 3/4/2024 in #workers-help
Performance: Sequential queue processing instead of parallel (horizontal scaling)
The stats do show some concurrency. But im updating my DB and streaming the status to my webapp. And the really seem to be updated one at a time
9 replies
CDCloudflare Developers
•Created by makaku6232 on 3/4/2024 in #workers-help
Performance: Sequential queue processing instead of parallel (horizontal scaling)
I am also confused as to how batches effect scaling.
My messages can be processed separately.
But i see no difference between batches of 10 messages and sending single messages
9 replies
CDCloudflare Developers
•Created by makaku6232 on 3/4/2024 in #workers-help
Performance: Sequential queue processing instead of parallel (horizontal scaling)
I saw that limit in the docs. But as I mentioned the messages are processed 1 at a time.
9 replies
CDCloudflare Developers
•Created by makaku6232 on 3/4/2024 in #workers-help
Performance: Sequential queue processing instead of parallel (horizontal scaling)
Sure.. my brain is a bit fried. But I'll give it a go.
To simplify it, let's say a have 2 workers: A and B.
1. Im sending a API request to worker A
2. Worker A processes the request preprocesses the data and splits it into 200 objects.
3. Worker A (producer) sends the objects to the queue, asynchronously using Promise.All
4. Worker B (consumer) processes the messages.
My expectation would be that worker B would automatically scale to process the messages.
I feel like the queue is blocking somehow and resulting in the messages being processed sequentially.
9 replies