Hi, what kind of consumer delays are
Hi, what kind of consumer delays are normal with queues? I have a simple low volume case where a durable object puts individual messages to a queue, and then a worker picks up and processes those messages.
I want to process the messages one by one without any delay, so I have configured delay as 0 and batch size as 1. The actual execution in my worker is very fast, 100-300 ms as expected. But there seems to be some strange delay before the worker picks up the message from queue, and when I look for the queue metrics in console it says "Consumer Delay: 3.4 sec". So I feel I am loosing an extra 3 seconds somewhere, which means this is not OK for any customer facing online use case.
Don't have much experience with queues so I don't know if this normal or not, but I was expecting the added latency to be in tens or hundreds of milliseconds.
10 Replies
I cannot speak for how workers have implemented the queue consumer side. But a delay like that on a low-throughput queue is expected. Usually a queue is polled for messages. The longer the there are no new messages the slower the polling is usually. So it takes some time to ramp up the "delay" of picking up new queue items.
I am wondering if you need realtime processing why you are using a queue in the first place. Why not do the processing immediately or using
ctx.waitUntil to do it after you serve a response? This way there is never a delay. But maybe I am misunderstanding your use case.
In my personal experience. Queues are usually used for slow operations that are OK to take a short amount of time to be executed. So for asynchronous tasks. You don't sound like you are working on a use case.Thanks for the input! I have not seen any such mentions in the Queues documentation (regarding the slow operations), and was kind of expecting/hoping this to work roughly the same way as AWS SQS (to allow async/batch operations, but still execute more or less immediately).
I don't need strictly real time processing (but "as fast as possible still"). I don't want to block a single threaded Durable Object, thats why I was thinking to use a queue to offload some slow (and slow in this context means 1 sec) calls to be performed by a worker in a reliable manner which would then call back to Durable Object once its done.
I don't know how that
ctx.waitUntil would work in the context of a durable object?For my sanity; you configured
max_batch_timeout to be 0 right? THat's what you meant with the delay you configured as 0?
Or are you doing delaySeconds: 0 when queueing the message? Or both?
IIRC context.waitUntill in a DO does not block the object's ability to handle a new request. Just the same as for a worker. But I also can't find explicit documentation at this moment confirming or denying this. ChatGPT seems to agree with me but obviously don't take that as gospell 😛
Curiously I have a similar consumer delay on a very low throughput with a wait time of 1shere are the settings in console:
I do have another queue for a different use case, that one also with similar traffic pattern, and I see the same 3-4 sec consumer delay there.
Regarding
context.waitUntill I was exploring something like that, but Cursor thought I might be in trouble if the durable object decides to hibernate with any open promises. I guess its not very likely it would hibernate just seconds after handling a request though..No it explicitely never should hibernate when using waitUntil, it could to that if you simply not await a promise.
But the delay is interesting, maybe someone with more insider knowledge can comment on this because I also can't find any reference to this and I think the delay should be shorter in theory but it also makes sense in my head why it isn;t but would be nice if that would be documented if there is indeed a case where the delay is coming from the CF side.
Here is what cursor / claude 3.7 said about the use of waitUntil in a durable object... 🤷
AI's are not agreeing with each other 😛 But it would make sense for both answers which is annoying
This would indicate it's basically a no-op which also makes sense but would also mean that Claude might be closer to the truth: https://developers.cloudflare.com/durable-objects/api/state/#waituntil
Unknown User•6d ago
Message Not Public
Sign In & Join Server To View
Ok, thanks a lot for the info! Then I know to hold back with any response time sensitive use cases with Queues for now. Hope you can squeeze it way below 1 sec, it will definitely open a lot of new use cases for the queues!
And one more thing, it would be super nice to mention something about this in the documentation. Especially since with
wrangler dev it pretty much worked without any delay, so people who are new to the product (like me) easily waste a bit of time before discovering these limitations.