Hello - I have a Queue with polling client running behind firewall. What is appropriate interval tim
Hello - I have a Queue with polling client running behind firewall. What is appropriate interval time to poll?
I also want to understand cost for this continuous polling.
15 Replies
Hello! I would be interested in FIFO ordering and also interested to know if you are going to support something like AWS SQS Message Group ID?
I tried to send an email to [email protected] as specified in the docs (https://developers.cloudflare.com/queues/configuration/javascript-apis/#messagebatch) but that bounced.
Thanks!
Hello, is there an issue with queues at the moment? I've got messages being reported as being in my deadletter, however I can't see them when listing messages and nothing else is reading from the queue 🤔
Is it possible to add multiple messages to a queue from the dashboard? Seems like no matter what JSON I put in there (even trying to replicate a message batch) it doesn't work
Small Question about Concurrency - if i publish a message with the same content twice, will only one be processed? Example: i have a worker that publishes an event to tell the queue to update an object in an database, if this step is triggered twice, and two queue workers will be invoke, this would possible create a race condition
Yes, each message is a message so both will be processed possibly concurrently if you have a busy queue. You can disable concurrency though if you need to process them more controlled. See: https://developers.cloudflare.com/queues/configuration/consumer-concurrency/
thx, yeah i have concurrency disabled currently, just hoped i overlooked something. we have a really busy queue and without concurrency it takes a while. i might add a second queue, for events where race conditions are a non-issue
Yeah either remove it being a problem that it runs concurrently (not sure about your use case obv. but maybe possible) or seperate the queues. You can even add more queues and "shard" them so they are still not concurrent but you have multiple queues processing. There are ways around it 😄
we handle cash register transactions async so if anything is triggered twice we really have a problem :KEKW:
You can even pipe your concurrency challenged jobs through a durable object which can work as a gate to only allow a single operation to run.
thats, actually a really good idea
CF queues also aren't guaranteed to deliver the message once. They are at least once so I hope having the same message multiple times is not an issue actually 😅
in germany we have to securley record every transation and transactional update, there are exactly two cloud providers for this, and one doesnt provide the other features we need... they do in fact have idempotency, but not really because they need an revision id
yeah but as long as only one consumer runs this is a non issue, because the worker filters out requests that have already run
(at least thats what i've coded lol)
That makes sense! In that case a DO is probably a very easy concurrency limiter which should allow your queue to process a lot faster and only block a little in case a concurrent request arrives. Little extra cost but should not be too bad.
it's just a bit annoying, cause our tenant isolation means we have an worker, d1, queue, etc for every tenant
but definitly doable
btw: our third party api provider has responsetimes of around 500ms per request, and we need 3-4 requests per transaction, its a pain
Fun times, you might be one of the few who will run into the script limits then 😛