Cloudflare Developers•2w ago

Hello. What would the average queue backlog be, on a healthy queue? Our right now keeps on being 760

Hello. What would the average queue backlog be, on a healthy queue? Our right now keeps on being 760, with a delayed backlog of around 530..

9 Replies

sathoro•2w ago

if you need more concurrency, create your own queue system with Durable Objects so it can be scaled infinitely

Pranshu Maheshwari•2w ago

A quick feature update to share: you can now customize the retention period for a queue https://developers.cloudflare.com/changelog/2025-02-14-customize-queue-retention-period/

Cloudflare Docs

Customize queue message retention periods

achesui•7d ago

Hello, is there a way to bind a queue Event Notification to a local R2 bucket?. For development

Unknown User•6d ago

Message Not Public

scotto•5d ago

setting this in queues consumer

[[queues.consumers]]
queue = "gopersonal-llm-ranking-queue"
max_batch_size = 100
max_batch_timeout = 180
max_concurrency = 1

[[queues.consumers]]
queue = "gopersonal-llm-ranking-queue"
max_batch_size = 100
max_batch_timeout = 180
max_concurrency = 1

and getting the error ✘ [ERROR] A request to the Cloudflare API (/accounts/920b1a6e159cf77dab28969103a4765b/queues/c30e85e6726c455b880498715d8a0b4c/consumers/5bad5c90b2644f498177c45a164e96ac) failed. Queue consumer (type worker) has invalid settings: maximum wait time must be between 0 and 60000 ms. [code: 100127]
If you think this is a bug, please open an issue at: https://github.com/cloudflare/workers-sdk/issues/new/choose if we put the value of max_batch_timeout to 60 it works.., that means the error message is in milliseconds and the actual limit is 60 seconds?

GitHub

Build software better, together

GitHub is where people build software. More than 150 million people use GitHub to discover, fork, and contribute to over 420 million projects.

John Spurlock•5d ago

yep you got it, the max batch wait time is 60 seconds [1], the backend rest api takes milliseconds [2], so wrangler multiplies the config value by 1000 [3], making the error message confusing : ) [1] https://developers.cloudflare.com/queues/platform/limits/ [2] https://developers.cloudflare.com/api/resources/queues/subresources/consumers/methods/create/ [3] https://github.com/cloudflare/workers-sdk/blob/0322d085f634c1a0a12a59b4db293088d0cadb62/packages/wrangler/src/deploy/deploy.ts#L1236

scotto•5d ago

thanks, any plans to increase those limits?

John Spurlock•5d ago

someone orange would have to speak to that - personally I would love to see larger values allowed there and also the batch size 100msg/256kb lifted a bit as well

HowlOftheSun•5d ago

~~What would cause a queue to send the same msg multiple times? Is there a timeout i might have set wrong?~~ Errors cause retries.... 🤦 which cause more errors 🤦 🤦

Gaming

Programming

Hello. What would the average queue backlog be, on a healthy queue? Our right now keeps on being 760

Did you find this page helpful?