Local Development · Cloudflare Queues
Hey, is there any timeline when you plan to support running separate producer and consumer Workers bound to the same Queue locally? Usually you use queues to decouple your application therefore it is quite uncomfortable to not have the same level of local development on queues then on other Cloudflare components. And yes I'm already aware that I can trick this by "wiring" the consumer handler into the worker into the handler of the producer worker using the queue handler function out of the other worker and managing the consumer / producer mapping via wrangler.toml. It's just uncomfortable. (https://developers.cloudflare.com/queues/configuration/local-development/#known-issues)).
Cloudflare Docs
Local Development · Cloudflare Queues
Queues support local development workflows using Wrangler, the command-line interface for Workers. Wrangler runs the same version of Queues as Cloudflare runs globally.
4 Replies
As I did not get any feedback on this and also did not find any branch on GitHub with someone working on this one question from my side. Are you accepting contributions on this? I could imagine replacing the current array interface with a backwards compatible queue storage interface where the default implementation is the current mesages array but you could change the storage strategy to a different one.
Initially I was conidering just implementing a filequeue but for local development a sqlite based implementation would be even nicer and simplistic.
I'm currently located close to Lisbon and I'm joining the meetup on Thursday in case someon of the worker queue guys is there and wants to have a chat.
Hi Stefan! I missed your first message. Yeah this one is a big annoyance with queues for sure. Most folks get around it by deploying to a staging environment, but it's not as good as doing everything locally
Contributions are always welcome! none of the queues team is in Lisbon, but you might run into some of the folks who work on other parts of the Workers platform - you should definitely stop by
It's particular annoying as the rest of the services are quite good designed for local development. Also looked into the development effort to extend it and it's not that big.
So will try to talk to some people and otherwise just open a PR. If you have anyone in mind to aline on before starting the implementation feel free to connect.
I started a discussion about the feature / a potential implementation. @Pranshu Maheshwari would be nice if you could point the right people towards this: https://github.com/cloudflare/workers-sdk/discussions/7548
GitHub
Enable Cross-Worker Queue Testing Locally · cloudflare workers-sdk ...
Proposal: SQLite-based Queue Implementation for Local Development Background Currently, wrangler's local queue implementation uses an in-memory array for storing queue messages, which prevents ...