workers-help
pages-help
general-help
durable-objects
workers-and-pages-discussions
pages-discussions
wrangler
coding-help
kv
🦀rust-on-workers
miniflare
stream
general-discussions
functions
zaraz
âš¡instant-logs
email-routing
r2
pubsub-beta
analytics-engine
d1-database
queues
workers-for-platforms
workerd-runtime
web3
🤖turnstile
radar
web-research
logs-engine
cloudflare-go
terraform-provider-cloudflare
workers-ai
browser-rendering-api
logs-and-analytics
next-on-pages
cloudflare-ai
build-caching-beta
hyperdrive
vectorize
ai-gateway
python-workers-beta
vitest-integration-beta
workers-observability
workflows
vite-plugin
Hi again! Is it possible to test pull
Hi again! Is it possible to test pull consumers locally? I.e. fetch() from some wrangler local dev host rather than api.cloudflare.com/.../queues/../messages/pull. Big thanks!
is there any kind of queue bindings-per-
is there any kind of queue bindings-per-worker limit? (sorry if I missed it anywhere in the docs). curious whether it's possible/sensible to have a queue-per-customer architecture all written to by the same producer worker. thanks!
I'm a big fan of CF Queues and
I'm a big fan of CF Queues and especially the Consumer Worker model! The biggest thing I wish for is easier way to publish items to a Queue from external systems (like external Kafka Topics).
For example I'm using Debezium for change data capture on an external Database.
This then leads me to having to use Kafka -> HTTP -> Worker -> Queue to get these messages to a CF Queue which is a lot of overhead and you have to deal with all of the usual Webhook headaches...
Appointments
Hi @everyone! Pranshu here. I’m a product manager at Cloudflare and I’ve recently taken on Queues. I’d love to learn more about how all of you are using Queues, and how you'd like to see it improve.
I’d especially love to hear from folks building startups. I’ve been a 2x founder myself, so I’ve been in your shoes!
If you’d like to grab some time to chat in the next two weeks, here’s my calendar: https://calendar.app.google/pdzfMFmwACg2PN8w8...
I am currently following this example
I am currently following this example https://github.com/cloudflare/workers-rs/blob/078d15bb60d2b05af0236c7b1370f598f5e17ada/examples/queue/src/lib.rs#L113
I am facing the following error message, similar to https://discord.com/channels/595317990191398933/1237042263150956667/1237062080117473280
```
Queue test-queue (5 messages) - Exceeded CPU Limit @ 30/05/2024, 19:52:42...
I am trying to deploy the basic queue
I am trying to deploy the basic queue template worker (which worked last week) and am getting this error.
✘ [ERROR] A request to the Cloudflare API (/accounts/[REDACTED]/queues/[REDACTED]/consumers) failed.
workers.api.error.unknown [code: 10013]
...
could i have some help understanding
could i have some help understanding what all of the stats mean? im a little confused.
`Conflict` I've only seen once. Still
Conflict Error: Queue send failed: Internal Server Error occasionally on .send, which calling-worker-side retrying does not seem to fixGiven you can't use a pages app and a
Given you can't use a pages app and a separate worker to consume a queue locally what is people's current workflows for handling this? I was thinking of deploying a queue and then just using that but I'm not sure how to have 1 binding be remote but the rest local
Queue processing stops locally
Stopped locally? Do you see errors? Can you open a bug report on GitHub (including the wrangler version & your code) - ?
This is something on the R2 side, but we
This is something on the R2 side, but we're also working on R2 Event Notifications - e.g. uploading to R2 via a Signed URL or the S3 API will publish an event to one of your Queues that you can consume.
Some updates for @Queues users:
Some updates for @Queues users:
- We're working on landing HTTP Pull, which will allow you to poll a queue and consume it from outside of a Worker (or use a Worker in a poll-based flow vs. push-based). This will likely land just before/just after the holidays.
- Queue throughput: our major milestone is about 3-5k msg/sec per queue (message size plays a role here), up from the ~400 msg/sec/queue currently
- We're also working on landing controls around setting message delays - e.g.
await env.MY_QUEUE.send(msg, { deliveryDelay: 600 }) on the producer side, and retry delays on the consumer side - e.g. messages.batch.retryAfter(300) or .retryAfter(seconds: number) on a per-message basis. API not final....No: batch timeout is on the producer
No: batch timeout is on the producer side. It does not determine how the consumer scales. Your volume/message rate is very low, and (right now) we prefer to avoid scaling too quickly on the consumer side.
I have a single queue marked both as a
I have a single queue marked both as a producer and a consumer queue, attached to the same worker. The worker takes input from the queue and produces output to the same queue until the task is done. What I observe is, the task runs for couple of iterations and stops running.
Of course : )
Of course : )
So basically I want to locally debug the queue producer and queue consumer. The queue producer is a cloudflare pages app, built on Remix. The queue consumer is a worker with plain typescript, no framework.
I kept the worker that simple intentionally because it doesn't need database access or routing or whatever, it's basically a proxy for queue consumption (because cloudflare pages apps can't consume from a queue yet and other limitations)....
elithrar
@Matt !
Thanks for 1 and 3.
For 2: we finally got a reply from our Cloudflare TAM and yes, they stated we need to add a payment method on our non-prod account. I'm working on us setting that up....
Debugging fetch in a consumer
Yes, I'm
awaiting the fetch and I also put inside a try/catch, but no error is logged.
The last log I see is the API URL log, and then the code skips everything below the log
```ts
try {...are there any plans to offer amqp
are there any plans to offer amqp support for remote integration with queues? I'd love to use this as an alternative to RabbitMQ.
RuntimeError: memory access out of bounds
Som of my Rust queue consumers throw exceptions very frequently before even starting the queue event handler, see https://github.com/cloudflare/workers-rs/issues/374. Or it times out without anything happening. It somehow feels like there's an issue outside of my code. Any idea what could cause this? I've tried instrumenting for a core dump, but the recordCoredump in wasm-coredump expects a request object, see https://github.com/cloudflare/wasm-coredump/issues/3.
I’m just trying to make sure that there
I’m just trying to make sure that there isn’t any unexpected issue or asterisk with keeping them colocated