Cloudflare Developers•3mo ago

Is there any plant to increase this message retention period ? (https://developers.cloudflare.com/qu

Is there any plant to increase this message retention period ? (https://developers.cloudflare.com/queues/platform/limits/#:~:text=retention%20period%203-,4%20days,-(96%20hours)) 4 days is too low to build any serious application. for e.g. AWS SQS has a max retention period of 14 days. I thought it would be also increased during GA announcement.

15 Replies

Zack•3mo ago

Yea, kind of agreed on the above. Would be nice to be able to set 7 days at least. With 4 days, if you receive a failure on Friday, you basically only have Monday to pick it up and replay

princeMN•3mo ago

#queues

an•3mo ago

Can I let a queue message run in a specific colo/ region? any idea to achieve this

Unknown User•3mo ago

Message Not Public

tadhglewis•3mo ago

Working fine for me 🤔

Unknown User•3mo ago

Message Not Public

pat•3mo ago

Hey fellows, I've got a really simple queues and d1 setup that is deployed as a worker. It is a consumer and a producer of the job that it handles. It basically uses the default wrangler.toml and I followed the tutorials on the respective page to set it up. The fetch just publishes a single event, the queue just logs it

export default {
    async fetch(request: Request<unknown, IncomingRequestCfProperties<unknown>>, env: Env, ctx: ExecutionContext): Promise<Response> {
        await env.aramDataCollectionQueue.send({
            riot_puuid: "secret",
            region: RegionGroups.SEA
        })

        return new Response("Success");
    },

    queue(batch: MessageBatch<CollectAramDataJob>, env: Env, ctx: ExecutionContext): void | Promise<void> {
        console.log(batch);
    }
} satisfies ExportedHandler<Env, CollectAramDataJob>

export default {
    async fetch(request: Request<unknown, IncomingRequestCfProperties<unknown>>, env: Env, ctx: ExecutionContext): Promise<Response> {
        await env.aramDataCollectionQueue.send({
            riot_puuid: "secret",
            region: RegionGroups.SEA
        })

        return new Response("Success");
    },

    queue(batch: MessageBatch<CollectAramDataJob>, env: Env, ctx: ExecutionContext): void | Promise<void> {
        console.log(batch);
    }
} satisfies ExportedHandler<Env, CollectAramDataJob>

I am trying to develop locally with bunx wrangler dev but when I go to the url it runs on it just hangs for a while with no error output. After a reasonable amount of time it will start lagging out my computer and starts spamming to console

[wrangler:inf] GET / 500 Internal Server Error (78202ms)

[wrangler:inf] GET / 500 Internal Server Error (78202ms)

I am very new to cloudflare in general but I couldnt find anything in the docs

Mateus Eduardo•3mo ago

hello help me, even though the batch size is 1 when I trigger a message in the queue with delay seconds of 5 minutes. After this time, this message is not sent to my consumer. Is it only triggered when I add another message to the queue?

flof.fly•3mo ago

How do I create a queue for a test env worker, and another queue for another worker in prod? I have a bit of confusion on how multi-env works in wrangler

kian•3mo ago

[[env.???.queues.consumers]]/[[env.???.queues.producers]]

flof.fly•3mo ago

Nocapping thanks a lot kian

pao ramen•3mo ago

Hello! I have a very misterious bug with queues. I have a job that iterates a batch of 3 messages, but it only iterates 2 messages and then it never gets to the third:

export async function queue(batch, env) {
    console.log({ messages: JSON.stringify(batch.messages) }) // This logs an array of 3 objects

    for (const msg of batch.messages) {
        console.log({ message: JSON.stringify(msg) }) // This only logs twice

        try {
            // ...do things
            msg.ack()
        } catch (e) {
            console.error(`adjustUser error: ${e}`) // This never logs
            msg.retry()
        }
    }
}

export async function queue(batch, env) {
    console.log({ messages: JSON.stringify(batch.messages) }) // This logs an array of 3 objects

    for (const msg of batch.messages) {
        console.log({ message: JSON.stringify(msg) }) // This only logs twice

        try {
            // ...do things
            msg.ack()
        } catch (e) {
            console.error(`adjustUser error: ${e}`) // This never logs
            msg.retry()
        }
    }
}

This is my wrangler configuration

[[queues.producers]]
queue = "adjust-user"
binding = "ADJUST_USER_QUEUE"

[[queues.consumers]]
queue = "adjust-user"
max_batch_size = 10
max_batch_timeout = 5
retry_delay = 600
dead_letter_queue = "failed-adjust-user"

[[queues.producers]]
queue = "adjust-user"
binding = "ADJUST_USER_QUEUE"

[[queues.consumers]]
queue = "adjust-user"
max_batch_size = 10
max_batch_timeout = 5
retry_delay = 600
dead_letter_queue = "failed-adjust-user"

Any clue?

Saqib (Langbase)•3mo ago

Hey folks, I am trying to do some processing on queues, surprisingly they timeout after 60s inside the consumer. On the paid plan, currently testing locally. Any thoughts? It just stops processing no error, nothing.

toadfans•3mo ago

hello, is queue down now?

{

            "name": "Error",
            "message": "Queue send failed: Internal Server Error",
            "stack": "Error: Queue send failed: Internal Server Error\n    at async Object.mutation (bundledWorker-0.9232964110947641.mjs:9562:3)\n    at async Object.handler (bundledWorker-0.9232964110947641.mjs:28094:17)\n    at async o (bundledWorker-0.9232964110947641.mjs:4128:13)\n    at async r (bundledWorker-0.9232964110947641.mjs:4210:92)\n    at async o (bundledWorker-0.9232964110947641.mjs:4128:13)\n    at async Object.handler (bundledWorker-0.9232964110947641.mjs:14535:28)\n    at async o (bundledWorker-0.9232964110947641.mjs:4128:13)\n    at async r (bundledWorker-0.9232964110947641.mjs:4210:92)\n    at async o (bundledWorker-0.9232964110947641.mjs:4128:13)\n    at async Object.handler (bundledWorker-0.9232964110947641.mjs:8361:42)"
          }

{

            "name": "Error",
            "message": "Queue send failed: Internal Server Error",
            "stack": "Error: Queue send failed: Internal Server Error\n    at async Object.mutation (bundledWorker-0.9232964110947641.mjs:9562:3)\n    at async Object.handler (bundledWorker-0.9232964110947641.mjs:28094:17)\n    at async o (bundledWorker-0.9232964110947641.mjs:4128:13)\n    at async r (bundledWorker-0.9232964110947641.mjs:4210:92)\n    at async o (bundledWorker-0.9232964110947641.mjs:4128:13)\n    at async Object.handler (bundledWorker-0.9232964110947641.mjs:14535:28)\n    at async o (bundledWorker-0.9232964110947641.mjs:4128:13)\n    at async r (bundledWorker-0.9232964110947641.mjs:4210:92)\n    at async o (bundledWorker-0.9232964110947641.mjs:4128:13)\n    at async Object.handler (bundledWorker-0.9232964110947641.mjs:8361:42)"
          }

requestId: 8cfd3fdf8f3984b7

Lance•3mo ago

Hi, anyone encounters similar issue that queue's backlog just grows and no message is delivered to worker handler? Nothing changed from code side. It just suddenly happened a few hours ago.

Gaming

Programming

Is there any plant to increase this message retention period ? (https://developers.cloudflare.com/qu