Cloudflare Developers•2mo ago

Hello! I have a very misterious bug with

Hello! I have a very misterious bug with queues. I have a job that iterates a batch of 3 messages, but it only iterates 2 messages and then it never gets to the third:

export async function queue(batch, env) {
    console.log({ messages: JSON.stringify(batch.messages) }) // This logs an array of 3 objects

    for (const msg of batch.messages) {
        console.log({ message: JSON.stringify(msg) }) // This only logs twice

        try {
            // ...do things
            msg.ack()
        } catch (e) {
            console.error(`adjustUser error: ${e}`) // This never logs
            msg.retry()
        }
    }
}

export async function queue(batch, env) {
    console.log({ messages: JSON.stringify(batch.messages) }) // This logs an array of 3 objects

    for (const msg of batch.messages) {
        console.log({ message: JSON.stringify(msg) }) // This only logs twice

        try {
            // ...do things
            msg.ack()
        } catch (e) {
            console.error(`adjustUser error: ${e}`) // This never logs
            msg.retry()
        }
    }
}

This is my wrangler configuration

[[queues.producers]]
queue = "adjust-user"
binding = "ADJUST_USER_QUEUE"

[[queues.consumers]]
queue = "adjust-user"
max_batch_size = 10
max_batch_timeout = 5
retry_delay = 600
dead_letter_queue = "failed-adjust-user"

[[queues.producers]]
queue = "adjust-user"
binding = "ADJUST_USER_QUEUE"

[[queues.consumers]]
queue = "adjust-user"
max_batch_size = 10
max_batch_timeout = 5
retry_delay = 600
dead_letter_queue = "failed-adjust-user"

Any clue?

2 Replies

harshil1712•2mo ago

Are the messages getting delivered with the specified max_batch_timeout? Also, do you see the third message getting delivered in the next batch, at all?

pao ramenOP•2mo ago

Are the messages getting delivered with the specified max_batch_timeout?

Yes, I receive the 3 messages on the batch. The problem is not on the batching, but that somehow the loop does not run 3 times. the first console.log prints 3 messages:

{ messages: [
  {"attempts":1,"body":{"userId":"XXX"},"timestamp":"2024-10-08T12:00:39.876Z","id":"XXX"},
{"attempts":1,"body":{"userId":"YYY"},"timestamp":"2024-10-08T12:00:40.130Z","id":"YYY"},
{"attempts":1,"body":{"userId":"ZZZ"},"timestamp":"2024-10-08T12:00:40.337Z","id":"ZZZ"}
]}

{ messages: [
  {"attempts":1,"body":{"userId":"XXX"},"timestamp":"2024-10-08T12:00:39.876Z","id":"XXX"},
{"attempts":1,"body":{"userId":"YYY"},"timestamp":"2024-10-08T12:00:40.130Z","id":"YYY"},
{"attempts":1,"body":{"userId":"ZZZ"},"timestamp":"2024-10-08T12:00:40.337Z","id":"ZZZ"}
]}

but then the console.log inside the loop only prints twice:

{ message:
  {"attempts":1,"body":{"userId":"XXX"},"timestamp":"2024-10-08T12:00:39.876Z","id":"XXX"}
}

{ message:
  {"attempts":1,"body":{"userId":"YYY"},"timestamp":"2024-10-08T12:00:39.876Z","id":"YYY"}
}

{ message:
  {"attempts":1,"body":{"userId":"XXX"},"timestamp":"2024-10-08T12:00:39.876Z","id":"XXX"}
}

{ message:
  {"attempts":1,"body":{"userId":"YYY"},"timestamp":"2024-10-08T12:00:39.876Z","id":"YYY"}
}

And I can't see any error on the logs. uhh... I think I've got the bug. It's on my end. On the code that I've omited, there was an early return, and it looks like returning acks all messages automatically

Gaming

Programming

Hello! I have a very misterious bug with