R2 file processing inconsistent behaviour

Hi, I have a worker that fetches a file from R2 and then processes it. I'm looking to find the biggest file size that I can process in reasonable time on workers platform. The worker takes r2 path as parameter in the url. The files I'm testing with are: - 1, 10, 15, 20, 25, 50 MB (each file is tested in separate request) When I start testing using increasing size order it works up to 25 MB then it fails on 50MB with CPU time exceeded (that's ok). If I go back to test 25MB or even 1MB I'm still getting Error 1102 Worker exceeded resource limits. Why? It was working before. The environment didn't change and I'm processing the file exactly the same way. Currently I'm on Workers Free plan - does it matter? Just to be clear how to reproduce - requests order: https://xxx.workers.dev/?f=file10mb (OK) https://xxx.workers.dev/?f=file25mb (OK) https://xxx.workers.dev/?f=file50mb (FAILS Error 1102 Worker exceeded resource limits) https://xxx.workers.dev/?f=file25mb (FAILS Error 1102 Worker exceeded resource limits) https://xxx.workers.dev/?f=file10mb (FAILS Error 1102 Worker exceeded resource limits)
4 Replies
kian
kian2y ago
What's your Worker code?
duch
duchOP2y ago
Pastebin
export interface Env { MY_BUCKET: R2Bucket;}function objectNotFoun...
Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
kian
kian2y ago
Looks like you're doing lots of processing so that's expected You've got more data to process so it'll use more CPU CPU time isn't deterministic, sometime it'll use more and sometimes it'll use less.
duch
duchOP2y ago
Now, I also tested the same file (10MB) between 10-20 times and after around 10th try it exceeded resource limits. Thank you, that makes it clearer - I'll test with a paid plan.
Want results from more Discord servers?
Add your server