I am trying to test locally getting a file from R2 bucket which i added via dashboard
I created a worker cron job and am testing locally via wrangler dev --test-scheduled. calling await env.MY_BUCKET.get('myfile.json'); returns null even though the file exists in my R2 bucket dashboard. Why is this the case?
29 Replies
default dev is local/all simulated, have to use --remote if you want to use remote resources
Is there way to manually add files to r2 simulated bucket via dashboard?
It wont let me use production one:
In development, you should use a separate r2 bucket than the one you'd use in production. Please create a new r2 bucket with "wrangler r2 bucket create <name>" and add its name as preview_bucket_name to the r2_buckets "MY_BUCKET" in your wrangler.toml
The whole thing with the simulated bucket is that it's all local
so nothing in the dashboard is going to be able to touch your local resources
Thanks
Im getting a 504 Gateway Time-out cloudflare workers for my cron job at 120214ms. I have no cpu limit set and paying for the standard plan
What are you doing in the Worker?
Making hundreds of requests to a third party api which im using that fetched data from those api requests and putting them in a json which at end i will push to R2
and where you seeing the 504?
i'm not surprised you see it if you fetch anything - after 100 seconds (which you're doing 120 total) it'll return that
yeah so my guess - that third party api is timing out
you can debug to confirm but that's my theory
Getting 504 in console after running npx wrangler dev --test-scheduled --remote
I ran this script locally though and it worked fine
Would cloudflare not timeout ever in this case?
I'm not sure what you mean
Say the third party api wasn't timing out, what else could cause the 504 gateway timeout on cloudflare's side for this cronjob
I wonder if --test-scheduled is essentially doing a http request and that's why it's timing out too
since your Worker isn't responding in 100 seconds - timeout
Gotcha. It's always timing out around 122411ms as well
I'd need to check the code and see what --test-scheduled is doing
but 504 in cf == no response for 100 seconds
(or whatever your configured limit is - for you it'll very likely be 100 seconds)
But it seems to give error after 122 seconds of running
yeah the 20 seconds is not that relevant, bunch of reasons there could be 20 more seconds
hell you could have 650 seconds and it still be this limit
it's a long explanation for why
Gotcha, how would i go about debugging this then
would this theoretically work in prod withou --test-scheduled flag?
Well, see how long your fetches and things are taking to return
if --test-scheduled was doing a http request then maybe
that theory is that we're timing out on the worker response
rather than the third party response
i'd debug the third party first as that's easier than throwing in prod and waiting
Test scheduled events by visiting /__scheduled in browser
the third party response isn't timing out. I know that because we're looping over the same api request multiple times with diff params
and randomly on a certain loop it times out
oh yeah you're doing a http requesthere so very possible we're timing out that way
this is a fun little not even bug but behaviour, you test crons over a request but request timeout is a thing that way -- however that isn't the case for proper crons
will check
Times out after
How is a cpu time limit exceeded possible i have no timelimit set
there's still a 30 second cpu time limit
you can set a lower one but standard still has a limit
so is there no solution besides make script run shorter?
i mean if you're hitting cpu limit then you can try to optimise that
otherwise push to a queue and process in shorter amounts
Ill try pushing to a queue. Thanks for help
Instead of a cron job for calling a third party api to get the inidividual point score for an array of user account id's to have a live leaderboard, what if i used durable objects (one for each user account id) and the alarms api? Would that make sense
👋 @Walshy I'm running into a similar timeout issue that @f8 mentioned above. I'm running browser rendering to scrape pdfs. Running wrangler dev locally ($ npm run dev -- --remote -e stage --test-scheduled) and triggering scheduled event with $ curl "http://localhost:8787/__scheduled", once the worker execution time hits around 80 seconds, it starts executing all over again as if the curl was called again - but it wasn't called again. Is this by design in workers?
pls make a new thread rather than re-use