worker functions locally, but not when deployed
In production this code fails to retrieve data like 80% of the time. I added some debugging code to retry the request in the worker and it sometimes will fail the first x requests and eventually works and sometimes it fails all the requests. When running locally it works 100% of the time.
Is there a way I can run cloudflare workers on my own servers (open to suggestions for cloud providers) , or a way to fix my code to make this work 100% of the time?
Here is a link you can open to test
and the source code is attached
https://banner.mines.rocks/?season=fall&year=2021&courseNumber=250&subject=CSCI
16 Replies
The failure is some sort of cookie authentication failure I think because the request doesn't get rejected it just returns a successfull request with no data
I'm guessing this is some kind of bug or quirk with running workers on cloudflare, really odd that it works locally but not in production
Does the base URL use a custom port?
Nevermind, it's at the top.
referrer isn't the name of the header, so I don't see how that would break anything.referrer gives me an error, but it isn't necessary
What specifically isn't working in this code? Is it requesting a service you host? Are you sure they aren't just blocking or redirecting Workers?
So I'm pinging an API that I'm not hosting
this could be the case but it is odd that some workers go through and that the same worker can retry successfully and get a response
here is a response where the first 4 requests fail and the 5th request suceeds
when the 3rd party api I'm pinging fails it doesn't give an error or anything useful, it just sends no data :/
Running locally it works 100% of the time with only 1 request every time
hmm globally the 3rd party api seems to add a load balancing cookie for some reason
ok deleting the load balancing cookie doesn't seem to change anything
yeah I don't understand either took me a while to figure it out, first request gets the cookies or something and tells the server what term is specified
are you saying that you ran the worker locally and are running into the same issue that I am having running it on cf's servers?
maybe I should file a bug report, idk where that is
It could be the 3rd party has a rate limit based on IPs. I know some APIs do and with shared address of workers can run the limit out
this isn't really an API that should be getting hit super often, and it fails and works on the same worker without any error response from the server
unless each fetch req is getting sent from a different IP
even in the same worker run
It's possible - this makes 5 requests in one go and records the IP it was sent from. https://worker-withered-tree-21ff.kianorguk.workers.dev/
that is super helpful, can you send me the src for that? can see if there is a pattern to successful api calls
and if you run it locally do you get the same IP every time?
bc I'm wondering if the API I'm calling expects the IP to remain static
Locally as in
wrangler dev just runs on your PC, it always would.
wrangler dev --remote runs as if it was published, on Cloudflare's edge, and would have different IPs.wrangler dev --remote is borked as well for me, wrangler dev works fine though, so it could be the IP rate limit or the non static IP thing
Cloudflare Community
Worker keep same Ip
This is also a use case for me ā Iām using an external API that generates a download link that is locked to the IP that requests the link. So I just need to do two back-to-back fetch() requests from the same IP. Note that the IP does not have to be consistent across invocations.
A particular Cloudflare server doesn't have a specific IP assigned to it, it'll always vary - https://blog.cloudflare.com/cloudflare-servers-dont-own-ips-anymore/
The only way for it to be static is an Enterprise add-on (Aegis) or routing it all through your own HTTP proxy.
guess I'll have to figure out how to set up an HTTP proxy, no way I can afford Enterprise
if you have any experience or tips for that lmk
ok, I set up an EC2 instance and I'm running a cloudflare tunnel on it as a reverse proxy, and it works every time now š
super annoying that you cant set workers to be static ips or at least have the same damn ip for consecutive fetch requests