Error 500 (Internal Server Error)! on assets
We are getting a strange 500 Error occasionally on assets - e.g. gif, jpg and JavaScript. We get back HTML displayed as raw text: <html><title>Error 500 (Internal Server Error)!</title></html>
It does randomly also affect PHP files - but our assets are not hitting PHP, they are served directly by nginx, and the error page is not an nginx 500. Has anyone else seen this? I did see some people have similar issues and get sent into Discord!
Thank you! 🤞
23 Replies
Is it branded Cloudflare? Can you give a url of a broken asset, or do you have a screenshot/request headers/etc?
If it was a Cloudflare issue, usually you would see the normal nginx error page, but with "cloudflare" instead of nginx at the bottom. If that payload above is all it returned, doesn't sound like Cloudflare. If you see CF-Cache-Status in the response headers, it's from your origin
There's a generic community guide here, if it helps:
https://community.cloudflare.com/t/community-tip-fixing-error-500-internal-server-error/44453
It seems the error is happening on another website too that is nothing to do with ours, also powered by Clouflare - https://beta.character.ai/ I wonder if its related to beta being in the url! our url is: https://edinburgh-beta.org/
Forever Edinburgh
The Official Guide to Edinburgh - Forever Edinburgh
Edinburgh’s beauty is both staggering and stunning but the city is more than just a pretty face.
THe other site is alos using Google hosting
yea that doesn't seem cloudflare error pagey, I think that's the normal google one though
Searching he web for "<html><title>Error 500 (Internal Server Error)!</title></html>" gives nothing, we found this interesting Twitter thread: https://twitter.com/search?q=%22%22%3Ctitle%3EError%20500%20(Internal%20Server%20Error)!%3C%2Ftitle%3E%22&src=typed_query
This unrelated person was getting the issue a few minutes ago, Cloudflare and Google - and it seems to have settled down now - I think the fact we both have beta in the url is definitely interesting!
We turn on Developer Mode to disable CLoudflare then it doesn't happen, so seems related to Cloudflare, maybe also Google
We are thinking its possibly Google issue, still got it with vcache disalbed, perhaps Google is doing something weird with beta in url
so this is a totally uinrelated site to ours
our site is: https://edinburgh-beta.org/
Forever Edinburgh
The Official Guide to Edinburgh - Forever Edinburgh
Edinburgh’s beauty is both staggering and stunning but the city is more than just a pretty face.
So its a mystery! THinking it must be Google and "beta" in url
It only just happened this evening, and this other site just started complaiing on TWitter tonight too
If you keep refreshing this you will see it: https://beta.character.ai/
When we disable Cloudfront cache it seems to not happen, cant trigger it by refreshing.
The exact string "Error 500 (Internal Server Error)!" appears nowhere in the Internet until today!
Hre is a braking asset:
https://edinburgh-beta.org/wp-content/themes/forever-edinburgh/theme/images/fe-logo-orange.svg?v=8794be0d
Nice, yea that error is from your origin, you can tell from the request headers
(The fact you have cf-cache-status means it's a response CF fetched from an origin)
We turned on Google logging, we can see it returned a 200 to Clourflare (we checked the IP address) but the site gave us the 500
but the plot thickens! We can also see a 500 from Google end too
I'm not too familiar with Google products, but it looks like it's breaking pretty early in the request flow, a successful request has trace headers (x-cloud-trace-context) and x-powered-by, there's no such headers at all in the error page, load balancer or something early failing
Yes, the fact the other site we are not related to getting this same exact strange exact error string is pointing to something! They hosted on Google too
Yup this from Google, matching the 500 we seeing
So something nuts - seems when Clourflare asking, Google giving a 500, whewn we bypass Clourflate we getting 200
So what we can see: when Cloudflare enabled, Google is serving 500's to it, and we get them in browsert. When we disable Cloudflate cache, Google is only delivering 200's from same urls - so an issue relating to Goolge serving content to Cloudflare?
unless it's 100% of the time, I would guess it's just intermittent errors. I don't see anything much on Google's Status Page, guessing the logs there don't give any more information to why it is erroring out
We'll open issue with Google, and I've tweeted to @character_ai to let them know we are hiaving similar issues. I can see thay are doing console.logs of their response codes!
So we are basically two separate poor groups of developers with the same issue frantically debugging 🙂
@chaika.me and @Leo Thank you both for your time on this, its midnight so we're off to bed and will see tomorrow if the issue is resolved - If character ai is no longer broken we can check response headerts and see if they still using Cloudflare and Google. Somewhere out there, there's some code spitting out "<html><title>Error 500 (Internal Server Error)!</title></html>" ðŸ˜
Wee update, the problem has mysteriously vanished, I can see that the other site is no longer conmsole logging to debug. Can only assume it was Google, and something to do with Beta in the name, and it got fixed in the night! Oh well, lost an evening but it was fun diagnosing, thanks for helping out!