Service up but giving timeouts
Hi! Since 5-10 minutes ago I cannot connect to my webapp (neither via normal url, https://lostrego.up.railway.app/, or my custom one).
The build logs and everything looks good, but its unreachable.
Any incidents going on?
39 Replies
Project ID:
24229b64-e963-4d48-8c97-2f4befd0cf5d24229b64-e963-4d48-8c97-2f4befd0cf5d
Are you using the port that railway provides
Yes, as always. I actually didnt change anything.
site works for me right now, is it still timing out for you?
I did redeploy to see if that could solve it, but couldnt 😦
Oh... that's odd
Times out for me too 🤯
I take that as a yes?
It times out for me both via wifi and 4G
what are your regions
US West (Oregon)
And I'm in Spain
Hey, I am having the same issue actually with my own project and also Richy's project times out for me too, and I'm in Spain too
The other service I have, a mongodb one, seems to be working
Fyi times out for other people seems to be platform wide , happens on my project too
okay going to notify the team
Thank you so much for the quick response
In case it's helpful: All of my sites (proxied through Cloudflare, if that makes any differences) has been timing out for about 10-15 minutes. I'm in the UK, my services are hosted in US West (Oregon) region.
All our sites are down as well ERR_CONNECTION_TIMED_OUT
I'm also receiving Cloudflare 522 errors on my service
team is looking into it, an incident should be called soon
+1, customers calling already
incident has been reported, please keep an eye on #🚨｜incidents
Hey all, the team is actively on the case, we believe it to be related to hostile traffic targeted at our EU proxy which is affecting routing over to workloads no matter the region the workload is hosted on.
We apologize for the inconvienience.
Folks, quick ask, do you have URLs that we can ping and test just to make sure that we know the routing tables are re-populating?
We are seeing our proxies recover, but that means nothing if links don't work.
Looks accessible now?
Not on my end, yet?
Gotcha- we are re-populating the routing tables as we speak, sending those examples to our team.
Seems we hit a snag and the hostile traffic resumed from a different source, we apologize, we have the whole team throwing everything we can to recover all access gracefully.
Keep heads cold, good luck
We have updated the #🚨｜incidents accordingly. We will update y'all as more information comes up.
Will changing region resolve this?
Unfortunately no, as this is with our edge proxies, so it's a matter of how requests are routed on the Railway platform.
We are losing money by the second, we need a quick fix
Acking @ladet - we can help you deploy on another platform and help you route your workloads via Cloudflare. The Railway team wants whats best for your business. All I can promise you is our transparency, and our resolve to fix this issue.
Thanks for the updates on the situation!
Least we can do...
Appreciate the understanding, and genuinely, we feel beholden to your business and workloads. After all this, we will write up a report on what's been occurring on the platform over the last week.
incident has been mitigated, closing out 