Suspicious traffic from a mystery railway app
Hello! My Sentry reported 10k errors just today that look like this:
Invalid HTTP_HOST header: 'liam.up.railway.app'. You may need to add 'liam.up.railway.app' to ALLOWED_HOSTS.
This is not a server / service owned by me. I don't know what it is. But I definitely wasn't prepared to pay for Sentry's 10k errors! I don't really know how to report this, or even what kind of problem this really is, so I posted it here. I don't know if this is a hacking attempt or scanner or an internal service or what. Is there anything I should do? I am a bit worried, to be honest.17 Replies
Project ID:
00e357e0-9d2a-45be-8a64-a58b8ee28694⚠️ experimental feature
00e357e0-9d2a-45be-8a64-a58b8ee28694
Hey! are you a railway customer? that url is linked to someone’s project on Railway. Can get the team to look into it
Yeah, I'm also using Railway. I stuck my project ID in there. I took my project down for now to hopefully reduce these errors. It was like 20 a minute D:
Very odd, definitely will get the team on this
Thank you!
Let me know if you need any more info. I'm not completely certain what would even cause this.
Is there any update on this? Hate to bother anyone, but I don't want to re-deploy my services until I understand better what's going on. Thanks!
Just want to come back to this one more time. I've been avoiding using Railway until I have more info, not sure what I can do.
team will check this out
Thank you!
i think percy broke
1sec
nvm it worrked lmfao
Heya! We're still not 100% certain but we've looked into it and we think it was a stale route propagation mixed with a racecase that allowed someone else to grab the "slot" on which your instance used to live
So, we'd proxy that URL to your instance, and presumably, you have some sort of Django or Ruby host header validation, which would have bounced those requests
Does that make sense?
I could see a universe in which this might be able to happen if your instance crashed, someone created a new deployment, you got REALLY unlucky with the port allocation, and then your instance got rebooted
Yeah this is Django header validation.
I've put up an RFC internally for this. I need to put in place the above port mutex for another feature (app sleeping), so I should be able to roll it out late this week or next
Perfect! I really don't want it to happen again, but if it's probably a low likelihood blip, I might deploy my services out again.
I cannot express to you how unlucky you got there. By my math, it's quite literally a 1:1m+ event
(If this is what I think it is, which, we think it is after jamming on it with another engineer)
Sounds like my luck 😄
likelyhood of it happening in the time between now and when I roll out this fix
Is even lower