US East, super slow
I just moved "everything" over to US East. And .... it was slow. Like broken levels of slow. The requests were all delayed by seconds. And <Link>s would simply not work (or take close to 10 seconds to load).
For now, I moved everything back to US West. And it is fast again.
I put everything in
"" because there was one thing not in US East. the DB was in West. This could be something that slows it down. But no, this was seconds of delay between requests. Requests that do not take long at all to make.
I really wish I could sit here and help debug but I can't afford to have prod be in a broken state for my testing purposes. I might test again and put everything in US East on off working hours (IE when my servers won't be used). And while I do that, I will test with the DB in US West and then move it over. But beyond that, idk how much testing I will be able to do to help debug this issue.
I don't know what the issue actually is though. Becuase it isn't like like request are taking a long time to respond based on the request durations (both client and server were claiming quick responses. Lies!!!!)
project id: 1b326884-0c17-43ed-9b52-f443662e8f5031 Replies
Project ID:
1b326884-0c17-43ed-9b52-f443662e8f50Sorry for breaking things again :cry_gil:
I wish I took a video like I normally do. But Like I said, I was on pure "fix prod" mode. So getting a video to debug the platform was not on my mind
:angrykirbs: and of course I didn't have my analytics tool on that would normally have picked up this recording. Welp, wish I could provide more data. Really want to know what happened. But oh well I guess. ....
qq where you connecting to the database privately?
The backend was, yes
And while I could see that messing up the backend, I am not convinced it would mess with the frontend so much. In theory possible. But I have a hard time seeing that be the case. I will investigate like I said. But either way ....
so would static assets take up to 10 seconds to load too?
Yea
No, its ok. Its ok to laugh.
thats not me
i promise
i assume you have the edge proxy enabled on the frontend service right?
To remind you on the structure of this project; the frontend and backend are privately talking to a reverse proxy that is exposing itself. So that reverse proxy did have the edge proxy enabled. It was actually one of the first things I tried; toggling between it on and off
did the proxy service have the edge proxy enabled the moment you move the region of it?
I honestly don't remember if it was enabled before or after moving regions.
may i ask your region
US East
Hence me wanting it on east
this sounds eerily similar to what another user reported
https://discord.com/channels/713503345364697088/727689277219012669/1251764921960501258
they arent pro, so it was just after first deploy to us-west
but i dont know what me or the team could do here tbh
Not really asking you to do anything. This is more like a pure, throw hands up in the air, bug report.
I really love the tools and ideas of railway. But overall it has been a very buggy experience. It is a lot of little things. Most all of the bugs can be excused and don't bother me at all. And I will still be using it for this project. But I am not convinced on migrating other projects onto the platform. The little things do add up.
And I am not saying this to make anyone feel bad. Actually, if anything, it is the opposite. Despite the buggy experiences I have had, I am still really interested in using railway. And I really do want to see this company / products grow and succeed. I am just not sure I will be putting any project that requires pro status on the platform just yet. But even at that, I am sure I will migrate some of the projects over here.
Examples of "the little things"
* Groups sometimes just decide to bunch up (pic 1 & 2)
* Randomly the screen will flash with 404 and then everything will return to normal
* Sometimes views just don't load (like if I click on "logs" on a service) so I have to escape and then do it again. And then it will load
* All the bugs I have been pointing out in my other posts.
* Some other little things I hit with root directory and adding a repo (will likely add details about that in a different thread)
Again, despite that, I still do like the platform. I like the ease of use. I like the way it aggregates logs. I like the the observability. I like the auto deployment on push and ease of environments. I like the structure of projects and services. I like a lot of things. But I am also a little frustrated. I am sure I will get right back to it though after I get some sleep. So I am going to go do that now
i feel you, ive been here with you and i can tell you are a tad frustrated, and i can't blame you one single bit, you have all the right to be because you for sure have not had the best experience with the platform, if you have the time would it be possible to ask for a list of all the threads you've opened so that the team has all them brought together into a single place?
Yea I can get that all together. Tomorrow is either going to be super busy or super not. No in between. So if I get it tomorrow, it was super not busy.
no worries, and thank you for your tremendous patience despite your poor experience with the platform
I put everything in "" because there was one thing not in US East. the DB was in West.If you're doing something weird with your querying, 100% it could blow up like this seconds would have to be something thats all kinds of messed up, but, this is the first I'm hearing of this here And I think that the conductors are? We run blackbox testing with latency pings and they look fine
that would be a safe bet to think it was backend to database issues, but even static assets from their frontend where slow to load
I'm genuinely not sure what this could be
I run an app that pings the internal applications
https://us-east-production.up.railway.app/
https://asia-production.up.railway.app/
https://us-west-production.up.railway.app/
https://europe-production.up.railway.app/
hmm
intermittent failures, I refreshed a couple times and ~2/3 of the time it came back with this
I should be sleeping; but ummm. I am not. Anyways, can confirm. I see the same thing
As should Brody and I lol
well it is SSR so anyone would see the same
I really really don't think so. It again could be the db. However, yea, the static assets would take a long time. But also, the client and server were claiming that the response duration were ms. But it was definitely not. So something strange with something was going on. Maybe it was the private network. Maybe it was a fluke failed/messed up deploy. Idk.
Oh and also yea, I don't use a VPN
Think I need to redeploy that one. But if you check any of the other ones, they’re rock solid
Is there a possibility that this issue is related to https://discord.com/channels/713503345364697088/846875565357006878/1252985781375471637 ?
Maybe? Has it resolved itself
idk. I moved everything back to west. I might test moving things over to east later today. We shall see.