Railway•2y ago

Deployed services suddenly cannot connect to Postgres database

Hello, I have several services deployed that were working fine as of a few days ago that I've realized are down. I haven't made any changes on this project in quite a while, so I'm pretty sure I didn't misconfigure anything. There is a Postgres database in my project that suddenly cannot be connected to using Railway's preconfigured variables (${{Postgres.PGHOST}}, etc).
Rails gives the error:

ActiveRecord::ConnectionNotEstablished (connection to server at "(IP address redacted)", port (Port redacted) failed: Connection timed out
Is the server running on that host and accepting TCP/IP connections?

ActiveRecord::ConnectionNotEstablished (connection to server at "(IP address redacted)", port (Port redacted) failed: Connection timed out
Is the server running on that host and accepting TCP/IP connections?

However, if I use a desktop app (Postico) to try to connect to the Railway database directly, it's fine.

21 Replies

Percy•2y ago

Project ID: b138184c-c87d-46ec-b684-4965890ad049

jedmundOP•2y ago

Project ID: b138184c-c87d-46ec-b684-4965890ad049

Brody•2y ago

can you connect to the database with pgadmin?

jedmundOP•2y ago

I don't have it installed but I'll try now. Postico is fairly similar so I bet it will work Give me a few minutes to install it and loop back

Brody•2y ago

yeah just some desktop client that can connect to postgre, doesnt have to be pgadmin

jedmundOP•2y ago

In that case, yes. Works instantly using the postgres URL

Brody•2y ago

can you try to configure your code to use the url

jedmundOP•2y ago

notably, my desktop client is using a containers-us-west-XX.railway.app host but inside of my app it looks like railway is giving my app an IP address yeah, thats what I was gonna try

Brody•2y ago

using ip is fine, as long as the connection is made with the host containers-us-west-XX.railway.app

jedmundOP•2y ago

redeploying now so we'll see what happens

Brody•2y ago

youre now using DATABASE_URL in code? and you have a variable reference setup for that in your service?

jedmundOP•2y ago

yep everything was working fine for months until some undetermined date in the past 3ish days

Brody•2y ago

interesting, well let me know how that change goes

jedmundOP•2y ago

I have a staging environment that I was able to upload my build to that uses the exact same variable references as the production environment, and that one works completely fine. The production one still will not connect despite being the same code. I tried removing the variable references and re-adding them and that didn't do anything obviously. It's Rails so it's not like I'm writing any database connection logic... This makes absolutely no sense lol For context, another service in the same project also is trying to activate the production database and is completely failing to as well

Error: connect ETIMEDOUT 34.127.35.192:6620
[1]     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16) {
[1]   errno: -110,
[1]   code: 'ETIMEDOUT',
[1]   syscall: 'connect',
[1]   address: 'IP REDACTED',
[1]   port: PORT REDACTED
[1] }

Error: connect ETIMEDOUT 34.127.35.192:6620
[1]     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16) {
[1]   errno: -110,
[1]   code: 'ETIMEDOUT',
[1]   syscall: 'connect',
[1]   address: 'IP REDACTED',
[1]   port: PORT REDACTED
[1] }

Adding a new Postges instance and migrating my data fixed the issue. I have no clue what went wrong but that container was made in January 2022 so maybe its something to do with the age? I haven't deleted the old instance yet for if someone at Railway wants to debug

Brody•2y ago

could you send the plugin id?

jedmundOP•2y ago

where can i find that?

Brody•2y ago

open the old database and copy the id from the url /plugin/<plugin id>

jedmundOP•2y ago

d17b4bb2-8c6e-4e95-9490-8c8d8d3f9438 thats what i thought :P wasnt sure if it was in the UI somewhere too

Brody•2y ago

thanks, i have shown the team this thread

jedmundOP•2y ago

👍🏾

Ray•2y ago

This was caused by https://railway.instatus.com/clik6k1hk294312awn8v8rox7a1. It should be fixed now, try again?

Gaming

Programming

Deployed services suddenly cannot connect to Postgres database