Railway•10mo ago

Vapor (Swift) Postgres connection failure

I have a Swift Vapor project that I'm trying to set up on Railway. I've tried to connect using both DATABASE_PRIVATE_URL and DATABASE_URL, but both are failing at the deploy stage with SSL handshake errors.

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOSSL.NIOSSLError.handshakeFailed(NIOSSL.BoringSSLError.sslError([Error: 268435581 error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED at /build/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSL/ssl/handshake.cc:393]))) [database-id: psql]

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOSSL.NIOSSLError.handshakeFailed(NIOSSL.BoringSSLError.sslError([Error: 268435581 error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED at /build/.build/checkouts/swift-nio-ssl/Sources/CNIOBoringSSL/ssl/handshake.cc:393]))) [database-id: psql]

The project is working on Heroku, but I'd hoped to migrate over to Railway. Is there some other config that I've missed? 969baf4b-c7d4-413b-bcc3-c69b178b8450

17 Replies

Percy•10mo ago

Project ID: 969baf4b-c7d4-413b-bcc3-c69b178b8450

Effing.BombastOP•10mo ago

Just a quick follow-up to this, I can run the Vapor project against the Railway database in my local dev environment with the following config:

app.databases.use(DatabaseConfigurationFactory.postgres(configuration: .init(
    hostname: "roundhouse.proxy.rlwy.net",
    port: 57922,
    username: "postgres",
    password: "###########",
    database: "railway",
    tls: .disable // FIXME: re-enable
)), as: .psql)

app.databases.use(DatabaseConfigurationFactory.postgres(configuration: .init(
    hostname: "roundhouse.proxy.rlwy.net",
    port: 57922,
    username: "postgres",
    password: "###########",
    database: "railway",
    tls: .disable // FIXME: re-enable
)), as: .psql)

On Railway I've attempted to deploy the Vapor project with the following configs:

DATABASE_HOST=${{RAILWAY_TCP_PROXY_DOMAIN}}
DATABASE_PORT=57922

DATABASE_HOST=${{RAILWAY_TCP_PROXY_DOMAIN}}
DATABASE_PORT=57922

and

DATABASE_HOST=${{RAILWAY_PRIVATE_DOMAIN}}
DATABASE_PORT=5432

DATABASE_HOST=${{RAILWAY_PRIVATE_DOMAIN}}
DATABASE_PORT=5432

and both crash on deploy with the CERTIFICATE_VERIFY_FAILED error above.

Brody•10mo ago

use the private network and disable certificate verification

Effing.BombastOP•10mo ago

Thanks @Brody, I've tried on ports 5432 and ${{Postgres.PGPORT}} (16604) and both crash with a socket address error.

DATABASE_HOST=${{RAILWAY_PRIVATE_DOMAIN}}
DATABASE_PORT=5432
DATABASE_USERNAME=${{Postgres.POSTGRES_USER}}
DATABASE_PASSWORD=${{Postgres.POSTGRES_PASSWORD}}
DATABASE_NAME=${{Postgres.POSTGRES_DB}}

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOPosix.NIOConnectionError(host: "(redacted)-site.railway.internal", port: 5432, dnsAError: Optional(NIOCore.SocketAddressError.unknown(host: "(redacted)-site.railway.internal", port: 5432)), dnsAAAAError: Optional(NIOCore.SocketAddressError.unknown(host: "(redacted)-site.railway.internal", port: 5432)), connectionErrors: [])) [database-id: psql]

DATABASE_HOST=${{RAILWAY_PRIVATE_DOMAIN}}
DATABASE_PORT=5432
DATABASE_USERNAME=${{Postgres.POSTGRES_USER}}
DATABASE_PASSWORD=${{Postgres.POSTGRES_PASSWORD}}
DATABASE_NAME=${{Postgres.POSTGRES_DB}}

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOPosix.NIOConnectionError(host: "(redacted)-site.railway.internal", port: 5432, dnsAError: Optional(NIOCore.SocketAddressError.unknown(host: "(redacted)-site.railway.internal", port: 5432)), dnsAAAAError: Optional(NIOCore.SocketAddressError.unknown(host: "(redacted)-site.railway.internal", port: 5432)), connectionErrors: [])) [database-id: psql]

DATABASE_HOST=${{RAILWAY_PRIVATE_DOMAIN}}
DATABASE_PORT=${{Postgres.PGPORT}}
DATABASE_USERNAME=${{Postgres.POSTGRES_USER}}
DATABASE_PASSWORD=${{Postgres.POSTGRES_PASSWORD}}
DATABASE_NAME=${{Postgres.POSTGRES_DB}}

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOPosix.NIOConnectionError(host: "(redacted)-site.railway.internal", port: 16604, dnsAError: Optional(NIOCore.SocketAddressError.unknown(host: "(redacted)-site.railway.internal", port: 16604)), dnsAAAAError: Optional(NIOCore.SocketAddressError.unknown(host: "(redacted)-site.railway.internal", port: 16604)), connectionErrors: [])) [database-id: psql]

DATABASE_HOST=${{RAILWAY_PRIVATE_DOMAIN}}
DATABASE_PORT=${{Postgres.PGPORT}}
DATABASE_USERNAME=${{Postgres.POSTGRES_USER}}
DATABASE_PASSWORD=${{Postgres.POSTGRES_PASSWORD}}
DATABASE_NAME=${{Postgres.POSTGRES_DB}}

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOPosix.NIOConnectionError(host: "(redacted)-site.railway.internal", port: 16604, dnsAError: Optional(NIOCore.SocketAddressError.unknown(host: "(redacted)-site.railway.internal", port: 16604)), dnsAAAAError: Optional(NIOCore.SocketAddressError.unknown(host: "(redacted)-site.railway.internal", port: 16604)), connectionErrors: [])) [database-id: psql]

Effing.BombastOP•10mo ago

Here are the error details from https://github.com/apple/swift-nio/blob/f4c61cfdb7b2322935f242f0c292e6bd7e08c53f/Sources/NIOCore/SocketAddresses.swift#L57-L67:

/// Special `Error` that may be thrown if we fail to create a `SocketAddress`.
public enum SocketAddressError: Error {
    /// The host is unknown (could not be resolved).
    case unknown(host: String, port: Int)
    /// The requested `SocketAddress` is not supported.
    case unsupported
    /// The requested UDS path is too long.
    case unixDomainSocketPathTooLong
    /// Unable to parse a given IP string
    case failedToParseIPString(String)
}

/// Special `Error` that may be thrown if we fail to create a `SocketAddress`.
public enum SocketAddressError: Error {
    /// The host is unknown (could not be resolved).
    case unknown(host: String, port: Int)
    /// The requested `SocketAddress` is not supported.
    case unsupported
    /// The requested UDS path is too long.
    case unixDomainSocketPathTooLong
    /// Unable to parse a given IP string
    case failedToParseIPString(String)
}

GitHub

swift-nio/Sources/NIOCore/SocketAddresses.swift at f4c61cfdb7b23229...

Event-driven network application framework for high performance protocol servers & clients, non-blocking. - apple/swift-nio

Brody•10mo ago

you seem to be using just ${{RAILWAY_PRIVATE_DOMAIN}} without a namespace this references the apps own private domain, not the postgres private domain

Effing.BombastOP•10mo ago

With the namespace it would be the following?

DATABASE_HOST=${{Postgres.RAILWAY_PRIVATE_DOMAIN}}
DATABASE_PORT=5432
DATABASE_USERNAME=${{Postgres.POSTGRES_USER}}
DATABASE_PASSWORD=${{Postgres.POSTGRES_PASSWORD}}
DATABASE_NAME=${{Postgres.POSTGRES_DB}}

DATABASE_HOST=${{Postgres.RAILWAY_PRIVATE_DOMAIN}}
DATABASE_PORT=5432
DATABASE_USERNAME=${{Postgres.POSTGRES_USER}}
DATABASE_PASSWORD=${{Postgres.POSTGRES_PASSWORD}}
DATABASE_NAME=${{Postgres.POSTGRES_DB}}

I deployed with these vars but still got a socket address error:

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOPosix.NIOConnectionError(host: "postgres.railway.internal", port: 5432, dnsAError: Optional(NIOCore.SocketAddressError.unknown(host: "postgres.railway.internal", port: 5432)), dnsAAAAError: Optional(NIOCore.SocketAddressError.unknown(host: "postgres.railway.internal", port: 5432)), connectionErrors: [])) [database-id: psql]

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOPosix.NIOConnectionError(host: "postgres.railway.internal", port: 5432, dnsAError: Optional(NIOCore.SocketAddressError.unknown(host: "postgres.railway.internal", port: 5432)), dnsAAAAError: Optional(NIOCore.SocketAddressError.unknown(host: "postgres.railway.internal", port: 5432)), connectionErrors: [])) [database-id: psql]

I also tried with DATABASE_PORT=${{Postgres.PGPORT}} but same result.

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOPosix.NIOConnectionError(host: "postgres.railway.internal", port: 16604, dnsAError: Optional(NIOCore.SocketAddressError.unknown(host: "postgres.railway.internal", port: 16604)), dnsAAAAError: Optional(NIOCore.SocketAddressError.unknown(host: "postgres.railway.internal", port: 16604)), connectionErrors: [])) [database-id: psql]

[ ERROR ] Opening new connection for pool failed: PSQLError(code: connectionError, underlying: NIOPosix.NIOConnectionError(host: "postgres.railway.internal", port: 16604, dnsAError: Optional(NIOCore.SocketAddressError.unknown(host: "postgres.railway.internal", port: 16604)), dnsAAAAError: Optional(NIOCore.SocketAddressError.unknown(host: "postgres.railway.internal", port: 16604)), connectionErrors: [])) [database-id: psql]

Is ${{Postgres.RAILWAY_PRIVATE_DOMAIN}} correct, or is there another Postgres variable I should use? Thanks again

Brody•10mo ago

that's correct, but just to be clear, you aren't modifying any variables on the postgres service itself right?

Effing.BombastOP•10mo ago

Right, I haven't modified the Postgres service variables.

DATABASE_PRIVATE_URL=postgresql://${{PGUSER}}:${{POSTGRES_PASSWORD}}@${{RAILWAY_PRIVATE_DOMAIN}}:5432/${{PGDATABASE}}
DATABASE_URL=postgresql://${{PGUSER}}:${{POSTGRES_PASSWORD}}@${{RAILWAY_TCP_PROXY_DOMAIN}}:${{RAILWAY_TCP_PROXY_PORT}}/${{PGDATABASE}}
PGDATA=/var/lib/postgresql/data/pgdata
PGDATABASE=${{POSTGRES_DB}}
PGHOST=${{RAILWAY_TCP_PROXY_DOMAIN}}
PGPASSWORD=${{POSTGRES_PASSWORD}}
PGPORT=${{RAILWAY_TCP_PROXY_PORT}}
PGUSER=${{POSTGRES_USER}}
POSTGRES_DB=railway
POSTGRES_PASSWORD=#################
POSTGRES_USER=postgres
SSL_CERT_DAYS=820

DATABASE_PRIVATE_URL=postgresql://${{PGUSER}}:${{POSTGRES_PASSWORD}}@${{RAILWAY_PRIVATE_DOMAIN}}:5432/${{PGDATABASE}}
DATABASE_URL=postgresql://${{PGUSER}}:${{POSTGRES_PASSWORD}}@${{RAILWAY_TCP_PROXY_DOMAIN}}:${{RAILWAY_TCP_PROXY_PORT}}/${{PGDATABASE}}
PGDATA=/var/lib/postgresql/data/pgdata
PGDATABASE=${{POSTGRES_DB}}
PGHOST=${{RAILWAY_TCP_PROXY_DOMAIN}}
PGPASSWORD=${{POSTGRES_PASSWORD}}
PGPORT=${{RAILWAY_TCP_PROXY_PORT}}
PGUSER=${{POSTGRES_USER}}
POSTGRES_DB=railway
POSTGRES_PASSWORD=#################
POSTGRES_USER=postgres
SSL_CERT_DAYS=820

Brody•10mo ago

and these condition attempts are done on railway, with tls disabled?

Effing.BombastOP•10mo ago

Yes, that's correct

Brody•10mo ago

does this database client have difficulties with ipv6? the internal network is ipv6 only there's also an empty array where there should be connection errors, so there not much for me to go off of can you connect to the database locally through the public tcp proxy with software like dbgate?

Effing.BombastOP•10mo ago

Not sure about difficulties w/ ipv6. I can do some research. I was able to connect locally to the Postgres service on Railway yesterday using:

app.databases.use(DatabaseConfigurationFactory.postgres(configuration: .init(
    hostname: "roundhouse.proxy.rlwy.net",
    port: 57922,
    username: "postgres",
    password: "###########",
    database: "railway",
    tls: .disable // FIXME: re-enable
)), as: .psql)

app.databases.use(DatabaseConfigurationFactory.postgres(configuration: .init(
    hostname: "roundhouse.proxy.rlwy.net",
    port: 57922,
    username: "postgres",
    password: "###########",
    database: "railway",
    tls: .disable // FIXME: re-enable
)), as: .psql)

Brody•10mo ago

are you providing a Dockerfile?

Effing.BombastOP•10mo ago

Yes, it's the standard one when you start a new Vapor project.

Dockerfile

Brody•10mo ago

hmmm well I'm out of ideas at the moment, the error doesn't actually contain any reason for the error

Effing.BombastOP•10mo ago

Okay, I will dig into the Vapor code and see what I can discover there. Thank you again for your help.

Gaming

Programming

Vapor (Swift) Postgres connection failure