Railway•8mo ago

PGLoader Connection Error

I'm using pgloader to migrate data from MySQL to Postgres. Pgloader connects to the target database using TCP/Sockets and performs the migration tasks in parallel. Everything runs smoothly until I attempt to migrate to a Railway Postgres database, at which point I encounter an error.

Failed to connect to pgsql at "RAILWAY_PROXY" (port RAILWAY_PORT) as user "postgres": Database error: Socket error in "connect": EINTR (Interrupted system call)

Failed to connect to pgsql at "RAILWAY_PROXY" (port RAILWAY_PORT) as user "postgres": Database error: Socket error in "connect": EINTR (Interrupted system call)

The migration process using pgloader initially proceeds without issues, but after a certain period, an error is thrown as described above. I suspect there might be an underlying issue such as a firewall restriction or similar configuration that I may have overlooked.

21 Replies

Percy•8mo ago

Project ID: 56ecf4a7-da41-4ba0-a70c-7708dac4054f

Henrique Antonio•8mo ago

56ecf4a7-da41-4ba0-a70c-7708dac4054f

Brody•8mo ago

may i ask if this is planetscale to postgres on railway?

Henrique Antonio•8mo ago

Yes!

Brody•8mo ago

would you happen to know how much data you have in planetscale?

Henrique Antonio•8mo ago

Not much, less than 1GB of data.

Brody•8mo ago

any reason you arent going to a mysql database on railway?

Henrique Antonio•8mo ago

Internal needs, my team it's more familiar with Postgres now. Is this a problem?

Brody•8mo ago

nope just curious how long does the connection stay alive before it dies like that?

Henrique Antonio•8mo ago

Max 3 minutes running

Brody•8mo ago

does any data make it into the postgres database?

Henrique Antonio•8mo ago

Yep It's working, it seems to stop for a non particular reason

Brody•8mo ago

what pgloader command are you running?

Henrique Antonio•8mo ago

LOAD database
     from mysql://root@localhost:3306/DATABASE
     into postgresql://postgres:PWD@HOST/railway

 WITH include drop, create tables, create indexes, reset sequences, snake_case identifiers,
      workers = 16, concurrency = 4,
      multiple readers per thread, rows per range = 50000;

LOAD database
     from mysql://root@localhost:3306/DATABASE
     into postgresql://postgres:PWD@HOST/railway

 WITH include drop, create tables, create indexes, reset sequences, snake_case identifiers,
      workers = 16, concurrency = 4,
      multiple readers per thread, rows per range = 50000;

I tested with 1 worker, 1 concurrency, 5k chunk. Same error... :/

Brody•8mo ago

if you had just deployed a postgres database on railway, it would be postgres 16, is pgloader fully compatible with that version?

Henrique Antonio•8mo ago

Yep, just checked. I've made once without errors when testing, but it's inconsistent

Brody•8mo ago

pgloader that was released in 2022 is compatible with postgres 16 that was released in 2023?

Henrique Antonio•8mo ago

Just worked again. I plan to keep running the process until I have the final version of my data. It seems to lost the connection when processing sometimes and pgloader does not have a mechanism to reconnect automatically.

Brody•8mo ago

i think this is just instabilities with pgloader, no one else has reported any issues with their database connections in a good while

Henrique Antonio•8mo ago

I'm going with this. Much appreciate the help anyway!

Brody•8mo ago

sorry i couldnt get you a better solution, i just dont have any way of testing this for myself

Gaming

Programming

PGLoader Connection Error