Railway•9mo ago

PGLoader Connection Error

I'm using pgloader to migrate data from MySQL to Postgres. Pgloader connects to the target database using TCP/Sockets and performs the migration tasks in parallel. Everything runs smoothly until I attempt to migrate to a Railway Postgres database, at which point I encounter an error.

Failed to connect to pgsql at "RAILWAY_PROXY" (port RAILWAY_PORT) as user "postgres": Database error: Socket error in "connect": EINTR (Interrupted system call)

Failed to connect to pgsql at "RAILWAY_PROXY" (port RAILWAY_PORT) as user "postgres": Database error: Socket error in "connect": EINTR (Interrupted system call)

The migration process using pgloader initially proceeds without issues, but after a certain period, an error is thrown as described above. I suspect there might be an underlying issue such as a firewall restriction or similar configuration that I may have overlooked.

21 Replies

Percy•9mo ago

Project ID: 56ecf4a7-da41-4ba0-a70c-7708dac4054f

Henrique AntonioOP•9mo ago

56ecf4a7-da41-4ba0-a70c-7708dac4054f

Brody•9mo ago

may i ask if this is planetscale to postgres on railway?

Henrique AntonioOP•9mo ago

Yes!

Brody•9mo ago

would you happen to know how much data you have in planetscale?

Henrique AntonioOP•9mo ago

Not much, less than 1GB of data.

Brody•9mo ago

any reason you arent going to a mysql database on railway?

Henrique AntonioOP•9mo ago

Internal needs, my team it's more familiar with Postgres now. Is this a problem?

Brody•9mo ago

nope just curious how long does the connection stay alive before it dies like that?

Henrique AntonioOP•9mo ago

Max 3 minutes running

Brody•9mo ago

does any data make it into the postgres database?

Henrique AntonioOP•9mo ago

Yep It's working, it seems to stop for a non particular reason

Brody•9mo ago

what pgloader command are you running?

Henrique AntonioOP•9mo ago

LOAD database
     from mysql://root@localhost:3306/DATABASE
     into postgresql://postgres:PWD@HOST/railway

 WITH include drop, create tables, create indexes, reset sequences, snake_case identifiers,
      workers = 16, concurrency = 4,
      multiple readers per thread, rows per range = 50000;

LOAD database
     from mysql://root@localhost:3306/DATABASE
     into postgresql://postgres:PWD@HOST/railway

 WITH include drop, create tables, create indexes, reset sequences, snake_case identifiers,
      workers = 16, concurrency = 4,
      multiple readers per thread, rows per range = 50000;

I tested with 1 worker, 1 concurrency, 5k chunk. Same error... :/

Brody•9mo ago

if you had just deployed a postgres database on railway, it would be postgres 16, is pgloader fully compatible with that version?

Henrique AntonioOP•9mo ago

Yep, just checked. I've made once without errors when testing, but it's inconsistent

Brody•9mo ago

pgloader that was released in 2022 is compatible with postgres 16 that was released in 2023?

Henrique AntonioOP•9mo ago

Just worked again. I plan to keep running the process until I have the final version of my data. It seems to lost the connection when processing sometimes and pgloader does not have a mechanism to reconnect automatically.

Brody•9mo ago

i think this is just instabilities with pgloader, no one else has reported any issues with their database connections in a good while

Henrique AntonioOP•9mo ago

I'm going with this. Much appreciate the help anyway!

Brody•9mo ago

sorry i couldnt get you a better solution, i just dont have any way of testing this for myself

Gaming

Programming

PGLoader Connection Error