NocoDB: deployment crash, migration failure, current transaction is aborted
Hi! I'm suffering from a migration failure and crashed NocoDb deployment on Railway.
It says that current transaction is aborted, commands ignored until end of transaction block.
Project ID: 611ca5b0-4e20-46b7-bbc8-993d9ad7024d
41 Replies
Project ID:
611ca5b0-4e20-46b7-bbc8-993d9ad7024d
Here are the Deploy logs:
did the migration itself complete?
I found these other errors too
How can I verify this?
did the migration modal tell you the migration went off without issues?
I'm a bit of an amateur here: where can I find the migration modal?
you would have opened it when you went to migrate your database
Ooooooh, yes. I migrated it last year after the email announcement was sent, and it migrated successfully.
is noco still using the legacy database?
I've checked the variables, and they use the db credentials for the new migrated service
have you confirned all data is in place in the new database?
Yes, just checked again now
what do you have for your raw NC_DB variable? if it contains sensitive information then that means its wrong
Which one is raw NC_DB?
would you be interesting in deploying a new nocodb? your deployment is very old
I'm not opposed to it. How can I deploy a new one while maintaining my db data?
you would first need to deploy a new nocodb template, then deploy a migration template to copy the data from the current postgres to the new postgres database
if thats something you want to do, i could write up a clear guide
That would be wonderful, thank you 🙏🏿
okay i will get to that when i have some time! would you like a ping when thats ready?
Ouf, I see what you mean now by my deployment being very old... The dockerfile from Railway isn't even the same as the one I have in my repo...
Do you think it would be enough to just edit my dockerfile to match it to the new one? If not, a ping would be perfect.
I think you are using variables that nocodb has since renamed, so a migration to a new noco deploy would be best.
will ping with guide once I've done that!
@canxmedia - didn’t expect it to be such a long guide, take it slow and if you need any clarification, please don’t hesitate to ask
hit the character count for my message, here is a gist instead https://gist.github.com/brody192/7689605241b25100135753029fc9484a
Thank you so much, Brody! I'm currently at the step
Monitor the logs of the postgres migration service
The deployment crashed, and was looping through the following error:
looks like you either are using the
DATABASE_URL
variable from the new postgres database as the NEW_URL
variable for the migration, or you got the new and old variables mixed up
now remember, railway is not responsible for any data loss that may ocourYou're right. I used DATABASE_URL for both old and new. Let me try again
Should I delete the Postgres Plugin Migration, or just edit the variables?
you can just edit the variables
Okay, I just want to make sure I understand the following passage from your guide:
Copy the DATABASE_URL variable from the Postgres database in your old NocoDB project into the note. making sure denote that it is the old database's URL Copy the DATABASE_PRIVATE_URL variable from the Postgres database in your new NocoDB project into the note. making sure denote that it is the old database's URLI wasn't sure if the last sentence was a typo, so I noted down:
OLD DATABASE_URL: A OLD DATABASE_PRIVATE_URL: B NEW DATABASE_URL: C NEW DATABASE_PRIVATE_URL: DAm I correct to place the following values? :
PLUGIN_URL: A NEW_URL: D
100% a typo, my bad (i've fixed the gist)
old
DATABASE_URL
-> PLUGIN_URL
new DATABASE_PRIVATE_URL
-> NEW_URL
Awesome, so far so good...
Well.... Good news: the db migrated once I put the right credentials.
Bad news: the deployment failed with a similar issue. I think there's something wrong inside the database's data, but I just don't know what...
unfortunately theres not too much i can do if the data in the database is malformed
might just have to use a fresh nocodb
Well, I really appreciate the guide and all the help! The updated dockerfile was definitely something I would've missed if it wasn't for your help, and I'm sure the guide will help others.
I'm going to try flagging down a NocoDB dev to see if the data error can be fixed, and report back in case someone else has the same error.
i wish you the best luck!
thank you for the trains!
So, update on this issue: I've been troubleshooting with NocoDB for the last little bit, and here's what was said so far:
I was asked to run
select *
from pg_indexes
The NocoDB dev looked over the results and said, "Somehow index list query returns empty list even if there are index and which is causing this issue."
I was then asked to do the following:
Please run the following query as well .... replace schema_name with your schema name used in connection. And you should connect to the database you are using as well.My question is: how do I find the schema name to replace and run this query?
I tried to look it up online and ran the following two queries, but got a bunch of results (I'm using Dbgate):
and
I'm sorry but I have no clue here, I have never done any database management ever
only migrations
No problem! I'll do some more digging and troubleshooting with them, and report the solution here.
sounds good, I again wish you luck!
Good news: the issue has been resolved. I didn't need to run that other query after all, just needed to deploy the following image provided by a NocoDB dev, which skips
transaction
so the deployment works as expected:
nocodb/nocodb-timely:0.204.0-pr-7425-20240113-1241
Then, once that deployed successfully, I just needed to switch back to nocodb:latest
.
Thank you again for your support, Brody! I've been dealing with this issue for months, and I'm sure the fresh deploy also made a difference, somehow. I'll probably refer to that guide in the future, and hopefully it helps someone else.oh wow i would not have ever come to that solution in a million years, very happy you where able to get it working!