mongoDB upgrade issue
The total data volume of mongoDB is about 50G. which is stuck at Migrate Data. Then mongoDB crashed The error message is as follows:
103 Replies
Project ID:
f522f23f-32d0-4d1e-b301-2f24ffd4d85care you on the pro plan by chance?
yes
Railway
Railway
Railway is an infrastructure platform where you can provision infrastructure, develop with that infrastructure locally, and then deploy to the cloud.
projectID: f522f23f-32d0-4d1e-b301-2f24ffd4d85c
what do the logs of the migration service say?
would this be your thread too? https://community.railway.app/questions/mongo-db-upgrade-issue-bf12de3a
also my thread
the log from mongoDB Migration
what do the volume usage metrics look like the new mongodb?
id like to see the volume metrics please
How to view volume metrics
click on the service's volume
where
that explains it
will have to flag the team
just a quick question, how much data would you say you have in your legacy mongo db?
More than 50G
youre right about that but would you happen to have a more accurate number?
perfect, thank you
Thread has been flagged to Railway team by @Brody.
when the team is available they will bump your volume limits so that you may run the migration again
Can it be as soon as possible? Our project is in a hurry and is currently suspended.
in the mean time feel free to remove the new crashed mongo service, the crashed migration service, and rename the database back to
MongoDB
delete MongoDB?
once you do the above, you can update your services reference variables so that your service connects back to the legacy database, make absolute sure you are using reference variables https://docs.railway.app/guides/variables#referencing-another-services-variable
correct
may i ask why you have two v2 redis databases with one being named
Redis Legacy?migration error then we click re migrate
then got two v2 redis db
what one is in use by your app
We have not modified the redis URL, it seems to be modified automatically.
Currently using redis legacy
okay then go ahead and delete the v2 redis database that is not redis legacy
*
correct as long as your app is using the v2
Redis Legacy redis databaseok
what to do next
delete this stuff
and a screenshot when thats done please
done
rename this to just
Redis since its not a legacy database, its a v2 now
done
rename this to just
MongoDB even though this is indeed a legacy database we dont want to end up with a v2 mongo database named MongoDB Legacy like you had with redis
done for rename
now update your variable references on your app for mongo
Program services still use this mongoDB
are you using variable references?
The current reference to the program service
right but thats not quite what i meant, have a look at this and let me know if this is the style of variable you are using in your service variables
https://docs.railway.app/guides/variables#referencing-another-services-variable
yes
perfect, can you confirm your app is back online and working as it should?
yes
Do I still need to upgrade again?
yes you do, but not until a team member confirms that they have bumped your volume limit.
right now i have you using the legacy database until the team confirms that.
you can also delete that
Redis Legacy databaseWhen can I upgrade
not until a team member confirms that they have bumped your volume limit
Can you give me a specific time?
as i dont work for railway i cant give any exact timeframe, but may i ask what the rush is now that your app is back online?
If I don’t upgrade on January 31st, will I be forced to upgrade?
thats a ways away, the team may not respond today but they will absolutely respond before you have to migrate
ok, thanks~
but to directly answer your question: yes
cc @thomas, for his todos
Always like a tag, thanks Angelo
I am still catching up @albertz8912 but do you have an idea as to the size you need?
Brody mentioned 100Gb
I think 100gb is a sufficient value?
That seems reasonable. I am in prd-job4 as the env because it was the default, but there are a lot of environements
maybe just an account wide volume limit bump?
It's a per volume change
yeah, we have account limits we can bumb but this isn't one of them
because for the really big volumns we have to check the host
Given the time this user was online, I suspect they are offline. I am going to wait until the end of my day then grow my best guess
Hello, I was wrong about what to do here.
There is a change to the migration process going into place for your team. I will advice when it is safe to migrate again.
Hey @thomas_railway_75694, remember to grow this volume - https://discord.com/channels/713503345364697088/1199980272649768981/1200160499480928357
@albertz8912
You are good to try migrating again, please let me know if you have any issues.
ok thanks
Can I migrate again?
yes
Is there anything I can do to help? I will check in on your project later today.
done for production
but there got a error in mongoDB migrate
by error do you mean the action required message?
yes
Is this a bug?
Or we can ignore it
it's looking like just a display bug, but config for me that your reference variables are referencing the database named
MongoDB and not MongoDB Legacyalready done for referencing
nothing is connecting to the legacy database?
yes
and your app is working fine, across all environments?
Still testing
I think the data in the database should be fine
okay let me know if you run into problems!
in the mean time, @thomas would you have any clue why they are still seeing the migration message? that would cause a good amount of confusion if it's not a one off bug
I will look into that
it's not meant to look that way
seeing if I can figure out why it does
one of them changed status to the correct one and I don't see anything wrong with the Redis one, so giving it some time and then asking the team
!remind me to look at this again in 4 hours
Got it, I will remind you to
look at this again at Fri, 26 Jan 2024 22:45:53 GMT
Hey @thomas, remember to look at this again - https://discord.com/channels/713503345364697088/1199980272649768981/1200511948480057505I am sorry I was not able to figure out why that banner is still there.
You migrated this to the new Redis instance, correct?
@jr when you get back online could you take a look at why this service still has a migration banner after migrating?
I double-checked, and no services are hard linked to it.
cannot view the service logs, always loading
are you the owner of that project?
Checking back here, everything looks good? I can see both plugins are migrated and all services are up and running
what version of MongoDB railways have upgraded to?
mongo:latest docker image?
correct