Running migrations on startup
First of all, hey there, I hope you are doing well.
My question as a new next js space/ecosystem is revolved around db migrations.
I'm building an app that aims to be easily self-hostable, but currently, I'm unsure what the best approach is when it comes to applying and continuously rolling out new migrations. I would imagine that running some code before server next js server start is what I am looking for.
Currently, my stack uses the default t3 stack, the only difference is that I use drizzle and turborepo.
Any kind of suggestion/recommendation/pointer to a resource is greatly appreciated.
Thank you for your time.
36 Replies
Depends on platform you want to make this hostable? Probably not good idea to run migration in server start.
How would one containerize a next js app that talks to a db?
It the dB is brand new out of the box , don't we need to run some kind migrations to get the schema we expect
I think running migrations at startup is ok as long as you backup your data regularly.
Why don't you run your migrations when building the container image?
that could potentially break the currently running instance.
If that's a requirement, then I'd suggest the container's entrypoint or a separate migration container.
I've seen projects usually ask you to run separate migration commands to handle this if self hosting. Unless data is properly backed up you should never do this. Especially if a person running an update doesn't know, you might just warn the user to always backup data before updating but people sometimes ignore this. So second step to migrate is explicit reminder. I usually have every thing as service in docker
All of my logic comes from observing other apps and how they work. For example, netdata and nextcloud expect a database they can talk to in order to log their different data. However, the database usually on startup is empty, so how exactly do they go about running migrations then? Well, as anyone could guess, I imaged that they ran migrations on container startup before the app connects.
I haven't checked their source code to confirm if that is the case, but this approach it makes the most sense to me.
Don't mistake practicality for a wide range of non-technical users for how things should be done.
How would you do it? How would you ship a nextjs app that needs to talk to a mysql for example, and the docker compose file has a mysql docker image and your app image in it
How would you go about making sure the mysql db that your app is using has the expected schema
By telling people in the upgrade docs to make a backup and then run the migration command inside the app container when needed.
That seems like too much friction
Then I'd suggest a separate migration service that runs once on startup to create a backup and migrate all changes.
I suppose that could work
That way you could scale your app container without scaling the migration task.
If that's not a requirement you can also just throw it into the app container's entrypoint.
Here is what i was thinking
I could have the migration logic inside of this https://nextjs.org/docs/pages/building-your-application/optimizing/instrumentation
Optimizing: Instrumentation
Learn how to use instrumentation to run code at server startup in your Next.js app
And that would run whenever the app container starts up
and since it's simple migration logic, it should be fine to run them even 1000 times over
Thoughts?
Another option is to do it the way WordPress does it: When you update the app version and the database needs to be migrated: Show a maintenance screen until an admin logged in and clicked the button to run the migration script.
I don't like this approach because it adds friction for the user. My goal here would be to ensure seamless user experience, while also trying to stay true to good practices and safety standards
That's a possibility though I prefer Docker's entrypoint mainly because you usually just need to run commands like Prisma's migration commands.
I'm not using prisma but drizzle, and i would prefer to strip down the main nextjs image and not include any dependencies
Seems simpler to call commands in a .sh than in a .ts
A Docker entrypoint file is not a dependency. It's just the command that the container calls on startup. In that command you then create a backup, migrate, delete obsolete backups and then run Next.js.
No i wasn't refering to that
All Dockerfiles have either a CMD or ENTRYPOINT
I was talking about how i would like not to have any node_modules when the container starts up
Yes, i'm aware, and i think i didn't express myself correctly
Hmm, why not? Dependencies such as Next.js' dependencies should be installed during the build process. Otherwise starting the container would take forever.
The plan is to build the app in a standalone mode, and then only ship the build directory in the docker container, no node_modules
So you don't want to ship the source code the way you wrote it? What's the harm in shipping .build + node_modules?
The image size gets needlessly large (we are talking about ~500mb + just for node_module)
While if i only dockerize the build directory (the output of the standalone mode), it comes way way down
A few hundred MBs for a docker image is pretty common.
But if it can be shaved off, why not?
Because we want fast startup times. Imagine you want to scale your app up and down. Do you want the scaling to take minutes?
I do not think i need to wory about scale, the main goal of this app is to be self-hostable
No user will need to scale this app, i'm 99.99% sure of that.
Then it doesn't matter, I suppose. But that's usually not the way it's done. The motto is: Build once, use often. That's why we put as much as we can into the build process to make using the image easier.
I think, right now i will try to make it work with the instrumentation feature of nextjs
if that doesn't work, i will compile a script with tsup that is ran before the nextjs app is started