What options are there to perform data migrations?
Hey, thanks for helping me out :)
What options do I have to handle data migrations? For example when renaming a column via planetscale, I create a new column and need to backfill the column with data of the old column. Planetscale handles the schema migration (create the new column), but not the data migration (backfilling the data), so what's the best way to handle this (using script files, direct terminal operations, or another method)?
Also what process or tool does Theo recommends for handling data migrations?
Thanks again.
9 Replies
Can't you just make your tool make the migration file, then edit it to rename instead of delete column + make new column.
Thanks for answering. Renaming columns is not allowed by planetscale, because this would produce down time. See here: https://www.youtube.com/watch?v=jeWrbAiA1D0&pp=ygUPdGhlbyBtaWdyYXRpb25z
It's allowed with deploy requests
Handling table and column renames
How to manage table and column renames with online DDL and no downtime
I am not sure what exactly you are referring to. The link that you sent seems to suggest to create a new column and then remove the old one:
That is exactly what I am trying to do :)
Deploy requests I am also using myself already
Backfill is not some feature, kinda just have to write your own little script
Backward compatible database changes — PlanetScale
Learn about safely using the expand, migrate, and contract pattern to make database schema changes without downtime and data loss.
Yeah, that is what my questions was about, I updated it to make it more clear. I wanna know whether there should be some script files (and where to put them), or rather direct terminal operations, or another method. For updating the schema there is no need to create script or migration files, one just changes the schema and calls
push, then the schema changes get automatically updated and pushed to planetscale. But then the data is not synced with the schema, it needs to be backfilled, etc. I am wondering how to proceed on from there, maybe create script files that migrates the data as you said (and where should these files be stored, in version control? Seems odd if data migrations scripts would be stored there but not the schema migration scripts), maybe do it through some other means