How does EFCore Migrations work for actual deployment?
So I understand that when locally developing I can do
and that I can even start the initial migration on startup via
ApplicationDbContext.Database.Migrate but this doesn't seem encouraged in production.
but when say deploying my app as a docker container, how do I exactly run those same commands? Do I do them as part of my dockerfile or dockercompose or something?118 Replies
this seems to be a possible way https://codebuckets.com/2020/08/14/applying-entity-framework-migrations-to-a-docker-container/
Tim Brown
Code Buckets
Applying Entity Framework Migrations to a Docker Container - Code B...
I'm going to run through how to deploy an API and a database into two separate Docker containers then apply Entity Framework migrations. This will create and
it depends on your app, your risk tolerance etc
so like... whats the project
I have an API, Blazor WASM, and .NET Database project. I plan on deploying the project solution in docker along with a PostgreSQL Db
You should configure a github workflow and run your dotnet commands from there
ok I can look into that
will that work even if I'm using Oracle Cloud to host the docker containers?
I dont see why not, the github workflow doesn't care about where you host your docker containers
I would put the dotnet ef database update here
Don't do that
Do the migration in the app, in a migration bundle, or in a SQL script
O right i was like hmm why not
but ofc the state of migrations i didnt think about that š
Applying Migrations - EF Core
Strategies for applying schema migrations to production and development databases using Entity Framework Core
Atm strategy at Givt == execute migration by dev on local machine ^^
In general, i wouldn't put that in the pipeline anyway. But that's my 2 cents and personal preference lol
this is certainly no where near ideal
an idempotent sql script is the preferred way to handle proudction deployments
with gates and checks etc
if its a suitably small/nimble enough app the api itslef can apply them
hence why i was asking "what kind of app is this" š
So lemme get this straight, you as a developer, make changes to the dbcontext, create a migration, your colleague reviews it, then he gives a go, then migration is deployed to the pipeline, you would then validate the migration again?
Yes you verify that the GENERATED sql is what you'd expect then that's approved to be applied
The runner connects to the database and applies the migration
Well i have verified the generated SQL even before i made a PR with it
so i see no reason why, after PR is approved, migrations could be applied from pipeline
you generate an idempotent script?
locally?
Idempotent is the script with the before and after changes ?
and if you do how do you verify that what your pipelines generate is the same thing
no
its a SQL script that contains a complete migration history for the database
it selectively applies what it detects that it needs
its pure sql
Thats in our DB ?
wut
yes, the sql script reads the migration history table and apples the corresponding sql
but its not the sql hah
at no point in time is c# involved here
i generally do: dotnet ef migrations add "blablabla"
dotnet ef database update
and its done. No sql is generated, only a c# class with data to generate sql
yes i know, thats not a great production strategy
and it CERTAINLY would never fly in any large scale operation
How so, whats so wrong bout it?
the migration generation is a very seperate process from the sql generation and application to the production database
well... its not automated for 1 lol
Ci/cd is a big deal, all deployments should be automated
devs where i work dont even have ACCESS to production
full stop
its all pipelined
lmao
at my place
i mean at Givt, we do everythinig
next is of course correctness, you dont KNOW what sql it will generate
every dev has full access
you know what the current version of ur ef tooling will generate
thats why we probably like to restrain from putting everything in pipeline
but you dont know the actual sql
thats why we probably like to restrain from putting everything in pipelinewrong just... so wrong good lord lol automate ur deployments or at the very least, if you ARENT doing that dont give advice about it in here its dangerous an idempotent sql script is a locked version of what changes will be made to your database that you can review the pipelines generate it and thats what is reviewed
So that's tight as a idk, but what about other things, certificates, secrets... i mean few days ago on discord, some dude was having problems, his boss didnt even let him on the AWS console.... we dont work like that at Givt... you join the company, your full access to everything, even financials and our bank accounts
yeesh, well my company has 100k employees lol
I understand your point of view, completly, but in the stage we are in with the company, we have no time for these kind of luxuries.
we only have 17 š
which only 5 are devsv
thats fine, im not gonna shove u one way or another, but dont offer advice on deployment strategy best practices when you are following absolutely none of them yourselves
š ok mr grumpy š remember, you have to let people fail so they make mistakes so they can learn and become better through it. Otherways you will just have a heard of sheep
you have to let people fail so they make mistakesthis doesnt mean you tell people wrong info lol
I told him a perfectly working solution
and saying "run dotnet commands in your pipeline" is not great advice at all
š¤¦
good lord
that its not the best, sure but wrong hmmm
What is wrong
why is it worng
Who values what is wrong vs what isn't wrong
because you are running development commands from a production pipeline lol
And that's an issue why?
if the command is fine
production is just like any other stage, just a different name š
you dont KNOW the command is fine lol, you are assumign it is. but you are running commands against your production database that you arent sure what they are going to do
hence why the generated script is the expected solution
the ef docs LITERALLY tell you not to use the cli for deployments
the are commands for testing and development, not deployments
ISNT IDEAL
they dont say we cant do it š
dont recommend approaches that the msdocs say are not ideal, how about that
its not fair to the rest of us to have to come back through and be like "actually dont do this..." etc
Why is it fair to the ones that come behind you that they cannot challenge that
challenging the idea of "dont run dev commands on a production database" doesnt hold much water
But what makes it reunning dev commands on production different from running dev commands on development? the only reason i can think off, is when the migration is converted to sql that by this time a change in the ef core package changes the outcome of the sql script
But that could be resolved by setting a fixed package version...
and if you pulled back a bad version, or something had changed in ur migration after review etc
schema database changes are incredibly dangerous, they need to be handled carefully
yoloing dotnet commands is not recommended for a reason
Ofcourse, thats why there are PR's, versioin numbers, and why we update our production databases from our dev machines....
See all the hassle, that comes with making a process air tight, it requires checks, double checks, and triple checks. Its 7 years since Givt started, and we have only recently started using PR's because we hired a devops specialized freelancer for the move into kubernetes. But i get it lol, we are processing about 1mill worth of donations a month. When something happens with the database or when enquerying about it with the founder, he's like aaaah we got backups" š
and we have only recently started using PR'saaaaaand i rest my case
š
Btw
wna have a real laugh
you can do your own pr's š
so lets not give advice to others about devops stuff lol
i dont really care what you do at your company, i very much do care what you tell people in here
Well, im sorry for trying to help lol. I now know you are the smartest person alive and you never make mistakes and assumptions š
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
the self contained one?
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
ya the self contained way is new, helps with runners that need to go from cloud to on prem
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
you have a simple app that literally just calls migrate?
why?
i thought you were talking about bundles
Applying Migrations - EF Core
Strategies for applying schema migrations to production and development databases using Entity Framework Core
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
err
š
is this a patrick thing?
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
ya im considering it
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
thats so pointless tho
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
well u have an entire docker container for something that you can apply from your runners cli as a script
or just call in ur apis startup?
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
if ur fine with the risks
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
or honestly, even mikes way is better then an entire container to call migrate
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
wut
why would scaling out matter
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
not fully how that should happen but yes point taken, if you are concerned abotu safety and you have an application that is large enough to need scaling like that you should be generating a script
and applying that
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
its more effort to have an entirely new dockerized project complete with secrets etc then it is to call a few cli commands?
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
like even just the mike way would be better then this lol
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
so how do you connect to the database?
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
^
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
so a secret?
how does connection string go from secret store -> pipeline -> docker container -> migrate app
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
so an env file gotcha
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
i mean, i use env files for my personal stuff
at work we dynamically link to kv with cycling expiration dates that are refreshed from some tool called cyberark
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
i mean ya its a ton of work, but security lol
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
devs cant read the secrets in our pipelines
not even i can read some of them and i set that shit up
Unknown Userā¢3y ago
Message Not Public
Sign In & Join Server To View
Trust, is very important in the world ppl š
This is going to bite you in the ass
They're going to learn the value of access controls the hard way
How so
Because now you can compromise the whole company by just compromising one person or their machine
That can always happen, no matter how tight you screw the valve. its never 100% tight. We choose to trust in eacother that we handle sensitive data with care, and idk for what its worth, our macbooks are encrypted š
Saturday Morning Breakfast Cereal - 2012-02-20
Saturday Morning Breakfast Cereal - 2012-02-20
I could tell you things, that would make the hair on your ass get up š
You mean this entire thread?
-_-
Dont blame me for following company policies š Cheer for me cus trying to change them š
Well the policies are braindead
I just want you to know that
So you don't recommend or replicate them on your own
Ofcourse the policies are braindead
Because they aint really actually there
its all spoken policies
no hierarchy
no titles but only roles
Its a mindfuck when coming from the usual backgrounds where there is always somebody with a higher title
ok so after reading all of this my understanding is that I should look into Migration Scripts?
Yes sir