Decoupling Nuget Packages from .csproj for Docker Optimization
Aight so this is a puzzle I have been noodling on for awhile and I am curious if anyone has come up with an easy to maintain, platform agnostic solution that doesnt add additional onboarding steps for new developers.
Which means:
1. It doesnt require a powershell or bash script
2. It doesnt require installing of extra software on the machine
3. Literally just
docker build should be sufficient enough to achieve the desired result
The challenge:
Right now, nuget packages are tightly coupled to a .csproj file for projects. csproj files also handle a lot of other random stuff like file includes and build steps and whatnot.
When you try and optimize a dockerfile, you typically start things off via these 4 steps:
1. Copy just the .csproj over
2. nuget restore the project to load in nuget packages, which takes a long time usually for large projects
3. Okay now copy all the rest of the project over
4. dotnet publish to build the project
This has the upside of optimizing out the nuget packages as a "cached" first couple layers, so theoretically steps 1 and 2 only ever run if you make changes to the nuget packages...
Except... any modification to the .csproj file itself will still trigger a recache of steps 1-2 as it "dirty"s step 1, since the file's hash has changed.
And turns out, a whole lot of random stuff can cause your .csproj file to change...
So, is there a way you:
1. sanely and easily extract out all the nuget package data to its own file, that only gets changes specifically when nuget packages get changed, uninstalled, added, updated
2. Keep the rest of the project intact and a fresh git checkout of the project still can build and run as is with visual studio (if you run it, it will still be able to automatically detect missing packages and install them for local dev)
Thoughts?16 Replies
The main thing I have been looking at is the
<Import Project="..."/> element you can add to a csproj, which lets you declare a seperate "partial" csproj
Im thinking the trick would be actualling having 3 csproj files:
1. ActualProject.csproj, which is the one the .sln references, and it imports _Packages.csproj
2. ActualProject.OnlyPackages.csproj which just has the bare bones minimum requirements to be a valid csproj, and also imports _Packages.csproj, and it is not referenced by the .sln
3. _Packages.csproj which is a "partial" csproj file, that strictly has the nuget packages declared in it
And then I think you can just copy over ActualProject.OnlyPackages.csproj + _Packages.csproj into the docker image as step 1, run nuget restore, and then copy the rest as usual, and it will work the same, but now all the other stuff that can mess with a csproj will happen to ActualProject.csproj?
However for the above, the issue I forsee is... will Nuget Package Manager be smart enough to modify and add and delete packages from _Packages.csproj or will it just keep slapping them into ActualProject.csproj and muck it up, and require developer intervention everytime you wanna modify the nuget packages...?Well there’s
Directory.Build.props already built in to do some of what you’re asking about, and doesn’t require any changes to the .csproj file to support.
That said, I’m not sure why you’re talking about the csproj file being a highly active file. Most of my projects, the only time the csproj file changes is when a nuget package version changes.Content inclusions modify it, its a constant issue with web projects when you are adding stuff like images and whatnot
everytime an image file gets renamed or moved or added or deleted or etc, that mutates the .csproj content inclusions which in turn "dirty"s the docker layers.They shouldn’t, though. If you set up content tags to include directories, then changing files inside the directory don’t update the csproj file.
also any other change that has nothing to do with nuget, like modifying the project config, modifying build steps, all sorts of stuff is in .csproj
until you add a new directory
On massive large scale monolithic projects its basically a constant issue
so ideally, just breaking out the nuget packages to their own file would be ideal
I still say you need to improve your content references. You can do it at a root level and include all subdirs automatically, like the wwwroot dir is already set up.
If you want to share one, I’ll point out ways to improve.
But otherwise, yes, pulling nuget packages to the d.b.props file works automatically.
Is there a way to customize Nuget Package Manager in some way so when you use it, it will modify the d.b.props file instead? or is it still gonna modify the .csproj and you'll have to manually intervene?
Unfortunately VS behavior around the file isn’t cleaned up yet, so you have to make package upgrades manually, or using dependabot
I see, I think then my plan of having 3x .csproj files (third being a "partial" included in the 2) might work better
because I believe you can nuget package manager the partial itself directly
I think Id have to create a seperate "just packages" .sln you have to open with VS
but then when you manage nuget packages itll install them into the proper .csproj files
But I wanna say the steps would become:
1. Open
JustPackages.sln with VS
2. Manage nuget packages as usual
3. it will "just work"That sounds… awful. I think if I were a dev on that team, I’d reject it outright.
The alternative is having to manually copy it over yourself which sounds worse imo
Better than having a separate solution to open… little cut and paste never hurt anyone.
I can open two files and move a line of xml faster than I can open another copy of visual studio.
I find most junior devs to be inflicted with psychological damage the moment the open up and try and read the .csproj file, it spooks em
Then do it yourself in til you can train them.
Right, but a .sln file is already something they know how to open, and using nuget package manager as normal is also something they are comfortable with
which means zero new training/onboarding
which is a lot easier to sell to the manager, 0 is better than any amount
🤐 you do you. I have nothing further to say.