163 Replies
maybe we thread this for discussion
I'll restate in a better way than my emotional 😭 earlier.
We approved this propsal: https://github.com/ublue-os/main/issues/369
GitHub
Proposal: reduce the scope of ublue-os/main images · Issue #369 · u...
Summary Revise scope of Universal Blue Main Images such that kmods (and associated kernel packages) are no longer included. Description The purpose of this change is to streamline build processes(c...
So I want to put together a set of of PRs in main, asus, surface, bazzite, and bluefin which demonstrate this scenario
This is @EyeCantCU's first draft of
main https://github.com/ublue-os/main/pull/375
I'll probably drop a distinct PR to demonstrate my thoughts better...
I appreciate the work on this PR, but I am most concerned about future maintenance... so, seems we should have a "legacy" and "normal" path that's a little easier to follow., so I'll probably branch from this branch for my PR, because I'll build on this work.Sorry. I've been wanting to give this a proper response but we're about to leave for the next day and a half or so. If we changed the implementation a little bit (prevented tagged PR images from being used outside the org so that things proceed as normal), I think it would be awesome to do
The limitation I ran into here was that as soon as someone from an external repo opened a PR, it fell apart for a couple reasons that I'm going to have a hard time explaining here
We can jump into tty or something later tomorrow and maybe even today if I have any time
Real quick, while I can, the cosign pairs have to match and the user also needs to be a part of that repo if I'm not mistaken. Meaning an outside PR from a different repo, will refuse to push to the tag
Again sorry
Two different things being discussed here
Disregard all that ^ lmao
i think this was all meant for the other thread 😉
@bsherman figured I'd necro this since F39 is due soon (release got pushed back a little). Just want to clarify what a legacy and normal path looks like to you. Do we split the Containerfiles away, removing all legacy stuff from the new stuff, then adapt the build workflow?
appreciate the necro
the last week got really busy
i'm working on that PR now
Awesome!
I'm almost done... Containerfile/scripts are finishing testing now...
will just need to tweak the workflow
do you want me to make a distinct PR or just update yours?
Either way would work for me. Whatever is easier for you
well, i'd prefer fewer PRs for less confusion, otherwise i don't care
see what I did here if interested: https://github.com/ublue-os/main/pull/375/files
I have NOT updated workflows/build.xml yet... doing that now
I'll have a look 🙂
This is much cleaner and much more straight to the point. Less confusing as well
thank you
it is working better than I had feared, too, which is good
That's awesome. Setting up CI shouldn't be too bad
almost ready to push again
yeah, the main issue will be adding logic to the downstream repos
Dunno if you saw, but I did add some improvements to the README for akmods repo
https://github.com/ublue-os/akmods#how-its-organized
Yeah. I'm going to need to investigate more of what that's going to take. I'll have a look
well
we're going to have 2 resulting artifacts from this push... I'm getting rid of
*-nokmods and instead... all *-main images will be nokmods by default... but on F37/F38 there will also *-kmodsTook my medicine on an empty stomach and it just hit hard. One sec. Brb
so... the
*-nvidia images... to keep current behaviors the same... will need to build from *-kmods for F37/F38 but *-main for F39 and forward...
but asus and surface already build from *-nokmods so they just need to switch to *-main since those will always be "nokmods"
Bazzite and Bluefin are more complicated becasue they build from main/nvidia/asus/surface repos.This sounds great to me
ugh
i just re-read our proposal
this approach conflicts with the proposal
Ackk
adding that timeline concession really made this hard 😦
Yeah. I was hoping to knock this out earlier but I've been busy as well
ok, i'm going to think on the matrix a bit more but WITH the proposal up on my screen
Sounds good. I'll get things pulled up as well and assess what this should look like
ok, just pushed again
i THINK this will work per the proposal
Awesome. I'll have a look
hmm
i think the logic works, but maybe it all needs to be renamed to be more obvious
i was trying to be clever calling the build targets "main" and "legacy" but... maybe it should just be "nokmods" "kmods"
on the build targets
or something
i'm watching builds and thinking "does this make sense"? https://github.com/ublue-os/main/actions/runs/6498273671
I think kmods and nokmods suits it better
my GOAL with "main/legacy" was to make it clear to future selves that "legacy" is going away
or to other readers of the code
maybe instead of legacy call it "deprecated" 😄
We could document that elsewhere, but I can see how that's also helpful
deprecated makes it sound like we don't support it haha
we don't
that's the whole point
deprecated literrally means "this will go away in the future, you probably don't want to use"
i mean, not dictionary definition, but that's the idea in software
That's fair
this current build IS doing what we said we'd do, though
Looks great to me personally
so... i'm fine with renaming stuff to be more "this is what it is" and less "legacy/deprecated" and just keep comments all over WARNING people
i think it may help
I agree
The only thing I'm not excited about is the "messy" bit in ci
eg, i'll probalby make the Containerfile build targets nokmods and kmods and rename the "legacy-*" stuff in filesystem to "kmods-*"
and then in the build.xml our targets will be nokmods and kmods and that'll show in the build job list ... but i'll keep the same logic to name the images approriately
Fantastic
Then in NVIDIA, that'll need work for differentiating between 39 > and < images since the logic won't be quite the same
yep
this is the hard part about keeping F38 "kmod" images around 😦
Luckily it's only 7 more months
37 can be turned off next month entirely
wait what!
why does our "main" kmods install
kernel-devel and kernel-tools?I have no idea lol
"The kernel-devel package is important for building out-of-tree modules and has a chance of failing to install on an update if layered. Having this package in the base image will ensure that downstream images and users don't get this update issue."
so that's probably why it moved to the install script where we isolate kmods
because we explcitly CANNOT have it in a "nokmods" build as it breaks custom kernels
That's what I figure as well
I wonder if this chance of failing to update is just due to a kenrel version mismatch though and whether or not it is actually worth including
And here's kernel-tools: https://github.com/ublue-os/main/pull/208
GitHub
feat: add kernel-tools by Arcitec · Pull Request #208 · ublue-os/ma...
"This package contains the tools/ directory from the kernel source and the supporting documentation."
Helps people debug the kernel and control CPU power states, etc.
Only 290kb.
Cl...
hmm.
i wonder if kernel-tools pulls in kernel-devel
lets revisit that question later
I wouldn't be suprised
unless you want to test
I'll pull the spec up
should be a quick check in a fresh distrobox or container
I'll go look
Although the spec doesn't list it: https://src.fedoraproject.org/rpms/kernel-tools/blob/rawhide/f/kernel-tools.spec
A dep could pull it in
hah, cool
looks lsafe
i think that should be in packages.json, but not kernel-devel-matched
Yeah, just tested in a container. Doesn't get pulled in
ok, pushed the renames... and this does look like it is a bit more obvious: https://github.com/ublue-os/main/actions/runs/6498632895
for 39, there are ONLY "nokmods" builds
and the images are getting named properly
I like this a lot! Awesome work. Stuff looks great
i guess we need to update the website too
i liked just having a useful README
Yeah, the site will need an update to reflect the change. Not good with docs
If you look at anything I have on GitHub for docs, it's nothing special
Dunno if you saw, but I did add some improvements to the README for akmods repo
https://github.com/ublue-os/akmods#how-its-organized
Saw this earlier and I think it looks great
😄
so...
we can't merge this main PR until nvidia, asus, and surface also have PRs staged...
and probably bluefin/bazzite, too
Except a couple things. ASUS and Surface 37 don't exist/aren't currently built for akmods either and surface-470 doesn't exist as all Surface devices have a recent enough NVIDIA GPU to use the latest driver
or can we? since we didn't change F38, and F39 is already a "best effort pre-release"?
oh, by all means please fix the akmods workflow or README if somethings wrong there
It would be easier to work with right now knowing builds downstream succeed rather than making a shot in the dark so I think we should merge now but I'd like another opinion
Will do
yeah, this is the upside to following akdev's suggestion and NOT changing F38/F37...
it may be some headache to maintain for 7-8 months, but... it's easier to merge and make changes
Definitely
btw, I saw that I'm the one who moved kernel-devel and kernel-tools from packages.json to shell script in this
https://github.com/ublue-os/main/commit/fa5d9c4007a01502225bde6e271fc81f4351657f
so I've just pushed a related change in this current PR
i think it fits now that kmods aren't in all images
Sounds good to me. I think it fits the scope as well
nice, i'm feeling good about the PR now
not sure if i want to do nvidia or website next 😉
ooh!
Should we still be including our
ublue-os-akmods-addons RPM even on nokmods?
this includes the secure boot key and some reposHmm... I think that's a good idea. Users would be able to layer anything they feel they need and secure boot is a plus
yeah, plus it connects to the config repo's justfile config RPM
I can take a look at what NVIDIA changes we need to make if you want to look at the website
i just looked, but you can check my assumptions:
I think if we add
ublue-os-akmods-addons to even the *-nokmods images ... then nvidia should work with no changes
it just builds from mainI will confirm that though I believe you're correct
You're correct! No changes to NVIDIA. I'll grab Bazzite and Bluefin
those i expect conditional changes on, because starting in F39, they'll want to add their specific choice of kmods
I was just going to throw in all the kmods that were in main and let them be more selective after the fact if they'd like to be
yeah, that's fair...
and... another way to handle it is...
use a version condition to change if they source from nokmods or main
but always install the desired kmods
so the "hard logic" will be done... and all that has to change later is removing the version conditional
i think that meshes well with what you were doing rather
That's what I had planned. Should be a piece of cake
Oh shoot
Are any of our kmods even compatible with 39 yet?
I noticed in main we do this:
hahah
So like... That's everything
i think they should be good now
https://negativo17.org/repos/steam/fedora-39/x86_64/
at least worth trying, AFAIK , the negativo17 RPMs were the ones holding us back
I'll give it a shot
i'm assukming that openrazer and xpad-noone have an F39 builds here https://copr.fedorainfracloud.org/coprs/ublue-os/akmods/packages/
yeah, looks like it
man
i wish we could just publish binary kmods to a repo like that LOL
Would be super nice
oh, crap i forgot to push a change LOL
lol
k, i need to get back to work work
Sounds good to me :). Looks like the Xbox kmods are ready but evdi isn't quite there yet so I enabled those. Only one kmod down for 39
where? PR link?
GitHub
feat: Re-enable Xbox kmods for Fedora 39 by EyeCantCU · Pull Reques...
These were disabled for the Fedora 39 bringup as they hadn't been built for 39 quite yet, but now that they are, we can enable these
ahhhh i was thinking on a different repo... very cool
Yup. PR opened for Bazzite: https://github.com/ublue-os/bazzite/pull/421
GitHub
chore(f39): Restore kmods removed from main by EyeCantCU · Pull Req...
This restores kmods removed from Fedora 39
Now getting Bluefin
GitHub
chore(f39): Restore kmods removed from main by EyeCantCU · Pull Req...
This restores kmods removed from Fedora 39
did you test build that bazzite Containerfile?
None of our changes have been merged yet. It's a draft
Failures are unrelated
i think it is related
obs-vkcapture isn't us
but... it MIGHT work if our main PR merge is completed
I think the main PR looks great
Bluefin 39 builds!
woot
i think our merge queue needs some improving to not need full rebuilds when it's just a READM/markdown change
I agree. Kinda ridiculous
so, your PRs are merged for bluefin/bazzite... is this "completely done"?
Yep. We're good
wild
ok, cool.
what about asus/surface?
F39 stuff?
So that would depend. Do we include them because they're already hardware support images or do we exclude them like we do in main and leave it to downstream
It would make downstream kinda messy
I agree asus/surface should not be customized beyond what is required for their custom kernels...
what i mean is... do they already build
FROM FOO-nokmods - on F37/F38
FROM FOO-main -on F39
?
Ah. Let me see
PR up for ASUS. Also going to remove the kmods. Then tackle Surface. Then PR Bazzite and Bluefin again. Hoping 3 copies of akmods downstream doesn't make images huge
OH OH OH I have an idea
One COPY for akmods that somehow incorperates image_flavor
So 3 versions aren't being copied to the image
Oh wait. We only install these below 39 so no more changes other than my PR that changes 39 to main
Bazzite and Bluefin will need adjustments for Surface/ASUS though. Working on that rn too
i'm not sure what i'm following you here 🙂 but yeah, i would expect any akmods installed would be "flavor" specific
i don't get the 3 copies concern
It makes the images larger as you copy over the entire container
yeah, i've started to trim down a bit... like... here:
https://github.com/ublue-os/main/blob/main/Containerfile#L19
only copying the single rpm in that directory rather than all the kmods
This is my ugly solution: https://github.com/ublue-os/bluefin/pull/584/commits/8a1c5fcc697a335b4b01d2b7eb86c1703a1fd18c
Thhhaaaat doesn't work
nope, because once you leave the RUN you've lost the flavor variable
but you have flavor... why can't use use it directly?
because surface-nvidia
asus-nvidia
yeah, conditionals in Containerfile is hard
bluefin is already pretty complicated
i don't think it's possible to do a conditional COPY only to pass a variable to it to change where it's coming from
i have wondered if we can use skopeo in a script to pull files out of images without having to COPY them... my reason has been to avoid that layer... but it could help with this, too
So we'd need to do 3 copies to different directories, assess image flavor and assign akmods flavor like I was then install in one RUN
It would be cool if we could somehow pull that off
no. I think you are always passing image flavor
so worst case, you copy akmods-FLAVOR:VERSION and akmods-FLAVOR-nvidia:VERSION
even if you don't use nvidia
That's not what I mean :/
I'm not a good communicator at times
I'll update the code
So we'd need to do 3 copies to different directories, assess image flavor and assign akmods flavor like I was then install in one RUNok 🙂 i'm happy to read your draft PR
You don't get both parsing just image flavor, you'd get either
So like surface-nvidia, using just image flavor, you're getting the surface-nvidia akmods
Not surface
yeah, i am following
like i said, i know bluefin is already pretty complicated... but trying to do "flavor-specific" spins is tricky
why not add something to the build-args....? you already pass IMAGE_FLAVOR ... so... also pass AKMODS_FLAVOR... but just strip anything past the
- of IMAGE_FLAVOR right in the buildah action defintion
i think i'm seeing what you mean now...
you have IMAGE_FLAVOR (one of [ asus, asus-nvidia, surface, surface-nvidia, main, nvidia ] or something ) and you need to copy the correct akmods image (not the nvidia image) to install akmodsakmods_flavor isn't a bad idea but that will need a long list of exclusions. Updated the PR
Ehhh... Let's do akmods_flavor rather than what I've got... 3 copies is so redundant to bring in the same thing
yeah, you definitely don't want to pull all 3 kmods layers in
Yeah, that's what I had been saying
Will lead to a huge image
i thought I was agreeing with you 🙂
lol. Got all backwards
Updated the PR... Reaaaallly long exclude list but it works
hmm
i was thinking something a bit simpler maybe
in the
Matrix Variables step, you can do your logic based on the image_flavor and store a env.AKMODS_FLAVOR and use that in your build-args later
the matrix DOES work, but i think a little bash conditional might be simplerOkay. I can get on that here in a little bit
i think you basically had the logic in an earlier commit, but it was in the Containerfile
Yeah. Just need that and to pass it to the environment
I'm not at my computer rn
But will be soon
All right, should be gtg
that looks good to me... feel better to you?
For sure. A whole lot better
ok, so i'm testing 🙂 because all the builds failed 😄
I fixed it @bsherman
I put quotes around the akmod flavors
ahhh
why is there only a single commit?
oh, you force pushed
yeah, so looks like there's some things needed for F39 to work
probalby because "nokmods" are being installed
I wonder... I think we may need another change to the
main repo... to install ublue-os-akmods-addons RPM even when not installing actual akmods
oh, that IS what we are installingYup. We left that earlier
I wonder what's going on
Given that framework and main succeed as well as vanilla Nvidia, something in either of those repos needs work
ASUS and Surface I mean
sed: can't read /etc/yum.repos.d/_copr_ublue-os-akmods.repo: No such file or directory
i think this is why adhoc adding/deleting repo files is a problem 🙂I'll take a look and see where things don't align
i'm pulling the asus 38/39 images to inspect locally
I see what's going on already
39 doesn't have ublue-os-akmods-addons
Because we left it, ASUS and Surface try to reinstall it
Fixing now
ah, so their 39 images are failing?
Yeah
Addressing it now
i KNEW we weren'te QUITE done yet 😉
thanks for sticking with me on this "simplify main" quest
not to mention this PR didn't get merged 😄
GitHub
fix(f39): Build from main instead of nokmods by EyeCantCU · Pull Re...
nokmods is deprecated from Fedora 39 onward
Yeah, just came to mind
And no problem 🙂
so fix it in that PR and we'll get it merged?
Yup. Working on it right now. Also looks like I need to fix kernel replacement since kernel-devel is no longer there
Should tie it up
that last commit for kerel-devel is bad
you removed a required
&& \God damn it
Lol
extra 👀
Fixed
YES FINALLY
Lol
🎉
ready to merge? cuz i pushed the button LOL
queued
btw, would you please approve this? https://github.com/ublue-os/nvidia/pull/162
GitHub
feat: enable ublue-nvctk-cdi.service by default by bsherman · Pull ...
We added this service to generate nvidia device definitions for container toolkit, but it isn't enabling by default like it should.
This corrects that.
Yep!
i gotta crash for tonight, tomorrow will be hit or miss, but i'm happy to review surface stuff etc... to get this wrapped up
Sounds good, we've covered a lot of ground today. This is the Surface PR: https://github.com/ublue-os/surface/pull/18
And here's the surface-nvidia PR: https://github.com/ublue-os/surface-nvidia/pull/5
The second PR won't build until the first has been merged, but other than that, we're good :). Don't worry about reviewing until you have the time. I'm going to be wrapping today up myself here soon
looks like you got these merged! sorry, i was out of touch yesterday
Yup! It's cool. 39 images for Bazzite now too. I just need to throw the gts tag stuff in ASUS and Surface at this point but other than that we're gooe