UB
Universal Blueโ€ข8mo ago
j0rge

CoreOS kernels

CoreOS kernels thread
73 Replies
j0rge
j0rgeOPโ€ข8mo ago
@bsherman the reason I ask is I asked timothee about Fedora but on coreOS streams instead of releases but no one's working on that. I was thinking of experimenting with running CoreOS kernels since they're less aggressive
bsherman
bshermanโ€ข8mo ago
makes sense
j0rge
j0rgeOPโ€ข8mo ago
so then I was like "maybe we don't need to rebuild the whole thing", I wonder if you can just use the coreOS kernel on fedora silverblue
bsherman
bshermanโ€ข8mo ago
yeah, it's the same kernels, it's just that the release cadence is different, and they have a manifest for building their base images with specific pacakges pinned and since all updates are rpm-ostree based, there's no "oops, I did dnf update and got a newer kernel"
j0rge
j0rgeOPโ€ข8mo ago
yeah, it appealed to me on a quality level like the extra stuff they test for
bsherman
bshermanโ€ข8mo ago
i mean, i very much think that the coreOS release model makes more sense long term... but... it would be hard to do since it's also built around the F39/F40 releases with manual delay on the only the specific pacakges deemed important for the CoreOS use case hard for us i guess or... not TOO hard... if we rewrote our "main" images to replace current kernel with whatever coreos stable uses i dunno
j0rge
j0rgeOPโ€ข8mo ago
yea that's why I was like "I wonder if there's just a copr"
bsherman
bshermanโ€ข8mo ago
put this on the "ideas to have Benjamin test someday" list ๐Ÿ˜„
Noel
Noelโ€ข8mo ago
I don't think Bazzite would want to use the CoreOS kernel, but realistically I don't think it would affect things given Bazzite replaces the kernel anyway when it builds?
fiftydinar
fiftydinarโ€ข8mo ago
I'm curious on what kind of testing they perform on CoreOS kernel that they don't on Fedora kernel CoreOS kernel might have less regressions in some areas
j0rge
j0rgeOPโ€ข8mo ago
nah this would be for bluefin
bsherman
bshermanโ€ข8mo ago
there IS no CoreOS kernel it's just an older Fedora kernel we can definitely identify the current CoreOS kernel and swap to it in a build mostly it should "just work"
Noel
Noelโ€ข8mo ago
The issue I could see for Bluefin is if it's an older kernel, there will be less hardware enablement out of the box I would think.
bsherman
bshermanโ€ข8mo ago
the main problem I imagine is when Silverblue moves to F40, but CoreOS stable is still on F39... so the stable kernel would likely not swap into F40 cleanly
j0rge
j0rgeOPโ€ข8mo ago
these are still pretty aggressive
bsherman
bshermanโ€ข8mo ago
๐Ÿ™‚ how old do you think "old" is here:? ๐Ÿ™‚
j0rge
j0rgeOPโ€ข8mo ago
6.7.7 two point releases behind F39 right now
Noel
Noelโ€ข8mo ago
does RPM fusion build it's stuff for CoreOS?
bsherman
bshermanโ€ข8mo ago
it doesn't, because it's just Fedora exactly, it's not old enough to worry about hardware enablement, it's just delayed to avoid crashing servers by giving more time for bugs to be caught in the wild
Noel
Noelโ€ข8mo ago
I guess I'm not trying to just nack this idea outright, I'm just trying to name some potential implications of the change.
j0rge
j0rgeOPโ€ข8mo ago
I'm not talking about changing anything
fiftydinar
fiftydinarโ€ข8mo ago
I had bugs with Intel ethernet driver, where it would need re-plug on every boot to work Since CoreOS cares about internet stuff, It would be beneficial for me to use CoreOS kernel to avoid these issues
bsherman
bshermanโ€ข8mo ago
root@orcrist:~# rpm-ostree status --booted
State: idle
AutomaticUpdates: stage; rpm-ostreed-automatic.timer: inactive
BootedDeployment:
โ— ostree-image-signed:docker://ghcr.io/ublue-os/ucore-hci:stable-zfs
Digest: sha256:96f25af297ae988b795c07a1de15f3c6f1d5bbadc40e45411caaccb0a082fff4
Version: 39.20240225.3.0 (2024-03-22T01:41:36Z)
root@orcrist:~# rpm -q --info kernel-core
Name : kernel-core
Version : 6.7.5
Release : 200.fc39
Architecture: x86_64
Install Date: Tue Mar 12 18:43:19 2024
Group : Unspecified
Size : 69228320
License : --- SNIPPED ---
Signature : RSA/SHA256, Sat Feb 17 19:39:36 2024, Key ID 75cf5ac418b8e74c
Source RPM : kernel-6.7.5-200.fc39.src.rpm
Build Date : Sat Feb 17 17:00:34 2024
Build Host : bkernel01.iad2.fedoraproject.org
Packager : Fedora Project
Vendor : Fedora Project
URL : https://www.kernel.org/
Bug URL : https://bugz.fedoraproject.org/kernel
Summary : The Linux kernel
Description :
The kernel package contains the Linux kernel (vmlinuz), the core of any
Linux operating system. The kernel handles the basic functions
of the operating system: memory allocation, process allocation, device
input and output, etc.
root@orcrist:~# rpm-ostree status --booted
State: idle
AutomaticUpdates: stage; rpm-ostreed-automatic.timer: inactive
BootedDeployment:
โ— ostree-image-signed:docker://ghcr.io/ublue-os/ucore-hci:stable-zfs
Digest: sha256:96f25af297ae988b795c07a1de15f3c6f1d5bbadc40e45411caaccb0a082fff4
Version: 39.20240225.3.0 (2024-03-22T01:41:36Z)
root@orcrist:~# rpm -q --info kernel-core
Name : kernel-core
Version : 6.7.5
Release : 200.fc39
Architecture: x86_64
Install Date: Tue Mar 12 18:43:19 2024
Group : Unspecified
Size : 69228320
License : --- SNIPPED ---
Signature : RSA/SHA256, Sat Feb 17 19:39:36 2024, Key ID 75cf5ac418b8e74c
Source RPM : kernel-6.7.5-200.fc39.src.rpm
Build Date : Sat Feb 17 17:00:34 2024
Build Host : bkernel01.iad2.fedoraproject.org
Packager : Fedora Project
Vendor : Fedora Project
URL : https://www.kernel.org/
Bug URL : https://bugz.fedoraproject.org/kernel
Summary : The Linux kernel
Description :
The kernel package contains the Linux kernel (vmlinuz), the core of any
Linux operating system. The kernel handles the basic functions
of the operating system: memory allocation, process allocation, device
input and output, etc.
i just want to make it really clear... the "CoreOS kernel" is literally, the exact same package which had been installed in Fedora XX few weeks back... i could downgrade my current Fedora 39 silverblue machine to this kernel if i wished without adding/swapping repos.
j0rge
j0rgeOPโ€ข8mo ago
I'll experiment with it in my custom branch, but that'll be after you give me a lightweight fork thing
M2
M2โ€ข8mo ago
ucore as a whole has my interest right now
j0rge
j0rgeOPโ€ข8mo ago
heh
p5
p5โ€ข8mo ago
Not too sure what the status of this thread is, but have been playing around with this a little. It's fairly simple to track CoreOS' kernel in our images if those kernel versions are being built for all Fedora versions we are using.
For example, I have current CoreOS :stable kernel installing in FC39, but it's not being built for FC40 so we wouldn't be able to track it for F40. https://github.com/rsturla/eternal-main/pull/106/files
bsherman
bshermanโ€ข8mo ago
this thread is a placeholder https://discord.com/channels/1072614816579063828/1222558445212012624/1222567447073128529 yep I'm not sure if I also said this elsewhere or in voice chat, but tying ourselves to CoreOS kernel releases would also mean whatever we tie to that (say bluefin), that "product/image" needs to more or less tie its release to CoreOS's stable, so bluefin wouldn't promote to F40 until coreos:stable is on F40
p5
p5โ€ข8mo ago
I guess that might be fine for the "gts" image (rather than tracking Fedora versions, track CoreOS releases), but not for people who want to use the latest releases
j0rge
j0rgeOPโ€ข8mo ago
right therein lies the rub
bsherman
bshermanโ€ข8mo ago
so what's the target? people who just want machines to work? or people who want bleeding edge and to change their DE and care about all that junk
p5
p5โ€ข8mo ago
In reality, what new kernel features are there that people would have asked for in the past year? I can't really think of any. GTS is the default, and should be easy to track CoreOS. Latest could be for bleeding edge users, and don't restrict the kernel
bsherman
bshermanโ€ข8mo ago
honestly, i'd like to table the whole CoreOS kernel discussion, maybe put in the planning backlog, but we have other stuff that needs doing, i have things which need to get added there
j0rge
j0rgeOPโ€ข8mo ago
understood. I'll go restart the Flatcar-Bluefin discussion and tag you. j/k
bsherman
bshermanโ€ข8mo ago
jerk ๐Ÿ˜‰
j0rge
j0rgeOPโ€ข8mo ago
So I'm having more thoughts about this And I'm leaning towards waiting for this and not going ga with bluefin this cycle. If every kernel release is going to be going through all this then I'm not digging that Maybe the play is coreos kernel with weekly releases for GTS to slow it down. We keep the client checking every day but only publish on a weekly cron and when we need to like the security fix. Thoughts?
bsherman
bshermanโ€ข8mo ago
i like this... and from a "fast kernel updates break some kmods" perspective... i think the worst kmod break we've seen was the jump to 6.8 for F38/F39 and how it broke v4l2loopback, but even that was at most a week before they got it fixed. so a GTS with weekly updates, even without coreos kernel, already starts to fit this pattern
M2
M2โ€ข8mo ago
Is v4l2loopback fixed?
bsherman
bshermanโ€ข8mo ago
OMG, i need to check... i prepped the PR LOL thank you @M2 for calling me out... i am wrong in my statement of how long v4l2loopback was (is still) not in our images
j0rge
j0rgeOPโ€ข8mo ago
Coreos is in 6.7.7 still So things like loopback would have more time to bake
bsherman
bshermanโ€ข8mo ago
yup
j0rge
j0rgeOPโ€ข8mo ago
And they're still supported, so it's not like we'd be moving to something wacky Plus also other thing If bluefinrora and ucore are more low touch then keeping bazzite up to date spreads out the workload When a new kernel comes out we only have skew and jank in bazzite to fix instead of everything
M2
M2โ€ข8mo ago
And Bazzite is on 6.7.7 right now. They just added an fsync-lts yesterday
j0rge
j0rgeOPโ€ข8mo ago
Then later in Aurorafin we get the same kernel, it's just a month later or whatever Right Lol the quest for the Ubuntu sweet spot I'll write this up on GitHub, I'm mostly shower thinking on the plane Anyway I wanted to run this by people since I'll be hanging with a ton of people with opinions on this all week and asking for ideas Don't worry I won't come back with crazy ideas Like redoing the entire silver blue compose on top of coreos . ๐Ÿ˜ˆ๐Ÿ˜ˆ๐Ÿ˜ˆ๐Ÿ˜ˆ๐Ÿ˜ˆ
M2
M2โ€ข8mo ago
I've considered that.... But it has fedora server defaults instead of workstation
j0rge
j0rgeOPโ€ข8mo ago
Also I haven't thought about builder use for this, which is pretty epic now since we added aurora
M2
M2โ€ข8mo ago
I have an idea in bluefin to reduce builder use for our images
M2
M2โ€ข8mo ago
GitHub
Split -dx and base Containerfiles and Workflows ยท Issue #1134 ยท ubl...
Currently our -dx images build the base image in the process of building them. With the reusable build workflow we can possible have -dx built directly from the just built base image.
j0rge
j0rgeOPโ€ข8mo ago
I like this idea!! Each pr currently fires off 59 builders
p5
p5โ€ข8mo ago
My concern was it will (nearly?) double the time we are waiting to merge PRs, but if you're not bothered, I'm not
M2
M2โ€ข8mo ago
I think it should be about the same amount of time. Since -dx builds base as well Maybe a little slower due to second start. Currently the pain is that the merge queue keeps failing on a single item And it's just the usual stutter
Noel
Noelโ€ข8mo ago
We should definitely get retries in Bluefin and Bazzite like we do in main.
j0rge
j0rgeOPโ€ข8mo ago
I think if we can come up with a reasonable facsimile we may be able to prove it's a good idea
M2
M2โ€ข8mo ago
There are a few ignition files that people have done.
j0rge
j0rgeOPโ€ข8mo ago
Then maybe when the coreos folks and us have time it'd be a neat thing to try sometime down the line Then we do everything together
p5
p5โ€ข8mo ago
How many problems do you think we'd have if we installed the CoreOS kernel from the older Fedora repo onto a more modern version of Fedora? E.g. CoreOS uses F39 kernel. We can install the kernel from the F39 repos onto a F40 image.
bsherman
bshermanโ€ข8mo ago
dependency problems
M2
M2โ€ข8mo ago
Firmware and ?
bsherman
bshermanโ€ข8mo ago
i'm actually running into a problem right now... i can't find the old kernels to install ๐Ÿ˜„ found em https://github.com/coreos/fedora-coreos-config/blob/testing-devel/fedora-coreos-pool.repo
j0rge
j0rgeOPโ€ข8mo ago
GitHub
User CoreOS kernels for :gts ยท Issue #1151 ยท ublue-os/bluefin
The 6.8 release hit F38 as fast as F39 so we ended up with broken v42loopback and stuff. I'd like to investigate a slower cadence for kernels. The CoreOS kernels are Fedora kernels, they just u...
j0rge
j0rgeOPโ€ข8mo ago
moving to github
p5
p5โ€ข8mo ago
My attempt at tracking the CoreOS kernel, and it's looking promising so far. https://github.com/rsturla/eternal-main/pull/116/files All I did was added a GitHub Action step to extract the kernel version string from a CoreOS container (6.7.7-200.fc39), and the version of the repo that kernel is coming from (39). I then pass the repo version into the FEDORA_VERSION build arg so the Containerfile can use that as a base. During the build, I am reading the kernel version string from the CoreOS build arg and installing that same version. If GTS can move from being latest-1 to instead track the CoreOS repos used, it should be fairly simple to implement in Bluefin. This means GTS would currently be on F39.
M2
M2โ€ข8mo ago
Maybe we can turn on the repo for the kernel install and then turn it off? Like what we do with negativio17? Kernel dependencies should be limited to firmware + kernel itself
j0rge
j0rgeOPโ€ข8mo ago
Does this have Nvidia implications?
M2
M2โ€ข8mo ago
It would likely mean using the akmods for uCore Which would be another complication since I believe we only build a subset of kmods compared to the main akmods repo
j0rge
j0rgeOPโ€ข8mo ago
Ok so we wouldn't fall behind on drivers? Oh ok
M2
M2โ€ข8mo ago
It would mean recreating a good portion of akmods, or (more likely) adding this as another kernel variant there
p5
p5โ€ข8mo ago
Deployments:
ostree-image-signed:registry:ghcr.io/rsturla/eternal-linux/lumina:stable-nvidia550 (index: 0)
Digest: sha256:9397374ac025309070d3a65cee71fb2a7968f91d14eccda58f2efa5fa182ba47
Version: 39.20240421.0 (2024-04-21T16:26:41Z)
BaseCommit: e3d848b1ccf60fc32e02718ed1c0f03ca7ad504a0735e76887417ba6b64ab177
Commit: 25ec031b16ee3731224783c15c7d72ba9af548392a42c705058a3b667d1c8d00
โ”œโ”€ 1password (2024-04-17T14:56:59Z)
โ”œโ”€ fedora (2023-11-01T00:12:39Z)
โ”œโ”€ fedora-cisco-openh264 (2023-12-12T17:22:46Z)
โ”œโ”€ fedora-cisco-openh264-debuginfo (2023-12-12T17:22:46Z)
โ”œโ”€ google-chrome (2024-04-19T17:36:02Z)
โ”œโ”€ updates (2024-04-21T01:12:02Z)
โ””โ”€ updates-archive (2024-04-21T01:50:30Z)
Staged: yes
StateRoot: fedora
Downgraded: kernel 6.8.6-200.fc39 -> 6.7.9-200.fc39
kernel-core 6.8.6-200.fc39 -> 6.7.9-200.fc39
kernel-modules 6.8.6-200.fc39 -> 6.7.9-200.fc39
kernel-modules-core 6.8.6-200.fc39 -> 6.7.9-200.fc39
kernel-modules-extra 6.8.6-200.fc39 -> 6.7.9-200.fc39
Removed: kmod-nvidia-6.8.6-200.fc39.x86_64-3:550.67-1.fc39.x86_64
Added: kmod-nvidia-6.7.9-200.fc39.x86_64-3:550.67-1.fc39.x86_64
LayeredPackages: 1password google-chrome-stable
Deployments:
ostree-image-signed:registry:ghcr.io/rsturla/eternal-linux/lumina:stable-nvidia550 (index: 0)
Digest: sha256:9397374ac025309070d3a65cee71fb2a7968f91d14eccda58f2efa5fa182ba47
Version: 39.20240421.0 (2024-04-21T16:26:41Z)
BaseCommit: e3d848b1ccf60fc32e02718ed1c0f03ca7ad504a0735e76887417ba6b64ab177
Commit: 25ec031b16ee3731224783c15c7d72ba9af548392a42c705058a3b667d1c8d00
โ”œโ”€ 1password (2024-04-17T14:56:59Z)
โ”œโ”€ fedora (2023-11-01T00:12:39Z)
โ”œโ”€ fedora-cisco-openh264 (2023-12-12T17:22:46Z)
โ”œโ”€ fedora-cisco-openh264-debuginfo (2023-12-12T17:22:46Z)
โ”œโ”€ google-chrome (2024-04-19T17:36:02Z)
โ”œโ”€ updates (2024-04-21T01:12:02Z)
โ””โ”€ updates-archive (2024-04-21T01:50:30Z)
Staged: yes
StateRoot: fedora
Downgraded: kernel 6.8.6-200.fc39 -> 6.7.9-200.fc39
kernel-core 6.8.6-200.fc39 -> 6.7.9-200.fc39
kernel-modules 6.8.6-200.fc39 -> 6.7.9-200.fc39
kernel-modules-core 6.8.6-200.fc39 -> 6.7.9-200.fc39
kernel-modules-extra 6.8.6-200.fc39 -> 6.7.9-200.fc39
Removed: kmod-nvidia-6.8.6-200.fc39.x86_64-3:550.67-1.fc39.x86_64
Added: kmod-nvidia-6.7.9-200.fc39.x86_64-3:550.67-1.fc39.x86_64
LayeredPackages: 1password google-chrome-stable
๐ŸŽ‰ For UBlue, we need to build akmods, hwe and main with the new kernel, but otherwise not too much hacking
bsherman
bshermanโ€ข8mo ago
I think this should definitely wait until ublue akmods have switched to negativo17 since we had some blocking issues in that PR right @M2 ? And yeah I have thoughts about how we handle the addition kernel variant. At minimum we just build like we do today, adding the new coreos stable kernel version.
j0rge
j0rgeOPโ€ข8mo ago
Fedora Discussion
Fedora CoreOS Community Meeting Minutes 2024-04-24
#meeting-1:fedoraproject.org: fedora_coreos_meeting Meeting started by @siosm:matrix.org at 16:30:59 UTC Meeting summary roll call (@siosm:matrix.org, 16:31:19) Action items from last meeting (@siosm:matrix.org, 16:35:53) LINK: h...
j0rge
j0rgeOPโ€ข8mo ago
UKI kernel notes might be interesting
j0rge
j0rgeOPโ€ข7mo ago
No description
j0rge
j0rgeOPโ€ข7mo ago
6.9 incoming at some point heh they even put 6.8.9 in F38 lol
Want results from more Discord servers?
Add your server