mkiso thread

mkiso thread
229 Replies
j0rge
j0rgeOP15mo ago
@hippiehacker
hippiehacker
hippiehacker15mo ago
Thanks @j0rge !
j0rge
j0rgeOP15mo ago
when you see any comment in discord, the octothorpe threads it after you click on it:
No description
hippiehacker
hippiehacker15mo ago
I have a big box up, with an ssh session, msg me if you want to join, I'll drop your key there.
hippiehacker
hippiehacker15mo ago
I have just and poetry up with a checkout of https://github.com/akdev1l/mkociso
GitHub
GitHub - akdev1l/mkociso: Tool to compose installer ISOs for OSTree...
Tool to compose installer ISOs for OSTree based containers - GitHub - akdev1l/mkociso: Tool to compose installer ISOs for OSTree based containers
hippiehacker
hippiehacker15mo ago
We'll impove the docs as we go, but I'm currently here:
ublue@cp1:~/mkociso$ just
podman build -v $PWD/build:/outdir --target builder -t mkociso:builder .
Error: The target "builder" was not found in the provided Dockerfile
error: Recipe `make-builder` failed on line 6 with exit code 125
ublue@cp1:~/mkociso$ just
podman build -v $PWD/build:/outdir --target builder -t mkociso:builder .
Error: The target "builder" was not found in the provided Dockerfile
error: Recipe `make-builder` failed on line 6 with exit code 125
ublue@cp1:~/mkociso$ poetry install

Creating virtualenv mkociso-wyKiXsrP-py3.11 in /home/ublue/.cache/pypoetry/virtualenvs
Installing dependencies from lock fileWarning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run `poetry lock [--no-update]` to fix it.Package operations: 17 installs, 0 updates, 0 removals • Installing six (1.16.0) • Installing jmespath (1.0.1) • Installing python-dateutil (2.8.2) • Installing urllib3 (1.26.16) • Installing botocore (1.31.40) • Installing certifi (2023.7.22)
• Installing charset-normalizer (3.2.0)
• Installing click (8.1.7)
• Installing idna (3.4)
• Installing mypy-extensions (1.0.0)
• Installing packaging (23.1)
• Installing pathspec (0.11.2)
• Installing platformdirs (3.10.0)
• Installing s3transfer (0.6.2)
• Installing black (23.7.0)
• Installing boto3 (1.28.40)
• Installing requests (2.31.0)

Installing the current project: mkociso (0.1.0)
ublue@cp1:~/mkociso$ poetry install

Creating virtualenv mkociso-wyKiXsrP-py3.11 in /home/ublue/.cache/pypoetry/virtualenvs
Installing dependencies from lock fileWarning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run `poetry lock [--no-update]` to fix it.Package operations: 17 installs, 0 updates, 0 removals • Installing six (1.16.0) • Installing jmespath (1.0.1) • Installing python-dateutil (2.8.2) • Installing urllib3 (1.26.16) • Installing botocore (1.31.40) • Installing certifi (2023.7.22)
• Installing charset-normalizer (3.2.0)
• Installing click (8.1.7)
• Installing idna (3.4)
• Installing mypy-extensions (1.0.0)
• Installing packaging (23.1)
• Installing pathspec (0.11.2)
• Installing platformdirs (3.10.0)
• Installing s3transfer (0.6.2)
• Installing black (23.7.0)
• Installing boto3 (1.28.40)
• Installing requests (2.31.0)

Installing the current project: mkociso (0.1.0)
ublue@cp1:~/mkociso$ poetry build
Building mkociso (0.1.0)
- Building sdist
- Built mkociso-0.1.0.tar.gz
- Building wheel
- Built mkociso-0.1.0-py3-none-any.whl
ublue@cp1:~/mkociso$ poetry build
Building mkociso (0.1.0)
- Building sdist
- Built mkociso-0.1.0.tar.gz
- Building wheel
- Built mkociso-0.1.0-py3-none-any.whl
Exploring what args to supply to 'poetry run'
ublue@cp1:~/mkociso$ poetry run

Not enough arguments (missing: "args")
ublue@cp1:~/mkociso$ poetry run

Not enough arguments (missing: "args")
just and poetry are both new to me.
ublue@cp1:~/mkociso$ just make-builder
podman build -v $PWD/build:/outdir --target builder -t mkociso:builder .
Error: The target "builder" was not found in the provided Dockerfile
ublue@cp1:~/mkociso$ just make-builder
podman build -v $PWD/build:/outdir --target builder -t mkociso:builder .
Error: The target "builder" was not found in the provided Dockerfile
Not sure if make-builder needs to refer to Containerfile.builder ...
ublue@cp1:~/mkociso$ git diff
diff --git a/justfile b/justfile
index bc5d78d..cbdcd34 100644
--- a/justfile
+++ b/justfile
@@ -3,6 +3,7 @@ make-builder:
-v $PWD/build:/outdir \
--target builder \
-t mkociso:builder \
+ -f Containerfile.builder \
.

make-rpm:
ublue@cp1:~/mkociso$ git diff
diff --git a/justfile b/justfile
index bc5d78d..cbdcd34 100644
--- a/justfile
+++ b/justfile
@@ -3,6 +3,7 @@ make-builder:
-v $PWD/build:/outdir \
--target builder \
-t mkociso:builder \
+ -f Containerfile.builder \
.

make-rpm:
hippiehacker
hippiehacker15mo ago
GitHub
WIP: ii / document mkociso by hh · Pull Request #1 · akdev1l/mkoci...
Let's make sure we have more eyes and documentation for mkociiso.
hippiehacker
hippiehacker15mo ago
It's unclear to me the relationship between the jobs and podman, I think it will require a walkthrough from @akdev . Even if it's async, I can walk back through and create docs I'll leave it up for others to access, have it create the iso.
ssh [email protected] -t tmux at
ssh [email protected] -t tmux at
akdev
akdev14mo ago
I probably broke this ah, let me take a look Okay I improved this I will document it I tried to leave some breadcrumbs last night I noticed it was a Ubuntu server - that makes things a little bit more difficult as everything needs to be containerized
hippiehacker
hippiehacker14mo ago
What would be a better base?
akdev
akdev14mo ago
Fedora😄 We depend on Lorax which is fedora tooling, so mkociso is strictly a fedora tool
j0rge
j0rgeOP14mo ago
hippie do it in an fedora lxc on your ubuntu host, that way you dn't need to mess with the host.
hippiehacker
hippiehacker14mo ago
@j0rge do you know where all that text went from our chat earlier in #tty
hippiehacker
hippiehacker14mo ago
Found it : little icon to the right
No description
hippiehacker
hippiehacker14mo ago
https://weldr.io/lorax/lorax-composer/livemedia-creator.html#appliance-creation looks like we might be able to use Lorax itself to create bootable cloud images
hippiehacker
hippiehacker14mo ago
Build Reliable Operating System Images
Build-Pipelines for Operating System Artifacts
hippiehacker
hippiehacker14mo ago
I wonder what the status of osbuild vs Lorax is
j0rge
j0rgeOP14mo ago
I asked them it's complicated but on the roadmap but yeah, this will half the project obsolete pretty soon. 😄
akdev
akdev14mo ago
You need custom Lorax templates to embed a container image in the installer osbuild is nicer to use than Lorax but there’s no documentation on how to use that with ostree containers
hippiehacker
hippiehacker14mo ago
Which roadmap? Would it make sense for me to go ahead and put our custom Lorax template + embed the container image into the osbuild installer / image creator?
j0rge
j0rgeOP14mo ago
the fedora one
hippiehacker
hippiehacker14mo ago
Maybe this is a chance to improve the docs, maybe we can get someone from their team to help unblock as we document
hippiehacker
hippiehacker14mo ago
GitHub
GitHub - osbuild/fedora-blueprints: osbuild-composer blueprint fi...
osbuild-composer blueprint files to build Fedora Editions and Spins. - GitHub - osbuild/fedora-blueprints: osbuild-composer blueprint files to build Fedora Editions and Spins.
j0rge
j0rgeOP14mo ago
yeah, I'd love to have a conversation with one of them to just understand what we should be doing long term I'm going on their podcast in a few weeks and hope to be able to get some answers or at least put out a call for help
akdev
akdev14mo ago
Yeah but I’m not even sure if the features we need exist yet For osbuild I think there’s literally 1 page worth of documentation 😦
j0rge
j0rgeOP14mo ago
yeah, this is one of those "I wish we can sit in on a team meeting" things I'll do a follow up with them
hippiehacker
hippiehacker14mo ago
I'd like to help them catch this now, before the release of 40, so we can be in lock-step with them from the beginning.
j0rge
j0rgeOP14mo ago
yeah me too
hippiehacker
hippiehacker14mo ago
GitHub
GitHub - osbuild/cockpit-composer: Composer generates custom images...
Composer generates custom images suitable for deploying systems or uploading to the cloud. It integrates into Cockpit as a frontend for osbuild composer - GitHub - osbuild/cockpit-composer: Compose...
No description
hippiehacker
hippiehacker14mo ago
I'm going to continue to explore this route today @akdev
hippiehacker
hippiehacker14mo ago
https://pagure.io/releng/issue/11500#comment-870105 I think we might do this in parallel with the current image builder "to allow iterating to a state where we can consider replacing livemedia-creator with osbuild-composer"
No description
akdev
akdev14mo ago
This is in reference to Workstation images not ostree-based even after this ticket is resolved the plumbing to get Fedora SB images built may not be there
hippiehacker
hippiehacker14mo ago
Fair enough, still exploring here and gaining a lot of vocabulary. @akdev were we able to get a working silverblue image built? I stepped away, for the weekend but I wasn't sure if the image we created worked.
akdev
akdev14mo ago
nope, it gave some grub error I kinda wanna download the image to my local pc to try Cuz I really don’t think I did anything differently
hippiehacker
hippiehacker14mo ago
kk, I might do the same locally
hippiehacker
hippiehacker14mo ago
DevConf
YouTube
Day 3 | E112 - DevConf.CZ 2023
Live stream from the last day of the conference from room E112. Join us during the conference at http://matrix.devconf.cz and ask questions to speakers or interact with other attendees. Today's schedule for this room can be found at https://devconfcz2023.sched.com/2023-06-18/overview/venue/E112+%7C+Talks
No description
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
Those last two are particularly interesting
hippiehacker
hippiehacker14mo ago
https://youtu.be/PsQySAEOeFs?t=17729
When you tell image-builder to embed the container during build time it will download the container from the registry and put it into the right directory so when you run podman it doesn't need to pull the container but it is immediately is on the system. This is great if you have a disconnected environment, then of course you don't have to contact the container registry...
DevConf
YouTube
Day 3 | E112 - DevConf.CZ 2023
Live stream from the last day of the conference from room E112. Join us during the conference at http://matrix.devconf.cz and ask questions to speakers or interact with other attendees. Today's schedule for this room can be found at https://devconfcz2023.sched.com/2023-06-18/overview/venue/E112+%7C+Talks
akdev
akdev14mo ago
Thats pretty interesting Maybe all we need is a blueprint
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
I'm going to finish the video and then try to set this up
hippiehacker
hippiehacker14mo ago
.
No description
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
Redhat will build the images for us if we have the right account via the Image Building Service in RH Hybryid Cloud Console svc has an API that we could use the hosted service and it'll spit out the image
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
We could host the opensource service ourselves
hippiehacker
hippiehacker14mo ago
DevConf
YouTube
Day 3 | E112 - DevConf.CZ 2023
Live stream from the last day of the conference from room E112. Join us during the conference at http://matrix.devconf.cz and ask questions to speakers or interact with other attendees. Today's schedule for this room can be found at https://devconfcz2023.sched.com/2023-06-18/overview/venue/E112+%7C+Talks
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
Good on you for checking out all these videos I would’ve never bothered 🤣 Turns out that’s where the info is now
hippiehacker
hippiehacker14mo ago
This looks interesting as well, we could do mulitple containers / ship them as services
No description
akdev
akdev14mo ago
Those are just quadlets Nothing new
akdev
akdev14mo ago
Enable Sysadmin
Make systemd better for Podman with Quadlet
Quadlet, a tool merged into Podman 4.4, hides the complexity of running containers under systemd to make it easier to maintain unit files written from scratch.
hippiehacker
hippiehacker14mo ago
I think we can embed multiple containers though in the installation / image
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
GUI one to one mapping to CLI:
No description
hippiehacker
hippiehacker14mo ago
Custom Services, Firewall, User, Partitioning, SSH Keys, Groups, Timezone, Locale, Kernel, Ignition (cloud-init):
No description
hippiehacker
hippiehacker14mo ago
Might have to much slightly with image output types:
No description
hippiehacker
hippiehacker14mo ago
Five minutes for his demo build:
No description
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
Might be good to connect with the speaker Ondřej Budai at RedHat : https://devconfcz2023.sched.com/speaker/obudai1
Ondřej Budai's schedule for Devconf.cz 2023
Check out what Ondřej Budai will be attending at Devconf.cz 2023
hippiehacker
hippiehacker14mo ago
We've been busy with implementing new features in Image Builder! If you already don't know, Image Builder is a tool for building up-to-date, customized operating system images of Fedora, CentOS Stream, and RHEL. It can not just build the image, but also upload it to your favorite cloud so you can launch it immediately. In this session, we will demonstrate what's possible with the latest and greatest Image Builder stack, including: - injecting custom files into an image - embedding podman containers - using the new completely rewritten UI - building customized Fedora IoT images The goal of this talk is to convince you that Image Builder is a great tool for building ready-to-be-used images that are runnable almost anywhere. Come and see yourself!
Given what we see here, I still don't see any ostree commands in the demo
hippiehacker
hippiehacker14mo ago
Other than what was in the first couple slides:
No description
hippiehacker
hippiehacker14mo ago
But it's a lot of new stuff for me so I might be missing something
j0rge
j0rgeOP14mo ago
yeah you're doing the research we needed to do!
hippiehacker
hippiehacker14mo ago
First Boot Automation in Image Builder
Interested in running commands on the first boot of your image built by Image Builder? This blog post explains it in great details!
hippiehacker
hippiehacker14mo ago
It seems like our first-boot service worked! 🎉 With this pattern, you should be able to run arbitrary code on the first boot, allowing you to further customize your instances. I’m planning to write up a follow-up to this post to show you more tricks that you can do, so stay tuned. 📻
This will help us connect in the "you don't have to do this alone, would you like some help" approach to cloud native desktops. https://osbuilder.sharing.io:9090/composer is up, now for some blueprints msg me for the root password I have to go find a place to live, but let's see if we can't find some example blueprints, I think the presentation had some, but I haven't found the source yet
j0rge
j0rgeOP14mo ago
No description
j0rge
j0rgeOP14mo ago
osbuilder has so much good stuff if this was just an OCI URL we'd be done
hippiehacker
hippiehacker14mo ago
Looking into things
hippiehacker
hippiehacker14mo ago
Matrix - Decentralised and secure communication
You're invited to talk on Matrix. If you don't already have a client this link will help you pick one, and join the conversation. If you already have one, this link will help you join the conversation
hippiehacker
hippiehacker14mo ago
@akdev you should probably join #image-builder on matrix :
No description
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
I’m not gonna lie The matrix client is pretty crap so I have it on my phone but never use it
hippiehacker
hippiehacker14mo ago
fair enough, I'll just post what I see and relay
akdev
akdev14mo ago
Seems like all the implementation and information is there
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
They just decided to put in places I would never look for in a million years
hippiehacker
hippiehacker14mo ago
(do we have a link to where we do that?)
akdev
akdev14mo ago
The Lorax templates I showed you
akdev
akdev14mo ago
Someone there should be able to read this and get it
hippiehacker
hippiehacker14mo ago
kk,
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
Is matrix available on desktop?
hippiehacker
hippiehacker14mo ago
I'm using some software called element
akdev
akdev14mo ago
Oh cool That’s the one I hate on mobile, probably better I download it on my Linux desktop Then it will be better
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
where do we have the kickstart entry with the ostreecontainer entry?
akdev
akdev14mo ago
We add it in interactive-defaults.ka
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
It does work that’s what we do now
hippiehacker
hippiehacker14mo ago
yea' just relaying to keep context between this channel and the other one
akdev
akdev14mo ago
There’s a problem though
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
Can you mention this issue though
akdev
akdev14mo ago
GitHub
how to create offline installer using ostree native container? · Is...
Hi, I am trying to create offline installers for Ublue ISOs which are based on ostree containers. I was able to use lorax to get things to work and I lightly modified lorax-configure-repo.tmpl and ...
akdev
akdev14mo ago
We lack —target-imgref in anaconda so that’s why we need an extra rebase after install
akdev
akdev14mo ago
GitHub
Add support for bootc · rhinstaller anaconda · Discussion #5197
So a while ago, we added the ostreecontainer kickstart verb. The advantage of this is that it works in exactly the same way/shape as the ostreesetup did. However, the bigger picture direction I&#39...
akdev
akdev14mo ago
Trying to join the matrix channel
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
how to do that? I tried my best to find a way of setting that but I couldn’t find out - that would be far less ghetto
j0rge
j0rgeOP14mo ago
heh
akdev
akdev14mo ago
maybe ask them if we could also prepopulate the container storage also Maybe that way we could just use the correct ref but if the container is already in storage it won’t need to pull
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
We don’t have an ostree remote to point to, I could only find stuff about ostree remotes
hippiehacker
hippiehacker14mo ago
https://pagure.io/fedora-kickstarts/blob/main/f/fedora-iot.ks#_49
ostree remote add --set=gpg-verify=true --set=gpgkeypath=/etc/pki/rpm-gpg/ --set=contenturl=mirrorlist=https://ostree.fedoraproject.org/iot/mirrorlist fedora-iot 'https://ostree.fedoraproject.org/iot'
akdev
akdev14mo ago
After all we want the future upgrades to pull from the registry also Not from Fedora’s upstream ostree remote
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
I’m gonna need a keyboard to answer that one, 2 mins
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
Fair enough
akdev
akdev14mo ago
From what I understand the ref is the source of the updates when ostree is working with containers It uses none of the ostree remote machinery
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
--target-imgref <TARGET_IMGREF>
Target image reference, e.g. ostree-remote-image:someremote:registry:quay.io/exampleos/exampleos:latest

If specified, `--imgref` will be used as a source, but this reference will be emitted into the origin so that later OS updates pull from it.
--target-imgref <TARGET_IMGREF>
Target image reference, e.g. ostree-remote-image:someremote:registry:quay.io/exampleos/exampleos:latest

If specified, `--imgref` will be used as a source, but this reference will be emitted into the origin so that later OS updates pull from it.
hippiehacker
hippiehacker14mo ago
What man page or link is that from?
akdev
akdev14mo ago
ostree container image deploy --help
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
yeah but afaik there's no way of configuring this ref via files unlike ostree remotes
ls /etc/ostree/remotes.d/
fedora-compose.conf fedora.conf ublue-os.conf
ls /etc/ostree/remotes.d/
fedora-compose.conf fedora.conf ublue-os.conf
this directory only works for ostree remotes from what I gather I tried like:
cat /etc/ostree/remotes.d/ublue-os.conf
[remote "ublue-os"]
url=ghcr.io/akdev1l/ostree-images/personal-silverblue:38
cat /etc/ostree/remotes.d/ublue-os.conf
[remote "ublue-os"]
url=ghcr.io/akdev1l/ostree-images/personal-silverblue:38
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
the files provided by fedora look like this:
cat /etc/ostree/remotes.d/fedora.conf
[remote "fedora"]
url=https://ostree.fedoraproject.org
gpg-verify=true
gpgkeypath=/etc/pki/rpm-gpg/
contenturl=mirrorlist=https://ostree.fedoraproject.org/mirrorlist
cat /etc/ostree/remotes.d/fedora.conf
[remote "fedora"]
url=https://ostree.fedoraproject.org
gpg-verify=true
gpgkeypath=/etc/pki/rpm-gpg/
contenturl=mirrorlist=https://ostree.fedoraproject.org/mirrorlist
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
truly no clue, Collin seemed to agree with me that we need anaconda support so that's the only thing I have to go by I can see that ostree refs prints out something like:
ostree refs | grep ublue-os
ostree/container/image/docker_3A__2F__2F_ghcr_2E_io/ublue-os/silverblue-main_3A_latest
ostree refs | grep ublue-os
ostree/container/image/docker_3A__2F__2F_ghcr_2E_io/ublue-os/silverblue-main_3A_latest
one thing I tried and failed was to manually deploy the container image and then try to use one of these refs as remote ... didn't work, it still wanted to pull updates from oci:/var/ublue-os/image oh here: https://github.com/ostreedev/ostree-rs-ext/blob/3e42da561d86ad95771fe1567baeb4612301899e/lib/src/container/deploy.rs#L87-L88 I would say it looks like it is stored as an "origin" in the deployment itself
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
I'm looking fairly smart here, I'll have to tell them at some point I'm just a puppet of the ublue community 🙂 Your voicebox 🙂
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
well I still can't figure out how to join
hippiehacker
hippiehacker14mo ago
I think you can use element the local desktop app and see if you can connect to matrixy things
j0rge
j0rgeOP14mo ago
yeah there's an app called element, it's awful gl
hippiehacker
hippiehacker14mo ago
until then there hippie relay sneaker networking
akdev
akdev14mo ago
I have the app And I am logged into the fedora server It just literally doesn’t show any image-builder channels and the link hippy shared doesn’t work 😂
hippiehacker
hippiehacker14mo ago
Matrix - Decentralised and secure communication
You're invited to talk on Matrix. If you don't already have a client this link will help you pick one, and join the conversation. If you already have one, this link will help you join the conversation
hippiehacker
hippiehacker14mo ago
That's what I "copy" when I'm in the matrix chat
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
akdev
akdev14mo ago
I was able to join, feel free to ping me if something interesting is happening there 🙂
hippiehacker
hippiehacker14mo ago
kk Thanks @akdev I think this is going to be fun and beneficial for lots of folks
akdev
akdev14mo ago
Thank you for putting all this effort into getting all the information and the right contacts
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
GitHub
osbuild-composer/test/data/manifests/fedora_38-x86_64-iot_installer...
An HTTP service for building bootable OS images. Contribute to osbuild/osbuild-composer development by creating an account on GitHub.
hippiehacker
hippiehacker14mo ago
GitHub
osbuild/stages/org.osbuild.kickstart at main · osbuild/osbuild
Build-Pipelines for Operating System Artifacts. Contribute to osbuild/osbuild development by creating an account on GitHub.
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
No description
hippiehacker
hippiehacker14mo ago
Zoom Video
Join our Cloud HD Video Meeting
Zoom is the leader in modern enterprise video communications, with an easy, reliable cloud platform for video and audio conferencing, chat, and webinars across mobile, desktop, and room systems. Zoom Rooms is the original software-based conference room solution used around the world in board, conference, huddle, and training rooms, as well as ex...
hippiehacker
hippiehacker14mo ago
feel free to join I'm also recording
akdev
akdev14mo ago
Ah I probably can’t today 😦 we have a big sale at work and I’m actually on call I’ll watch the recording Sorry the schedule was set a few days ago 😭
hippiehacker
hippiehacker14mo ago
No description
αchilleaς
αchilleaς14mo ago
Hello
hippiehacker
hippiehacker14mo ago
Should we be worried about Image has 4 non-ostree layers ? Thanks for your help @achilleas !!
αchilleaς
αchilleaς14mo ago
Here's how we did it
[ii@fedora osbuild-experiments]$ skopeo copy docker://ghcr.io/ublue-os/bluefin-dx:38-20231012 oci-archive:ublue-bluefin-dx-38.tar
[ii@fedora osbuild-experiments]$ mkdir ublue-repo/
[ii@fedora osbuild-experiments]$ ostree init --repo=ublue-repo
[ii@fedora osbuild-experiments]$ ostree container unencapsulate --repo=ublue-repo ostree-unverified-image:oci-archive:ublue-bluefin-dx-38.tar
Downloading...
error: Image has 4 non-ostree layers
[ii@fedora osbuild-experiments]$ ls ublue-
ls: cannot access 'ublue-': No such file or directory
[ii@fedora osbuild-experiments]$ ls ublue-repo/
config extensions objects refs state tmp
[ii@fedora osbuild-experiments]$ ostree container unencapsulate --repo=ublue-repo ostree-unverified-image:oci-archive:ublue-bluefin-dx-38.tar
Downloading...
error: Image has 4 non-ostree layers
[ii@fedora osbuild-experiments]$ echo $?
1
[ii@fedora osbuild-experiments]$
[ii@fedora osbuild-experiments]$ skopeo copy docker://ghcr.io/ublue-os/bluefin-dx:38-20231012 oci-archive:ublue-bluefin-dx-38.tar
[ii@fedora osbuild-experiments]$ mkdir ublue-repo/
[ii@fedora osbuild-experiments]$ ostree init --repo=ublue-repo
[ii@fedora osbuild-experiments]$ ostree container unencapsulate --repo=ublue-repo ostree-unverified-image:oci-archive:ublue-bluefin-dx-38.tar
Downloading...
error: Image has 4 non-ostree layers
[ii@fedora osbuild-experiments]$ ls ublue-
ls: cannot access 'ublue-': No such file or directory
[ii@fedora osbuild-experiments]$ ls ublue-repo/
config extensions objects refs state tmp
[ii@fedora osbuild-experiments]$ ostree container unencapsulate --repo=ublue-repo ostree-unverified-image:oci-archive:ublue-bluefin-dx-38.tar
Downloading...
error: Image has 4 non-ostree layers
[ii@fedora osbuild-experiments]$ echo $?
1
[ii@fedora osbuild-experiments]$
αchilleaς
αchilleaς14mo ago
rpm-ostree
ostree native containers
rpm-ostree documentation
αchilleaς
αchilleaς14mo ago
because osbuild (and osbuild-composer) doesn't currently work with ostree nativer containers just yet but that's being worked on
hippiehacker
hippiehacker14mo ago
@j0rge our derivative images... do they create non-ostree layers? We are having a bit of trouble import
αchilleaς
αchilleaς14mo ago
I wonder if that's because the derivative images are created using dockerfiles, so they're no longer proper encapsulated ostree commits
akdev
akdev14mo ago
What is a non-ostree layer exactly? Also do we have less non-ostree layers in our main image?
αchilleaς
αchilleaς14mo ago
GitHub
containers: support converting existing base images? · Issue #11 · ...
There's nothing conceptually stopping us from having tooling that takes a current container image (e.g. what's in debian:stable) and just makes it "ostree ready". Or even just dya...
GitHub
Unencapsulating OCI with extra layers (or: speeding up building chi...
In MicroShift CI we're building several ostree commits using osbuild. However, because we need to build a parent image (clean el92), and then build commits with different versions of MicroShift...
αchilleaς
αchilleaς14mo ago
so as soon as you derive an ostree native container and add extra container layers to it, ostree can't unencapsulate it anymore
j0rge
j0rgeOP14mo ago
yeah this is why our downloads are so big iirc
αchilleaς
αchilleaς14mo ago
Well Colin wants it solved so at least there's that
j0rge
j0rgeOP14mo ago
I think everyone wants it solved it's just kind of a complicated problem that touches a ton of projects and to be fair, we knew we were going early and most of this stuff didn't exist, part of the reason we're doing this is to explore the possibilities
iiamabby
iiamabby14mo ago
👋
αchilleaς
αchilleaς14mo ago
GitHub
stages/ostree.deploy: accept containers input to deploy by dustymab...
The ostree.deploy stage now accepts either a ostree input: - type: org.osbuild.ostree.deploy options: osname: fedora-coreos remote: fedora mounts: ...
αchilleaς
αchilleaς14mo ago
PR adds support for deploying an ostree native container to create an image in osbuild
j0rge
j0rgeOP14mo ago
It's there to add pain to remind people that they should be pulling containers signed via e.g. sigstore at least.
love it lol
hippiehacker
hippiehacker14mo ago
Would it be difficult to try this or should we just wait for merge / release?
αchilleaς
αchilleaς14mo ago
I intend to try it tomorrow morning. You can try it but you'll have to know how to write an osbuild mpp manifest, which is a bit advanced. Wiring it up through composer will need a bit more work.
j0rge
j0rgeOP14mo ago
this is amazing, who else should we loop in?
hippiehacker
hippiehacker14mo ago
Woo hoo!
hippiehacker
hippiehacker14mo ago
Zoom
Video Conferencing, Web Conferencing, Webinars, Screen Sharing
Zoom is the leader in modern enterprise video communications, with an easy, reliable cloud platform for video and audio conferencing, chat, and webinars across mobile, desktop, and room systems. Zoom Rooms is the original software-based conference room solution used around the world in board, conference, huddle, and training rooms, as well as ex...
hippiehacker
hippiehacker14mo ago
@akdev wanted to make sure you could glean from that time with @achilleas
hippiehacker
hippiehacker14mo ago
GitHub
Add support for initoverlay · Issue #3066 · ostreedev/ostree
Splitting this from #2867 (comment) which was inspired by #2867 (comment) Basically a major flaw with initramfs (whether baked into the kernel binary or separate) is that it's not lazy - the en...
αchilleaς
αchilleaς14mo ago
oh wow, that was 2 hours? :D
akdev
akdev14mo ago
I will take a look 👍 Thanks guys
hippiehacker
hippiehacker14mo ago
Yep I'm seeing lots of activity here!!
αchilleaς
αchilleaς14mo ago
Working out some details but the main thing works. I built (and booted) a CoreOS qcow2 from their container earlier today. The long discussion is about exposing signature verification. And I had some nitpicks about commit messages because someone has to :)
akdev
akdev14mo ago
Could you show that the blueprint looks like?
αchilleaς
αchilleaς14mo ago
It's not wired up to composer yet so you have to build a handwritten manifest directly (or make one with osbuild-mpp). I'm not at home now, so don't have access to the one I built. I'll put it somewhere tomorrow and link it here for you to test.
akdev
akdev14mo ago
Oh I see, thank you that would be awesome Another random question: Neal Gompa advised us to produce disk images with 4K alignment - is this possible in osbuild? I think for asahi Linux we will need to produce such image so just wondering
αchilleaς
αchilleaς14mo ago
Yes that's possible
αchilleaς
αchilleaς14mo ago
@akdev with the changes in osbuid PR 1402, this manifest will build a bootable CoreOS qcow2 image, from the container at quay.io/fedora/fedora-coreos: https://gist.github.com/achilleas-k/fefd53effb0c0d0d71df2ac4e2ff533e
Gist
fedora-ostree-container-deploy.json
GitHub Gist: instantly share code, notes, and snippets.
αchilleaς
αchilleaς14mo ago
I had to remove the ignition config stages because they cause boot failures. It's a known issue with some race conditions during boot and user creation. That means you can't log in because there's no way to configure a user account in the image.
akdev
akdev14mo ago
Thank you for this! I’m reading https://www.osbuild.org/man/osbuild-manifest.5 to help me understand What does this mean? "sha256:718d95c40b41c2f0ecc8dc2290ebb91b529ba3be7accbad9c30c88e9ce408349": {}, I thought this should be in the form of $hash: /path/to/file I see we have a lot of stages here: https://github.com/osbuild/osbuild/tree/main/stages but I couldn’t find org.osbuild.files This is where the magic happens: https://github.com/osbuild/osbuild/blob/main/stages/org.osbuild.ostree.deploy With input type: https://github.com/osbuild/osbuild/blob/main/inputs/org.osbuild.containers Ok I think I get it - this approach isn’t possible with derived containers correct? To have derived support we would need some stage that can do ostree container deploy? So stage: org.osbuild.ostree.container.deploy
akdev
akdev14mo ago
Ah makes sense
αchilleaς
αchilleaς14mo ago
About manifests: - "sha256:718d95c40b41c2f0ecc8dc2290ebb91b529ba3be7accbad9c30c88e9ce408349": {} is a reference to an item in the sources section at the end of the manifest. It can be a container, an ostree commit, or a file. If you see it in the inputs of the rpm stage, it's an rpm downloaded using an org.osbuild.curl source. if you follow the contend ID (sha256 hash), you'll see the URL for the file. - Similarly, in my manifest, sha256:f5ab2e0456ff52b0e95e99c59a1cc70fb690153f146c1c4163c4916781ac3ad5 is the resource ID for the container, downloaded using skopeo:
"org.osbuild.skopeo": {
"items": {
"sha256:f5ab2e0456ff52b0e95e99c59a1cc70fb690153f146c1c4163c4916781ac3ad5": {
"image": {
"name": "quay.io/fedora/fedora-coreos",
"digest": "sha256:44a1bd71e649f802f84c20836b1fa5c2fa58afc3b2f7af44a06cce2ddca3dc8c"
}
}
}
"org.osbuild.skopeo": {
"items": {
"sha256:f5ab2e0456ff52b0e95e99c59a1cc70fb690153f146c1c4163c4916781ac3ad5": {
"image": {
"name": "quay.io/fedora/fedora-coreos",
"digest": "sha256:44a1bd71e649f802f84c20836b1fa5c2fa58afc3b2f7af44a06cce2ddca3dc8c"
}
}
}
When you look at the deploy stage, it references this resource ID in the input:
{
"type": "org.osbuild.ostree.deploy",
"options": { "<snip>" },
"inputs": {
"images": {
"type": "org.osbuild.containers",
"origin": "org.osbuild.source",
"references": {
"sha256:f5ab2e0456ff52b0e95e99c59a1cc70fb690153f146c1c4163c4916781ac3ad5": {
"name": "quay.io/fedora/fedora-coreos:stable"
}
}
}
}
}
{
"type": "org.osbuild.ostree.deploy",
"options": { "<snip>" },
"inputs": {
"images": {
"type": "org.osbuild.containers",
"origin": "org.osbuild.source",
"references": {
"sha256:f5ab2e0456ff52b0e95e99c59a1cc70fb690153f146c1c4163c4916781ac3ad5": {
"name": "quay.io/fedora/fedora-coreos:stable"
}
}
}
}
}
The org.osbuild.skopeo source means "use skopeo to download this container and store it under this ID". The org.osbuild.ostree.deploy stage with its input says "run osbuild ... deploy using this resource as your input".
akdev
akdev14mo ago
Do the IDs need to be hashes?
αchilleaς
αchilleaς14mo ago
Generally yes, technically, I'm not 100% sure. It might work if you change it, but at some point there might be some checksum validation.
akdev
akdev14mo ago
Okay I see, how to get the hash for a container image?
αchilleaς
αchilleaς14mo ago
GitHub
osbuild/sources/org.osbuild.skopeo at d66d58ae8a5365cabb71f8edb1ec5...
Build-Pipelines for Operating System Artifacts. Contribute to osbuild/osbuild development by creating an account on GitHub.
αchilleaς
αchilleaς14mo ago
skopeo inspect --config <container> The hash of that is the container ID (per the OCI spec)
❯ skopeo inspect --raw --config docker://quay.io/fedora/fedora-coreos:stable | sha256sum
f5ab2e0456ff52b0e95e99c59a1cc70fb690153f146c1c4163c4916781ac3ad5 -
❯ skopeo inspect --raw --config docker://quay.io/fedora/fedora-coreos:stable | sha256sum
f5ab2e0456ff52b0e95e99c59a1cc70fb690153f146c1c4163c4916781ac3ad5 -
akdev
akdev14mo ago
Ah yeah I was gonna ask if this worked^ Okay I think I got a good grasp of how this works This is much better than Lorax
αchilleaς
αchilleaς14mo ago
haha good to know. Replacing Lorax was sort of the point of osbuild. (at least afaik, I wasn't around when it started)
j0rge
j0rgeOP14mo ago
I feel like we need the 10,000 foot view on all of this from someone someday. I knew osbuild existed but it wasn't clear to me that that this would be The Way until a few months ago and even then I wans't sure.
akdev
akdev14mo ago
I understood osbuild was the successor of Lorax - I only touched Lorax because I couldn’t figure out how to use osbuild 😭
αchilleaς
αchilleaς14mo ago
We had a couple of osbuild/IB talks at DevConf.cz. I think those might be good as a 10k ft view.
αchilleaς
αchilleaς14mo ago
Ondrej's "what's new" talk includes a usage example: https://www.youtube.com/watch?v=G_GYnBg1eYA
DevConf
YouTube
What's new in Image Builder? - DevConf.CZ 2023
Speaker(s): Ondřej Budai We've been busy with implementing new features in Image Builder! If you already don't know, Image Builder is a tool for building up-to-date, customized operating system images of Fedora, CentOS Stream, and RHEL. It can not just build the image, but also upload it to your favorite cloud so you can launch it immediately. ...
αchilleaς
αchilleaς14mo ago
And I gave a talk about how we're thinking about how to define images in the future: https://www.youtube.com/watch?v=_rBnqhWaU2A
DevConf
YouTube
If you wish to build a Linux image from scratch...- DevConf.CZ 2023
Speaker(s): Achilleas Koutsou Image Builder builds bootable OS images of Fedora, CentOS, and RHEL. With support for multiple versions of each distribution, four hardware architectures, and a growing number of cloud environments and workloads, it quickly becomes hard to manage the number of configurations, while still being very confident that a...
αchilleaς
αchilleaς14mo ago
@j0rge I should really write a lot of this down and put it in blog form somewhere...
j0rge
j0rgeOP14mo ago
oh awesome thank you!
αchilleaς
αchilleaς14mo ago
And this is Lars and Tom (who started the project) talking about osbuild from back when the project started https://www.youtube.com/watch?v=5vv-i6mPtrQ
DevConf
YouTube
Introducing OSBuild - DevConf.CZ 2020
Speakers: Lars Karlitski, Tom Gundersen OSBuild is a generic low-level tool for creating OS images. It is the new backend for Image Builder, and it is designed to make the definition and creation of OS images transparent, predictable, modifiable and reproducible. An OS image is fully specified in a declarative configuration format, which means ...
hippiehacker
hippiehacker14mo ago
@achilleas @akdev https://github.com/osbuild/osbuild/pull/1399 is there something we can test with Image Builder now?
akdev
akdev14mo ago
We probably need to modify the osbuild manifest created before To take advantage of these new features
hippiehacker
hippiehacker14mo ago
It's not clear to me what we need to change about the osbuild manifest we generate yet to utilize this PR
αchilleaς
αchilleaς14mo ago
Still needs https://github.com/osbuild/osbuild/pull/1402 to deploy the container
GitHub
stages/ostree.deploy: accept containers input to deploy by dustymab...
The ostree.deploy stage now accepts either a ostree input: - type: org.osbuild.ostree.deploy options: osname: fedora-coreos remote: fedora mounts: ...
αchilleaς
αchilleaς14mo ago
the PR you linked adds a feature to mpp (the macro preprocessor... or is it manifest preprocessor?) to help resolve an ostree commit from a repo + ref to a commit ID/hash. It's just a convenience for generating the manifest. in osbuild-composer we already do this anyway, so it doesn't change anything for IB osbuild-mpp is a development/testing tool that got a bit too popular and is being used by some projects to generate manifests and work with osbuild directly
hippiehacker
hippiehacker14mo ago
@akdev we are getting closer to what the manifests will look like: https://github.com/osbuild/osbuild/commit/f742e5eef45eeae767959e250fff3fb2e47e852f
GitHub
manifests: add ostree container input examples · osbuild/osbuild@f7...
These examples show how to run a pipeline with ostree containers as inputs. One of them pulls from a remote registry and the other builds an ostree commit first, then encapsulates it, then deploys ...
Want results from more Discord servers?
Add your server