158 Replies
yeah
data from before:
this exists: https://github.com/NixOS/patchelf
GitHub
GitHub - NixOS/patchelf: A small utility to modify the dynamic link...
A small utility to modify the dynamic linker and RPATH of ELF executables - NixOS/patchelf
and this might help prefixing our sysext binaries and libraries
by running something like
patchelf $MAGIC_FLAGS $LIBRARY_PATH $BINARY
readelf exists too and its output is very parseable by any script
readelf -d /home/linuxbrew/.linuxbrew/bin/nvim | grep 'R.*PATH'
0x000000000000000f (RPATH) Library rpath: [/home/linuxbrew/.linuxbrew/Cellar/neovim/0.9.5/lib:/home/linuxbrew/.linuxbrew/opt/gcc/lib/gcc/current:/home/linuxbrew/.linuxbrew/opt/icu4c/lib:/home/linuxbrew/.linuxbrew/opt/ncurses/lib:/home/linuxbrew/.linuxbrew/opt/readline/lib:/home/linuxbrew/.linuxbrew/opt/zlib/lib:/home/linuxbrew/.linuxbrew/opt/libxml2/lib:/home/linuxbrew/.linuxbrew/opt/gettext/lib:/home/linuxbrew/.linuxbrew/opt/unibilium/lib:/home/linuxbrew/.linuxbrew/opt/libtermkey/lib:/home/linuxbrew/.linuxbrew/opt/libvterm/lib:/home/linuxbrew/.linuxbrew/opt/luajit/lib:/home/linuxbrew/.linuxbrew/opt/luv/lib:/home/linuxbrew/.linuxbrew/opt/msgpack/lib:/home/linuxbrew/.linuxbrew/opt/tree-sitter/lib:/home/linuxbrew/.linuxbrew/opt/openssl@3/lib:/home/linuxbrew/.linuxbrew/opt/libedit/lib:/home/linuxbrew/.linuxbrew/opt/krb5/lib:/home/linuxbrew/.linuxbrew/opt/libtirpc/lib:/home/linuxbrew/.linuxbrew/opt/libnsl/lib:/home/linuxbrew/.linuxbrew/lib]side note
statically linked Go apps don't have any RPATH entries
just a "NEEDED" field
lol interesting
needed libc, libresolv
also brian, apparently bext cannot be statically built with nix :( or at all actually
this is fascinating
I read a blog post about doing static linking go apps with nix a million years ago
yeah!!!
we'll use goreleaser
it'll be fine
I have a pro license
that's a good $20/mo investment tbh
its just that like, bext required
lvm2, which requires systemdMinimal, which cannot be statically built without very weird binary patching things
i tried to build bext with pkgsMusl and it didnt work :(it is what it is
because
-lgpgme and -ldevmapper could not be found by pkg-conf....
its a very interesting rabbit hole but still very sadmaybe we break bext into two pieces
the developer piece and the user piece
the nix building thing and the management
yeah!
the user piece should just need to move things around and enable sysexts
the dev piece has all the hard requirements
i was thinking about doing that with the plugin architecture PR
im just doing preparatory work with UX stuff but ill be working on that right after these first PRs
yeah!!
bext-nix-build
bext-oci-sysext
bext-melange
plugins like that
im so excited for all these sysexts and everything
that could benefit a lot of people
im so happy LOL
you're crushing this so hard. keep up the exploring, creating, building!
hell yeah i will. this is stuff that i really wanna do
UMM
game over https://pkg.go.dev/debug/elf
LOL WHAT
that will help out a bunch
i was thinking we were gonna need a bunch of workarounds in go
welp not anymore
LOL we can literally just edit the ELF symbols
literally open the binary then walk the deps like filepath.Walk
this is awesome
yeah like jesus christ this is amazing
well my work here is done
i'm going to go feed the baby and go to bed
let me know when it all works
going to try to implement the plugin thing
tulip
then slowly integrate everything with bext
hmmm?
do a POC on the patchelf as a quick standalone test
THEN boil the rest of the ocean
yeah LOL makes sense
unless you have unlimited time and attention
in which case, go boil that ocean friend
hahah yeah
i have a bunch of time but not unlimited
trying out the POC approach
yeah you can just pick any dynamically linked app you already have installed and relocate it plus it's libraries into /opt/tulip
without even building a sysext
yeah!
simplest possible proof that this can work
then make a sysext
then conquer the whole linux world
and we'll send @dnkmmr their favorite beverage to celebrate the idea
hell yeah!!!
bonus points if you make a public go package to do this and name it after a carefully chosen elf from LoTR
Neo Encyclopedia Wiki
List of Middle-earth Elves
In J. R. R. Tolkien's legendarium, Elves are one of the races that inhabit a fictional Earth, often called Middle-earth, and set in the remote past. They appear in The Hobbit and in The Lord of...
is there an elf known for moving things?
chatgpt had a cool idea
he is shooting something after all
legolas means "greanleaf"
just in case it matters
also
do you know what the hell is this error
this is super weird
OH i got it nvm
LOL i just didnt read the error
hmmmmmmmmmmmmmmmmmmm
reading stuff works
hey I know what could solve the selinux issues with nix, have it installed as an rpm
shouldnt be impossible
but its not like that would help out :(
the issue is that some files inside the sysexts themselves arent properly getting their xattrs
les goooooo this actually works!
i got the dependencies!!!!!
you're on fire
found out the library paths
now i just need to figure out a way to change those to the new ones
GOD i just found out that RPATH and RUNPATH arent the same
this is SO annoying
LOL
now i can get library paths for even nix binaries
amazing.
works for dynamic non-nix binaries as well
.
wonderful.
@bketelsen well
im going to sleep right now
but i managed to make some progress
apparently elf files have sections and one of those sections is
DT_RUNPATH
and i can seek to the section by getting the .dynstr section offset and adding the DT_RUNPATH offset to that
by doing that the work is pretty easy
jsut cloning the elf file, and writing a new Dynamic64 elf section in its place
but sadly there is some sort of issue that is making my program not be able to write to that section properly
i feel like its something with the file opening perms or something like that
ill post the code to github. since you are quite knowledgeable about golang and all those things, can you help me out? i really cant figure this out!!
i think im naming this literally jsut legolas
sounds cool rolls of the tongue in a nice way
yeeeah
legolas it is.GitHub
GitHub - tulilirockz/legolas: Go-based utility meant to be a P.O.C....
Go-based utility meant to be a P.O.C. plugin for Bext - alternative to patchelf - tulilirockz/legolas
@bketelsen
i also really dont know how could i write that section without accidentaly overriding another section by accident... this is so tricky ngl
I think the go elf package is only for reading
get the info you need from that then use exec to drive patchelf maybe?
I don't see any way in the debug/elf api to save anything.
yeah it is
but i can copy the file and edit it manually
and thats what i wanted to do!
i wanted to override the old file's "DT_RUNPATH" section with new values
but i havent been able to actually do that :(
i think the best way to handle this would be to:
- make new ELF file with all ELF parameters and everything
- seek over to the
dynstr -> DT_RUNPATH (or other) section
- override DT_RUNPATH section with zeros, then truncate with length of bytes needed to write the new DT_RUNPATH section
- write the new DT_RUNPATH section
- Profit!
also there is a neovim plugin that makes comparing these binaries a breeze
amazing honestlyhere's my take which almost works
GitHub
GitHub - bketelsen/uhaul
Contribute to bketelsen/uhaul development by creating an account on GitHub.
Could sysext be used to update static bins in ~/?
the only error I have is because of a symlink. I need to figure out the right logic for that
this is comedic, tbh. I have no idea what I'm doing and i'm just throwing crap at the wall to see what sticks
This guy (https://github.com/pothos) from work is going to take a look and tell me what he thinks. He's a dev/maintainer and seems to be way smarter than me lol
GitHub
pothos - Overview
Software developer (Flatcar Container Linux, GNOME Disks, and more), interested in memory safe network stacks. - pothos
he asked a bunch of questions that suggested I'm not doing things right with the rpath stuff.
FIRST SUCCESS
neovim (just the binary, no supporting files) as a sysext loaded to /opt/bluefin
@j0rge !!
YO WHAT
LETS GOOOOOOOOOOOOO
THIS IS GREAT
LOL same
i used a bunch of your code from legolas
after we have a better clue of what we're doing we can merge ideas into one repo if you want
did you figure out what was wrong?
yeah!!!!
do you want to merge them into a repo in the ublue-os org?
specifically for my first test the
nvim binary showed a dependency on brew's ld.so linker
but I couldntfigure out why
had to change the INTERP of the binaryGitHub
uhaul/cmd/root.go at main · bketelsen/uhaul
Contribute to bketelsen/uhaul development by creating an account on GitHub.
ooooooh
but like why changing the interpreter would be necessary?
could that replace nix for basic package management?
it should be the same in any distro (even nixos)
sadly nah
its not the same
because the interpreter didn't exist in the sysext
ooooooh
that last line comes from the INTERP section, not an actual dependency
even worse it's a symlink to the one in /lib64
so once I fixed that everything started working!
great!!!!
ill test your utility out and send some PRs if necessary
legolas was just a POC so whatever idc
so steal any bits of that you want if you want to keep pushing legolas
no need to abandon yours yet!
there really is no reason to do so LOL in the end the idea is just to have this working
ok well after we get it tight we can put it in ublue. I want to make sure you get credit & ownership.
yup!!!!
next step might be to take an list of binaries instead of just one. Iterate over them and add them to the output dir
shouldnt be too much of an issue either
just looping over the args and everything should suffice
no, just looping in the right place
but there's a few more issues we need to work through
what would those be?
ill write them as issues in gh just so that we have a nice place to analyze them
when I opened neovim in the sysext, the binary had stuff compiled into it with paths
oh.
so that's something to figure out. I dont' know how or where, but I'm betting we can figure it out 🙂
wouldnt that be an issue with where you got the neovim binary itself?
try like patching some neovim binary from ubuntu or something
yeah!
next is: how do you know what else belongs with that binary? Like man pages, etc, supporting files
for that I'm wondering if we could be opinionated and use a particular source (say apk, for example) then read the metadata from the package to get the list of extra stuff
can you run this on your neovim binary?
strings $(which nvim) | grep Cellar
maybe we could straight up write the correct path into the binarylol we can literally just straight up patch that
the last one is the rpath
we CAN?
yeah
hell yeah we can
you can use a hex editor to patch that out
LOL
no need to recompile anything
:>
ok so if we can patch it, can we figure out what they are with automation?
like if it wasn't in a Cellar directory, how would we know they existed?
i think it would depend on where the binary is from
like
if the binary has a
/usr/VERYCOOLBINARY prefix the user would need to specify that
i guess
we could search for the string $PREFIX in the binary code, get the entire string from wherever it is, and override itohh
pretty sure debug/elf has sooome thing that we could use to search that
but for convenience sake we could make the prefix be
/usr by default
the only issue would be stuff like nix binaries that have very odd prefixes...
but any other kind should be finebut we don't want it to be
/usr we want it to be the prefix we specify in the appnono, its just that the original binary most likely would have a
/usr prefixah ok
then we can just patch that out to have the
/usr/coolbinary prefix and everythingso I'd want to figure out what the prefix is, then copy stuff from that prefix into the destination prefix
then patch the strings
yup!
whatever files are in those need to go in the sysext
like
we could use the binary code itself as a separator
like
those strings are most likely surrounded by garbled binary instructions
we can like search for
nvim -> get everything between $BINARY_GARBAGE1 and $BINARY_GARBAGE2
just an idea after all i really dont know if that is gonna work but stillwho knows!
here's the strings for vim that ships with bluefin
wait like
arent text strings in binaries stored in the
.text section?i'm wondering if we can use {share,lib,...} as the clue to figure out what the prefix is
idk
i think so
you tell me lol
i was watching a video about the elf binary format and that seeemms to be the case
when we are making assembly code usually variables and strings are in a separate section
like
.data or .text
here!!!!! section .data!!!
that narrows down a lot of stuff :>
are you thinking use debug/elf to read the .data section?
yup!
read_elf, err := elf.Open(file)
then use read_elf.Section(".data") and walk through it
checking out every chunk
if the chunk is just binary data we can ignore it
if not, we can read the text
sweet!
that was vim
actually maybe its the segments that we need to check out
the segments are exclusively used at runtime
and they seem to have an effect here
oh
there is
literally an "Open" method in each section
cool.
yes, but it's binary data so we have to figure out how it is delimited
how does strings even do that jesus christ
like reading stuff straight off of xxd results in this:
(this is the bash ubuntu binary)
oh!
its the
rodata section
@bketelsen
data doesnt have anything that we want
rodata has literally everything
brb
the prefix isn't listed in .data or .rodata
weird
i wonder where it is being listed
idk. back in a bit, meeting time
let me know if you find something!
we could iterate through all sections and check out which one stores the prefix
okay!
it's definitely .rodata
but the strings in there are more than must null character delimited
I found "Cellar" in one chunk that had a ton of other crap in it
that's one null delimited string
so I figured out why sysexts break "logout, login, shutdown" when they're mounted
if you mount something to /usr everything goes read-only
check the error:
trying to login the user database service tries to open a shared library but can't because it's RO
I just can't believe nobody else has hit this problem
I know userdbd is relatively new
basically if your screen locks you can't log back in
like how is this a thing?
I don't think they had desktops in mind when they designed her, chewie.
yeah well this is a non-starter
we may investigate doing a r/o overlay instead
i got pissed because my sysext finally worked, but I left the vm alone for more than 5 minutes and it locked me out
so I had to force reboot it then spent 30 minutes trolling through the logs
dammit
dammit
OHH
wait a minute
i wonder if you can put stuff in /opt/lib/extensions.d instead of /usr
nope3
fail
fsck
like
does golang have a strings.Find method or something like that?
like in JavaScript someone could just "find" the string and then we could get the seek pointer
https://pkg.go.dev/golang.org/x/text/search
apparently this exists!
if we can search for a string, we could get that string, transform it to bytes, search for these specific bytes in order in the
rodata section, and seek to the start of the byte thing
then override the file from theresearch bytes directly
but it doesn't matter if auth doesn't work on desktop
you can do bytes.Contains() too but you're converting either way
if it was performance code I'd care
but it's hack code
this way allocates memory that doesn't need to be allocated
i asked my coworker (who contributed to sysext development) about problems with GUIs and sysexts, will report back when I hear anything
heyo!
alright!
im messing around with a dumb little project today but ill be helping out with this asap
Hi, it's a recent(?) regression I think, with a mutable upperdir it works (from systemd main branch with extra symlink setup)if nix gets an rpm and it is part of the image, that should fix the nix selinux issues (for the service that starts the daemon)
bro i think you are a bit caught up on nix getting an rpm thing
rpms are literally just zip files but with metadata
they wont fix any selinux issue
we need to figure out ourselves how to make selinux like nix somehow
packaging as an RPM will do nothing honestly
would be awesome nonetheless, and is not a bad idea, but still
I prefer /usr be complete,y untouched while /opt would be used instead in my opinion
good to know. it has other benefits like being ready right after install, not having it very easily by accidentally typing
sudo nix profile remove 0 (which is when nix uninstalls nix) and nix packages could be installed after installation on the first boot to reduce the size of the oci image (for non critical programs like btop)honestly that seems like a good idea
because it would like
make us not require selinux
for nix-based sysexts
would be awesome
same
yeah! it would be sweet
there is no currently maintained rpm for nix either. the rpm could be built from nixpkgs#nixStatic
gotta bring it up with j0rge. I think he would be there to respond quickly because devs don't sleep
@bketelsen i know this doenst make much sense to do right now, but whenever we build the bext website, this would be super awesome as an element to represent bext (like in place of a logo or something like that) https://ui.aceternity.com/components/evervault-card -> we could write like "bext" in the middle and it would look so awesome
Evervault Card
A cool card with amazing hover effect, reveals encrypted text and a mixed gradient.
just writing this down so that i wont forget about this
neat!
BRIAN BRIAN OMG I JUST REMEMBERED SOMETHING VERY COOL
we can make integration tests with nixos vms
like, since we already have a nix package for bext, we can build testing through nixos testing frameworks that people use on nixpkgs
GitHub
nixpkgs/nixos/tests/podman/default.nix at master · NixOS/nixpkgs
Nix Packages collection & NixOS. Contribute to NixOS/nixpkgs development by creating an account on GitHub.
@bketelsen
i'm not a nix n00b, and it still hurts my head when I look at this tuff
i mean its really not that hard tho!
ive written some on a personal project of mine
GitHub
GitHub - tulilirockz/clonix: Declaratively manage rsync deployments...
Declaratively manage rsync deployments in NixOS/Home-manager - tulilirockz/clonix