B
BlueBuild•4mo ago
louissunk

High CPU Usage by systemd-udevd Leading to System Stuttering on Fedora Atomic-based Systems

I am experiencing severe system stuttering on Fedora Atomic-based systems, particularly noticeable when starting multiple Docker containers or performing tasks such as mvn clean install. The stuttering affects audio playback (Spotify, voice calls) and mouse movement. System Details: ryzen 7950x3d+64gb+amd 7900GRE+2tb nvme drive Previous Setup (No Issues): Fedora i3wm Spin - Used for over a year without encountering any problems. Problematic Setups: - ublue-os Aurora: Encountered the issue after switching from i3wm to this Atomic-based setup. Unable to resolve, reverted to previous setup. - Custom Image (based on wayblue sway image): Recently installed on the same machine, replicated the issue observed in Aurora. The same custom image works fine on a different machine the previous week. This issue manifests as system.slice/systemd-udevd/udev consuming an excessively high amount of CPU, up to 2000%. (only seen with systemd-cgtop, not with top,htop,btop, mission control) when monitoring udevadm monitor I do see
KERNEL[20546.524311] remove /devices/virtual/net/veth15367bb/queues/rx-0 (queues)
KERNEL[20546.524320] remove /devices/virtual/net/veth15367bb/queues/tx-0 (queues)
KERNEL[20546.524360] remove /devices/virtual/net/veth15367bb (net)
UDEV [20546.527429] add /devices/virtual/net/veth6686c49 (net)
UDEV [20546.557393] remove /devices/virtual/net/veth6686c49/queues/rx-0 (queues)
UDEV [20546.558055] remove /devices/virtual/net/veth6686c49/queues/tx-0 (queues)
UDEV [20546.596101] remove /devices/virtual/net/veth15367bb/queues/tx-0 (queues)
UDEV [20546.596834] remove /devices/virtual/net/veth6686c49 (net)
UDEV [20546.611493] remove /devices/virtual/net/veth15367bb (net)
KERNEL[20546.524311] remove /devices/virtual/net/veth15367bb/queues/rx-0 (queues)
KERNEL[20546.524320] remove /devices/virtual/net/veth15367bb/queues/tx-0 (queues)
KERNEL[20546.524360] remove /devices/virtual/net/veth15367bb (net)
UDEV [20546.527429] add /devices/virtual/net/veth6686c49 (net)
UDEV [20546.557393] remove /devices/virtual/net/veth6686c49/queues/rx-0 (queues)
UDEV [20546.558055] remove /devices/virtual/net/veth6686c49/queues/tx-0 (queues)
UDEV [20546.596101] remove /devices/virtual/net/veth15367bb/queues/tx-0 (queues)
UDEV [20546.596834] remove /devices/virtual/net/veth6686c49 (net)
UDEV [20546.611493] remove /devices/virtual/net/veth15367bb (net)
I do not know what this means, Only happens under load.... I do not know how to solve thisand how to find the cause. Are there any known issues or bugs with systemd-udevd on Fedora Atomic-based systems? Suggestions for further diagnostics ? Thanks
Solution:
So I was able to reproduce this on all my machines and on stock bluefin as well, i can remove the issue by shutting down systemd.udevd service. this can be closed, when I am back from holiday I will make a ticket with ublue os or silverblue fedora depending on if I can reproduce this on silverblue stock with docker layered as well
Jump to solution
24 Replies
louissunk
louissunkOP•4mo ago
custom image repo: https://github.com/tomrutsaert/sway Extra info, I start those docker containers from a distrobox , the same for running the maven commands. But is also the way I did is on my i3wm set-up this is the image I run them in: https://github.com/tomrutsaert/fedora-dev-distrobox
Luke Skywunker
Luke Skywunker•4mo ago
I've had this same issue I have no idea how to fix it though...
louissunk
louissunkOP•4mo ago
do you have the same issue on your current install as well? I noticed just now, That the stutter only happens with 'mvn clean install' when the testing start and the tetsing is using test containers. So I assume this issues has to to do with creating/removing docker containers... in mission control when some containers are running, I do also see now several vethxxxxx network interfaces, which corresponds with the log from udevadm monitor, I assume these are network intefaces for the docker containers..... @gmpinder what did you do to work around the problem?
fiftydinar
fiftydinar•4mo ago
Maybe some corrupted udev rule, which constantly pings systemd-udevd?
louissunk
louissunkOP•4mo ago
i can now reproduce on laptop as well and keep in mind this machine was installed this weekend , so brand new and spotless..... I have a create a docker-compos.yml file. If you run docker compose up -d while playing music I have stutter on all my machines (2) https://gist.github.com/tomrutsaert/0e7df26a882dcf6da0557504e29ce97e If some one would be willing to try that on their machine? a simpliefied version of what I start every day (work related) @fiftydinar I had the same problem on aurora 2 months ago
Luke Skywunker
Luke Skywunker•4mo ago
I didn't 😬
fiftydinar
fiftydinar•4mo ago
I wonder if vanilla Fedora has the same problem
Luke Skywunker
Luke Skywunker•4mo ago
Although, using docker buildx does help
louissunk
louissunkOP•4mo ago
vanilla none atomic does not I asked if some one in ublue-os discord could test as well https://discordapp.com/channels/1072614816579063828/1084120222400774325
Luke Skywunker
Luke Skywunker•4mo ago
I am on Bazzite
louissunk
louissunkOP•4mo ago
do you have docker?
Luke Skywunker
Luke Skywunker•4mo ago
Yeah From Docker's own yum repo
louissunk
louissunkOP•4mo ago
yeah could you test the file above? in the gist
Luke Skywunker
Luke Skywunker•4mo ago
I'm currently running in Cosmic and everything seems fine right now Let me try plasma wayland and x11
louissunk
louissunkOP•4mo ago
It only occurs while starting up and a bit when doing docker compose down
Luke Skywunker
Luke Skywunker•4mo ago
You're in i3 yeah? So that's X11 Let me try x11 real quick
louissunk
louissunkOP•4mo ago
no I am on sway On fedora i3-spin which is not atomic I never had any issues on my wayblue-sway version or on aurora i do have the issues (which are both ublue-os and fedora atomic based)
Luke Skywunker
Luke Skywunker•4mo ago
Oh Well I don't seem to be having these issues anymore Weird I remember being really irritated with it Oh you know what, I did recently upgrade my CPU
louissunk
louissunkOP•4mo ago
yeah I am aswell well my cpu there is not much better and it is worse on my 'good' pc
Luke Skywunker
Luke Skywunker•4mo ago
Yeah I'm not sure what else to tell you
louissunk
louissunkOP•4mo ago
I am gonna reinstall my laptop to bluefin-dx, and see if I can reproduce there But big thanks for trying it
Solution
louissunk
louissunk•4mo ago
So I was able to reproduce this on all my machines and on stock bluefin as well, i can remove the issue by shutting down systemd.udevd service. this can be closed, when I am back from holiday I will make a ticket with ublue os or silverblue fedora depending on if I can reproduce this on silverblue stock with docker layered as well
louissunk
louissunkOP•4mo ago
ths docker-compose i created above tot test is just not heavy enough to reproduce on some system and for example on bluefin, if no other heavy apps were running
louissunk
louissunkOP•4mo ago
Universal Blue
Mouse/Music stutter when starting many docker containers
I’ve experienced stuttering issues with my desktop environment, specificly when running Docker containers or performing Maven builds (that use Docker test-containers in tests), on three different UBlue-based operating systems (Aurora, Bluefin-DX, and a custom Sway image) on two separate machines. Background Information: I had been using Fedor...
Want results from more Discord servers?
Add your server