/var/tmp/ full of 'container_images_storageXXXXX*

Hiya, This keeps happening to me and I know what is happening. Basically I'll get a KDE notification about "You are running out of disk space", much like this GH repo [You are running out of disk space] (https://github.com/actions/runner-images/issues/2840). So I check /var/temp/ and its filled with loads of folders called: container_images_storageXXXXX, the X's being numbers. Each folder has three files: named "1", "2" and "3". After a while, maybe 20 mins, the files are removed by themselves somehow.
107 Replies
Elijah'
Elijah'OP•8mo ago
No description
No description
No description
Elijah'
Elijah'OP•8mo ago
There are about 400 of these folders I'm assuming its related to some podman activity?
Elijah'
Elijah'OP•8mo ago
and it happened again
No description
Elijah'
Elijah'OP•8mo ago
when I try to delete the folders, they just re-appear again..
No description
No description
Zetarancio
Zetarancio•8mo ago
Have u tried the ujust clean-system command?
wolfyreload
wolfyreload•8mo ago
It's probably trying to move the files to the trash folder and failing. What happens if you shift+delete the files?
Elijah'
Elijah'OP•8mo ago
Even if I do get to delete the files, they will come back after a while; like it did today.
No description
Elijah'
Elijah'OP•8mo ago
I will now :) So I did that yesterday, and cleaned out the tmp folder, but all the files are back again today :< podman info suggests one line saying "imageCopyTmpDir: /var/tmp", that must be it? Is this from the distrobox i set up with boxboddy maybe?
Kyle Gospo
Kyle Gospo•8mo ago
Yes
Elijah'
Elijah'OP•8mo ago
I have removed all distroboxes now, but the folders keep appearing :<
Elijah'
Elijah'OP•8mo ago
No description
Elijah'
Elijah'OP•8mo ago
they fill up slowly but surely, until all of my SSD space has been eaten up I really dont understand what is causing this, i really only use bazzite for browsing and gaming
HikariKnight
HikariKnight•8mo ago
check both toolbox and podman too (podman being the backend for toolbox and distrobox) podman ps --all also might be worth checking journalctl -xe and see if there is any clues there
Elijah'
Elijah'OP•8mo ago
No description
Elijah'
Elijah'OP•8mo ago
journalctl seems to be highly cluttered with some plasmashell issue im not familiar with what being either; https://paste.centos.org/view/0c8df791 seems like another bug;^ https://bugs.kde.org/show_bug.cgi?id=453876
HikariKnight
HikariKnight•8mo ago
huh, interesting i dont have that
Elijah'
Elijah'OP•8mo ago
since i last sent the screenshot of the container_images_storage, which is 15mins ago, there are now an addtional 6 new ones they're really piling up fast
HikariKnight
HikariKnight•8mo ago
hmm journalctl -xe ---follow | grep -v plasmashell that should maybe make it a bit more readable (sadly removes colors though, but you shouldnt be spammed by plasmashell) cant help much more since i am at work
Elijah'
Elijah'OP•8mo ago
Thats ok, but if I could have a hand with this later I'd greatly appriciate it, as I've been stuck with this problem for over a week now and cant seem to figure out the origin or a solution :<
HikariKnight
HikariKnight•8mo ago
there isnt really much else i can provide since thats the only other place i can think of looking. it is an issue only you have and i have never seen it before and all i can guess is that there is something interacting with containers in some way that is misbehaving (steam, flatpak, podman/toolbox/distrobox/docker, etc)
Elijah'
Elijah'OP•8mo ago
But where do I go to get help with this if not here? This is too far out of my knowledge spectre in Linux. I did find this Github issue on what looks similar, but again; too far out of my comfort zone to say what it is. https://github.com/containers/podman/issues/20839 Do I file an issue about this? herehttps://github.com/ublue-os/bluefin/issues
HikariKnight
HikariKnight•8mo ago
is it a bluefin issue or bazzite issue, if it is an issue in bluefin then start there if you use bazzite then it is here until we can figure out more whats causing it one thing i would like you to try though is make a new user and check if this is a problem on a blank user too at this point 🤔 i always try a new user when there are weird issues that i cant explain as it helps narrow down the scope and worst case i make a new user and migrate data to it
Elijah'
Elijah'OP•8mo ago
oh my bad, I am on Bazzite indeed I've set up a temporary systemd service to delete all the files in tmp, as I cant keep deleting them manually. One dude on Lemmy is suggesting that this is a homebrew issue. So have remove brew now, will see if that changes anything. Nope; folders keep appearing with brew removed.
Elijah'
Elijah'OP•8mo ago
using lsof gives this info, if that is to any help
buzz@fedora:~$ sudo lsof | grep '/var/tmp/'
lsof: WARNING: can't stat() fuse.portal file system /run/user/1000/doc
Output information may be incomplete.
podman 10445 buzz 15w REG 0,41 867465454 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10446 podman buzz 15w REG 0,41 867399918 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10447 podman buzz 15w REG 0,41 867399918 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10448 podman buzz 15w REG 0,41 867399918 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10449 podman buzz 15w REG 0,41 867399918 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10450 podman buzz 15w REG 0,41 867416302 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10451 podman buzz 15w REG 0,41 867416302 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10452 podman buzz 15w REG 0,41 867416302 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10453 podman buzz 15w REG 0,41 867432686 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10454 podman buzz 15w REG 0,41 867432686 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10455 podman buzz 15w REG 0,41 867432686 2315172 /var/tmp/container_images_storage1375523811/1

continues...
buzz@fedora:~$ sudo lsof | grep '/var/tmp/'
lsof: WARNING: can't stat() fuse.portal file system /run/user/1000/doc
Output information may be incomplete.
podman 10445 buzz 15w REG 0,41 867465454 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10446 podman buzz 15w REG 0,41 867399918 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10447 podman buzz 15w REG 0,41 867399918 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10448 podman buzz 15w REG 0,41 867399918 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10449 podman buzz 15w REG 0,41 867399918 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10450 podman buzz 15w REG 0,41 867416302 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10451 podman buzz 15w REG 0,41 867416302 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10452 podman buzz 15w REG 0,41 867416302 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10453 podman buzz 15w REG 0,41 867432686 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10454 podman buzz 15w REG 0,41 867432686 2315172 /var/tmp/container_images_storage1375523811/1
podman 10445 10455 podman buzz 15w REG 0,41 867432686 2315172 /var/tmp/container_images_storage1375523811/1

continues...
wolfyreload
wolfyreload•8mo ago
If you run podman ps do you have any podman containers running? Also iff you run podman system df you can get an idea of how much space podman is using
HikariKnight
HikariKnight•8mo ago
yeah so something with podman causing the issues for you
Elijah'
Elijah'OP•8mo ago
Right so; podman container ls lists nothing podman image ls lists nothing podman volume ls does list 3 different volumes, first being; using podman inspect volume <id>
[
{
"Name": "e22436bd2487a197084decd0383a32a39be8a4fcb1ded6a05721c2a7363f43c8",
"Driver": "local",
"Mountpoint": "/var/home/buzz/.local/share/containers/storage/volumes/e22436bd2487a197084decd0383a32a39be8a4fcb1ded6a05721c2a7363f43c8/_data",
"CreatedAt": "2024-03-15T23:52:10.800764956+01:00",
"Labels": {},
"Scope": "local",
"Options": {},
"UID": 1,
"GID": 1,
"Anonymous": true,
"MountCount": 0,
"NeedsCopyUp": true,
"LockNumber": 1
}
]
[
{
"Name": "e22436bd2487a197084decd0383a32a39be8a4fcb1ded6a05721c2a7363f43c8",
"Driver": "local",
"Mountpoint": "/var/home/buzz/.local/share/containers/storage/volumes/e22436bd2487a197084decd0383a32a39be8a4fcb1ded6a05721c2a7363f43c8/_data",
"CreatedAt": "2024-03-15T23:52:10.800764956+01:00",
"Labels": {},
"Scope": "local",
"Options": {},
"UID": 1,
"GID": 1,
"Anonymous": true,
"MountCount": 0,
"NeedsCopyUp": true,
"LockNumber": 1
}
]
The two extra ones, look similar; let me know if you need them posted too No idea what they are used for, can I safely remove them? I'm assuming this isnt gonna help as I am now running a systemd service to delete them every 5min
HikariKnight
HikariKnight•8mo ago
journalctl -xe -t podman might say what the hell is going on since we tracked it to podman
Elijah'
Elijah'OP•8mo ago
currently not showing anything, but I did also just start the PC up. Do I have to give it some time maybe?
HikariKnight
HikariKnight•8mo ago
maybe
Elijah'
Elijah'OP•8mo ago
should I delete the podman volumes tho?
HikariKnight
HikariKnight•8mo ago
when you see the folders appear i would check the journal
Elijah'
Elijah'OP•8mo ago
there is already one that have appered, but logs dont show nuthin
HikariKnight
HikariKnight•8mo ago
i have no idea then, if the volumes are not important to you then you can delete them
Elijah'
Elijah'OP•8mo ago
other than one single line saying; May 07 19:53:57 fedora podman[5637]: 2024-05-07 19:53:57.049252128 +0200 CEST m=+0.276989737 system refresh I have not created the volumes myself, afaik
HikariKnight
HikariKnight•8mo ago
remove them then
Elijah'
Elijah'OP•8mo ago
just deleted them, but saw another folder just appear after deleting them....
Elijah'
Elijah'OP•8mo ago
No description
Elijah'
Elijah'OP•8mo ago
logs only showing that they were removed
HikariKnight
HikariKnight•8mo ago
ok i have no idea where else to find out anything but if everything is removed now try rm -rf ~/.local/share/containers last thing i would try
Elijah'
Elijah'OP•8mo ago
Like I ever only made 1 distrbox through boxbuddy, beside that I only play games and browse the web.
HikariKnight
HikariKnight•8mo ago
idk if you ever even tried a new account like i suggested to see if its limited to something in your homefolder
Elijah'
Elijah'OP•8mo ago
do I do this with sudo? loads of permission denieds on this one
HikariKnight
HikariKnight•8mo ago
yes, might be caused by some permission issues then maybe, i dont have any rootful distrobox containers so idk where those get placed, but removing that folder will reset everything i k now with podman at least (container wise) im heading out in a few mins
Elijah'
Elijah'OP•8mo ago
okok they keep appearing after having ran this too :<
HikariKnight
HikariKnight•8mo ago
then i really do not know try make a new user which i still dont know if you have tried
Elijah'
Elijah'OP•8mo ago
well yikes that was an experience... So I made a new account to test, as you said. However upon logging into the new account Plasma would crash every 30 seconds. Had to keep starting plasma via terminal. And yes; it keeps happening on the new users as well
HikariKnight
HikariKnight•8mo ago
uh plasma shouldnt crash on a new user
Elijah'
Elijah'OP•8mo ago
No description
No description
Elijah'
Elijah'OP•8mo ago
It was non stop, partial and full crashes (as in sometimes all apps would crash) the second screenshot, that message is the same one as seen in that bug I posted about yesterday it seems.
https://bugs.kde.org/show_bug.cgi?id=453876
HikariKnight
HikariKnight•8mo ago
i think your install has bigger issues than just the container folders tbh at this point
Elijah'
Elijah'OP•8mo ago
Great :)
HikariKnight
HikariKnight•8mo ago
journalctl -xe -t rpm-ostreed last folder on there makes me think maybe look at that but im grasping at straws atm
HikariKnight
HikariKnight•8mo ago
ok i have absolutely no idea then why you have this and nobody else
Pez
Pez•8mo ago
does podman ps show rootful containers without privilege
Pez
Pez•8mo ago
i would also add --external to the flags
No description
Elijah'
Elijah'OP•8mo ago
No description
Elijah'
Elijah'OP•8mo ago
i dont have any containers
Elijah'
Elijah'OP•8mo ago
:>
No description
Elijah'
Elijah'OP•8mo ago
not sure if the systemd-private ones are relevant to this case? but they have also appeared for a while
wolfyreload
wolfyreload•8mo ago
please try sudo podman ps see if it gives you a different list
Elijah'
Elijah'OP•8mo ago
No description
Elijah'
Elijah'OP•8mo ago
none whatsoever :< I'm probably going to do a fresh wipe of my system with a new Bazzite install later today. As I cannot have my system in this state. Even now I canot drag files without KDE crashing on me.
Pez
Pez•8mo ago
best plan
Elijah'
Elijah'OP•8mo ago
Would have been nice to figure out what it was though
Elijah'
Elijah'OP•8mo ago
ran the cmd now, will see if it does anyhing yepp still appears ;DDDD
Pez
Pez•8mo ago
:\ don't run that with sudo whatever you do
Elijah'
Elijah'OP•8mo ago
crazy right i didnt
Pez
Pez•8mo ago
yeah that's a bit concerning actually i would run a fresh install and format the disk from scratch if you hadn't already can't advise sharing disks for different purposes
Elijah'
Elijah'OP•8mo ago
interestingly; the folders now contain more files than usual
No description
Elijah'
Elijah'OP•8mo ago
usually just 1,2 and 3
Pez
Pez•8mo ago
then maybe it actually has room to do something it needed to
Elijah'
Elijah'OP•8mo ago
Is it enough to format it when I install Bazzite again? ticking the box that says "remove partitions if necessary" tickbox?
Pez
Pez•8mo ago
it's kinda acting like it's trying to build an image and it can't because it hasn't got the disk space available
Elijah'
Elijah'OP•8mo ago
Its always had that tbf, I have a script to remove them every 5 min to not clogg my pc
Elijah'
Elijah'OP•8mo ago
No description
Pez
Pez•8mo ago
i would probably manually delete the partitions that were already on the disk and then go back and let it auto-create them well this is good data
Elijah'
Elijah'OP•8mo ago
not sure how to format the disk if im not in an OS
Pez
Pez•8mo ago
how did you install bazzite
Elijah'
Elijah'OP•8mo ago
usb
Pez
Pez•8mo ago
so run the installer and when you get to the partition tool, delete the ones that already existed on the disk and then let it create new ones automatically, should be good enough provided it's an SSD
Elijah'
Elijah'OP•8mo ago
Oh okay yeah
Elijah'
Elijah'OP•8mo ago
found one with readable data
No description
Pez
Pez•8mo ago
don't expect any of this to be meaningful it just tells me it's actually doing something and not spamming nonsense
Elijah'
Elijah'OP•8mo ago
oh ok ok backing up my stuff now;
Pez
Pez•8mo ago
also can you show us what lsblk looks like
Elijah'
Elijah'OP•8mo ago
No description
Elijah'
Elijah'OP•8mo ago
ex is a second ssd does it look ok?
Pez
Pez•8mo ago
yes i am kinda stumped by this one i would maybe checksum the iso you downloaded just to be sure you don't have broken install media
Elijah'
Elijah'OP•8mo ago
dont have that iso anymore, but ill checksum the one im downloading now
Pez
Pez•8mo ago
the button next to the download button has the image checksums
Elijah'
Elijah'OP•8mo ago
Hmm, that button doesnt do anything for me haha; just endlessly loads the browser tab
Pez
Pez•8mo ago
right click > save as?
Elijah'
Elijah'OP•8mo ago
save link as?
Pez
Pez•8mo ago
save link as should work yeah that just saves the url and in this case it's a hardlink to a file anyway
Elijah'
Elijah'OP•8mo ago
got it just so I know for future refrence, what was wrong with the posted pic? How is it supposed to look like?
Elijah'
Elijah'OP•8mo ago
No description
HikariKnight
HikariKnight•8mo ago
they are supposed to be identical which they are
Elijah'
Elijah'OP•8mo ago
yeah, just wanted to confirm that I did as told 😅 if there is nothing else I can contribute with in terms of figuring out what the problem is, I think im just going to do a fresh install.
HikariKnight
HikariKnight•8mo ago
i am completely clueless on what else to provide since i cannot replicate
Elijah'
Elijah'OP•8mo ago
Yeah totally fair that, thanks a lot of the help nonetheless 🌻
HikariKnight
HikariKnight•8mo ago
np! i am sorry we could not figure it out
Elijah'
Elijah'OP•8mo ago
All good, thats how it is sometimes. I still enjoy Bazzite very much and will continue to use it :)
HikariKnight
HikariKnight•8mo ago
i would have loved to have a factory reset function working for atomic desktop but sadly it is very hard to implement
Elijah'
Elijah'OP•8mo ago
haha yeah that does sound like luxury! Soon done with my education and will have much more time to learn more of Linux, looking forwards to that.
Want results from more Discord servers?
Add your server