Continuous File Growth
I have installed Immich few days ago, and only since then backed up my phone, which in total took 3gb, but usage in stats tab went from 20gb to 40gb now, and it keeps growing??
131 Replies
ping me if someone replies
i dont see any errors in any container, i originally thought thumbnails, but like 20gb for 500 files?? and i turned off machine learning features and map & gps
Is there only your pictures on your drive ?
you mean like only me as a user?
yes
and in meantime it went to 42gb
noone even uploaded anything, no jobs are running
i have this in env, is that directory in container or on machine?
on your machine
cant find it
Do you know where are stored on your machine the assets you uploaded with immich ?
uh no, that's kinda what i just asked
you can find your folder with
find / -name libraryfound it, what am i looking for now
Where is it stored, on what disk ?
there's only 1 disk
40.7 GiB is the size used on your disk; 467 is the size available
yes, but my disk was on 20gb last 20 days, since i installed immich, and uploaded 3gb of images, it jumped to 40gb
and it still grows for some reason and noone is uploading anything
Immich creates thumbnails, and encode videos, so even if you upload 3GiB of assets, Immich will store more than 3GiB of files
but 23gb??
du -shc library to know what size Immich usesshouldn't thumbnails be less size than file itself? it would be logical that it's like 6gb, but 23 more lol
thumbnails are light, but videos encoded with h264 can quickly fill your disk
upload dir has 24?
also i noticed that on mobile app, upload bugs out and its stuck on videos often
Do you have jobs running ?
nop
it uses ~30% cpu somehow
Has upload/ thumbs/, encoded-video/, ... within?
Oh I see, so in upload are just files?
also on phone
yeah
video files
The -8 is just a cosmetic bug, we know about that.
Could you open a random one and see if it's in immich already?
i mean by my logic i see some connectivity in video's having hard time uploading from phone and there randomly being 24gb upload folder with just videos
one sec
none is in immich
and they are all random
Hm, those appear to be generated by immich...
Are there files in your encoded video folder?
all empty folders
while we talk it increased for another 1gb
lol
thumbs is simillar? Folder structure but no files?
thumbs has image files
and simillar structure
Which version are you on?
1.18.1
whats most stable one
Can you verify that all original assets are in your library?
um yeah, all 50X files is there
except few videos on phone that are still failing to upload
Ok. I'm assuming that the files in upload are the generated encoded video files (idk how they ended up there lol). I think you should be able to just delete all of them and then run transcode video for all assets again
In order to remove some empty folders you could run the migration job after that's done
i mean they are still continuously generating
Oh, but there are no jobs running?
no
Do you have access to the immich_microservice logs?
And maybe immich_server as well while we're at it
yeah
server
microservices
and my and server time rn is 17:04
or 5pm
Ok, so microservices had no activity in the last time?
seems like, yeah
Just to check the obvious: You don't have multiple instances of immich or the microservices running, right? xD
nah
Could you stop the backup on your phone and check if there are still files being added?
i stopped it, now ig just wait and see if that 44gb changes?
ill let you know in few min
Either that or check if there occur any more files
ill be checking folder size of uploads and library
Btw in the meantime could you look into the app's logs?
which one, there are many
for some reason server is 2h behind
The size reported on the bottom left in Immich is the available and total size on your disk, not the size used by immich
i know, im also checking in terminal
Nice
nothing on whole server makes any disk activity except immich atm
Either stuff that you think could be related or just a lot and we try sieve through it together
all just seem to be idle
Even during the (failed) backup?
they were idle too at the time of disk expanding
yes
wtf.
i had random cpu spikes and i checked if there are jobs each time, there weren't
i think its mobile app causing issues(?) and somehow sending broken videos
and then server just fails to process
yeah logs are quiet like on graveyard
my phone tried to upload those videos since 11am as in logs, and it was just sending failed videos whole this time?
That's my best guessed currently as well. That somehow the app really is screwed up. But no logs?!
nop
Is your app on the latest version as well?
yep
1.81.0
Hmm
1.81.0 build.104
says both are up to date
Yep, that sounds good!
yeah nothing changes with files on server when backup is stopped
So the app seems to be the issue, sending (really) broken files to the server
seems so
I mean it's a long shot but could you try to just re-install the app? lol
lets see
reinstalled, turned on background upload and now at the bottom of backup page, i see 1 video uploading so slowwww
and whole server and phone are literally on same network
what exactly "delay new assets backup" does
Btw which operating system are you on? Not that that would help me personally, but maybe mobile devs would like to know that
android, one ui 5.1
samsung a32 5g
K, thanks!
Do you watch the file system again?
yeah
1 video got uploaded, now second one is, 7%, server storage is same
24gb upload, 2.7 library
and also there's a bug with that "start backup" button
if you swipe to notifications, its state resets and then again sets
and upload process resets?
Funny lol. Maybe you could open an issue for that (if there is none already)? That would be easiest for the mobile devs
dont they look here ig
Yes, but it's easier to keep track of it, mention it in the resolving PR, ask questions specific to that. Here it might get more mixed up and maybe lost
However they may help to figure out the actual issue with the upload * hint hint * :)
yeah
This indicates that the upload is working though, right?
And also the server hasn't done anything stupid just yet
upload works for images perfectly, it breaks on videos
Yes, you mentioned that 1 video got uploaded, right?
i sow it flash from 1 to 100, im not sure
it was like 1 second
Oh, that may indicate that I figured out the asset is already on the server
https://github.com/immich-app/immich/issues/4420 there
i did nothing and it again upped by 1gb
rip
will just turn off backup
Hm, sorry I feel like I don't have anything else...
Maybe @shenlong-tanwen can help? (sorry for the ping)
welp its fine
we found the cause
pretty sure
Just when I caught up with the thread lol
its not 100% sure but lol
i run immich via argo tunnels which have 100mb file limit
i mean button bug is still there so its not for nothing afterall
Ahah it’s often a network issue
The bot should have a networking command... This way we (I) might think of it more often... :P
Just to be sure you can connect to your immich instance with the ip/port of the server
I mean on your phone
Without using your argo tunnel
yes i can
while im here, any proxy y'all recommend?
no apache or nginx
I like traefik, no issues so far
I like nginx a lot but traefik and caddy are good too
yeah i just got reminded why i didnt use proxy in the first place
dynamic ip and server is in local network
im curious can immich just slice videos into smaller parts when sending them over network?
it would be slow, but at least safer and less headache
You can just use dynamic dns and a reverse proxy inside your local network. Even if you public IP changes the web address will still point to the correct public IP.
i have ddns and proxy, but i'd like to point it to cloudflare so i can just use it with dns, and i cant do that since A record requires ipv4, and proxying breaks it too
Not yet
rip
eta?
No
This project has no eta for features
that was sarcasm because yeah i know giving eta's suck
It’s great for entreprises not for foss 😄
true
But you can track GitHub discussions and PRs
i do that already 🫠
I see only two options
Wait or contribute 😜
i mean i might if i actually knew how to do that thing but yeah not an expert in such stuff with media
more just general stuff
At least, reporting issue like you did is really great