Video Upload Stop During cli Process
While running the command immich upload --key REDACTED --server http://immich-server:3001 --recursive /usr/src/app/photostoupload2, the process stops in the middle and does not complete upload of all found videos. Immich is installed using the text scripts for Portainer and has mostly default settings (minor edits). I've also tried pausing all the Jobs to ensure background jobs aren't causing issues. I was able to successfully upload over 80,000 images previously. I have tried with a few different sets of videos to determine if it was a certain video causing the issue but it doesn't appear to be. The videos are all locally on the server running immich. Attached is a screenshot of the output I see when it appears to fail.
29 Replies
how big is that Video?
482,000 KB
so not big at all
even uploading over the web doesn't work?
And that wasn't the only one it stopped on
I'm sure it will... do you think it failed on that video or the next one?
I can hunt that one down the try the ui
it should have failed on the reported one
I tried that the first time, took a video out of the folder and then it failed on a different one 🙂
but I will try the UI for this one
just a hunt, can you try rename the folder name and video name without space?
yeah I'll try that in a few, gotta jump on a work call
No problem, thank you for reporting
Same thing when I changed the folder name to not include the space
Trying that video in the ui
No error message popped up and it looks like it uploaded
Just jumping in, this looks similar to an issue I have as well. How do you know it did not upload those 4 assets it found? 4/1659 is almost 0%, so I think it is indicating it uploaded the 4 that it found from the total number of assets.
I have a similar problem where it will find the same 562 assets to upload... but im not sure where to check which ones https://discord.com/channels/979116623879368755/1126199582406221986
I've tried a few different folders to nail down what it is doing. I couldn't even figure out the order of upload.
Is there a log on the server that shows these failures? I can't seem to find one (assuming its failing)
It could be that these video files were potentially in another folder I already uploaded but I can't easily tell if that is the case since the output doesn't say why it stopped.
Yeah, since there is no output, my assumption is that that it uploaded everything correctly and maybe it can't tell that i uploaded it, so it keeps finding the same ones to upload. But it is hard to pinpoint a single missing asset in my case
it doesn't appear that the search feature allows you to search by file name... but I just installed immich last night so I'm not 100% sure
You can search for filename with "m:filename.mov" At least it generally works for me.
yeah my install is pretty fresh as well. In my case the mp4 file that is the last one that looks like it got uploaded is not found when I search
I'm going to try the UI to upload again, since I'm not moving a bunch of videos at the same time now
I wonder if this will be addressed in the CLI changes Alex mentioned in my thread. It is a bit concerning not know if there is an issue here, I would be more comfortable if it was clearly failing or something.
Same here
The CLI is going through a complete rewrite actually, that is why we are not spending much effort in the current version anymore
Something to let us know where/why it failed, which I'm sure Alex wants as well 😄
yeah the CLI was just a "quick/drity" work to get things to Immich without too much design/thoughts putting into it
the new CLI tool has just been merged to main and we will start adding more feature to make it mature from there
Thanks Alex, appreciate the response.
I think the reason the upload was ending early is that the video files were already uploaded from a previous upload. I'm not 100% sure but after some further testing maybe.
I did notice that when I changed my search, it found a very similar mp4 file already uploaded (the only difference was the file name had a number appended to it). So I am wondering if the upload is somehow checking if the files are duplicates (beyond just the file name)?
yes the server has hashing mechanism to detect duplicate file
and so reject the duplicate file when it get to the server
the new CLI will have client-side hashing as well AFAIK
ah, I wonder if that is what is happening then, thanks Alex!