I am seeing these errors on load of the photos page.
http://192.168.1.103:2283/api/asset/time-bucket?size=MONTH&isArchived=false&timeBucket=%2B060379-10-01T00%3A00%3A00.000Z 500 (Internal Server Error)
Ans some other related ones. All servers are running fine in docker. Any ideas?
189 Replies
What do the server container logs say?
And also, could you post your docker compose file?
Those are old logs. Could you show the most recent - right after you tried to open the page
Those are both just errors that occur from trying to access the server while it isn't started yet
this was grabbed right after reloading the page
hmm but the time stamps are wrong - the dockler log seems very shitty
there is nothing in the log for today???
2023-10-12 14:35:17. is the last entry confused
when I open the immich_server log screen in docker, the very bottom line is this...
"2023-10-12 15:08:53 Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close error from daemon in stream: Error grabbing logs: invalid character '\x00' looking for beginning of value" so it seems to be erroring while trying to grab logs - great
"2023-10-12 15:08:53 Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close error from daemon in stream: Error grabbing logs: invalid character '\x00' looking for beginning of value" so it seems to be erroring while trying to grab logs - great
Are you using portainer only for maintaining/watching the stack? Or did you deploy Immich using portainer in the first place?
its dockeer desktop on a mac - not sure what portainer is
Oh I see, sorry. I was just seing docker UI and were therefore assuming it must be portainer
my install has been running fine for about a week
Could you try running
docker logs immich_server?
See if that returns more logs and doesn't error out with "Error grabbing logs"
Could you get the logs of immich_microservices as well?sure
Why are all those logs so old?
Could you just try bringing the stack down and up again?
And also: I don't need 250k lines of logs normally :P
Just the last 50-100 lines should be enough
which method is best to bring stack down and back up again?
yeah its weird the logs are all old its liek logging is broken
In the folder where you've placed the
docker-compose.yml: docker compose down and then docker compose up -d should be enough
Was your docker instance running the whole time and just stopped working? If you're running it on a mac, is that running 24/7?its running 24/7 yes but i have restarted the mac in the last 24 hrs
Hm. The docker implementation on macos is pretty bad tbh, so it may just be that it had a hiccup and restarting it as described above fixes it
ok it went down and up cleanly
this is the bottom of server log
This is all just fine while it's still starting
ok
let me see if the errors are still present in the front end
The frontend probably won't be there just yet
Just give it a minute to let it start up
oh - it was and those errors are same
ok so server log...
so its this time displacement thing again
Could you try re-running metadata extraction for all assets?
Alex posted about that yesyerday but i cant find the thread becuase i am a noob at discord
Extract Metadata or "sidecar metadat"
extract metadata
And then click on "all"
This may take a while though
How many assets do you have?
lol yes.
Oh wow.
large library
So yeah, this will take a while
yes that is the one... @Daniel does that thread help with any insight?
Unfortunately, we don't maintain the immich-go export cli
Did you try the fix that is described in the thread?
I didnt use immich go
i used nodejs and the immich documented CLI process
Did you export those images from google photos?
I didnt see any fix - that thread was a little above my pay grade
no - these images were imported over many many days from external hdds
plugged directly into the mac that is hosting immich
Ah I see, then the thread is completely unrelated unfortunately :/
thus using the immich upload command
ok
Yep, that's perfectly fine!
so do i run this massive extract metdata job?
It appears at least one of your assets has really broken metadata
I'd like to try something else first
yes sure
i am gussing the borken asset is in teh viewport at the time i see the error
bcuase i can scroll through the rest and i ddont see it
and the "Rest" are not loaded until scrolling down to them
Exactly. It will be fetched and while trying to do so, the database breaks because that asset was apparently taken in the year 60379 xD
ok so how can i find it in the dom
I'm assuming you haven't changed the storage template, correct?
nope
Then your library should be sorted into folders by year
So, if the asset got imported successfully, it should be there
bingo
Perfect!
60379
Yes!
Don't delete it just yet
ok so there is only one pic in that folder!
K, so that seems to be broken
yeah but i am seeing two x the error msg
anyway lets clean this one out first
how should I do it
Yes, just give me a second
find it in the UI and remove frm there maybe?
I mean that would be easy but you won't find it in the UI since it can't be fetched
ahh
true
if you have a mobile application linked to your immich server, you are able to delete the offending image
Ok, so could first run
docker exec -it immich_postgres psql immich postgres. This should give you a database shell. Inside that, run SELECT * FROM "assets" WHERE "originalPath" LIKE '%60379%';Yes!!! I see the offending immch at the top of my mobile app
Oh, that's even easier haha
so should i use the app to delete it?
Try it, sure
Wait
Before doing so, you should somehow remember which asset that is
So that you can fix it and re-upload
there it is in the app
october 60379
the file is the 60379 folder
So you could just copy the image from the library folder to save it. Then delete it
i dont care about the image it seems to be a thumbnail its like 100px by 300px
not a video?
Why would it need to be?
Yeah ok, screw that haha!
yeah def a thumb
Then just delete it from the app
ok i hav innitated the dekete on the app and i am seeing a spinning wheel - lol
its not liking it
Hm
So maybe we have to go the database way then
Does the server throw errors again?
nope - its 07 psa the hour now
and last entry is at 17:00
thats immich_server
Yeah ok
Then just close the app
And try this ^^
ok app recovered - image is still there
will try db edit
in terms of what like an ID number?
Hm?
I am not sure how assets are identified
are they given unqiue IDs?
This should return you a table with many information
Can't you just run the query I gave you?
And yes, they are
You aren't supposed to delete it just yet
oh i was resposnding to "wait before that remember what the asst is"
will do db stuff now
i was scrolled up and didnt realise - ignore me
Oh I see. Yeah that was only to not have you search through all your files again to find that picture and edit it. Since it isn't relevant, it doesn't matter
Haha I see
Btw you can reply to messages by hovering over them and clicking on the the arrow (for messages you haven't written yourself)
~ % docker exec -it immich_postgres psql immich postgres
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "postgres" does not exist
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "postgres" does not exist
Did you change anything in your .env?
nop
Could you paste it then? (with confidential stuff erased)
sure
you can get the list of docker containers with the following command
eww - how do i put text in code mode
The container is running, otherwise the message would have been different
Three backticks
Also fine :P
So the username is postgres, wtf
i didnt set that
why is that suprising?
I was assuming the userame would be postgres, hence the command
But it failed, stating that it doesn't exist
Which doesn't make sense, since it apparently should exist
Could you try adding a
-U before "postgres"?
So docker exec -it immich_postgres psql immich -U postgres///~ % docker exec -it immich_postgres psql immich -U postgres~
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "postgres~" does not exist
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "postgres~" does not exist
Why is there a ~ at the end?
docker exec -it immich_postgres psql immich -U postgres
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "postgres" does not exist
Oh wait
Do you also have a .env file?
yes i posted it earlier
No, that file is called
example.env
The file must explicitly be called .envoh lol
sorry
Which you won't see by default in the finder btw
Easiest would probably be to to
ls -la
Don't rename the example.env now thoughwhen i do a ls - la. i cant see .env
You're in the right folder?
ls
ok got it
was in the wrong folder
😉
whats the comand line to print it
cat .envDB_HOSTNAME=immich_postgres
DB_USERNAME=postgrescar
DB_DATABASE_NAME=immich
There we go!
The user is postgrescar and not postgres
Idk why but whatever lol
Then try
docker exec -it immich_postgres psql immich postgrescar% docker exec -it immich_postgres psql immich postgrescar
psql (14.8)
Type "help" for help.
So you've got a new shell now, correct?
You could try executing
\d
Now we're getting somewhere!
SELECT "originalPath", "resizePath", "webpPath" FROM "assets" WHERE "originalPath" LIKE '%60379%';
Try this!🙂
immich-# SELECT "originalPath", "resizePath", "webpPath" FROM "assets" WHERE "originalPath" LIKE '%60379%';
ERROR: syntax error at or near "docker"
LINE 1: docker exec -it immich_postgres psql immich -U postgrescar
Just try again, maybe you have entered something before and missed a ';'
K nice
The seconds line is irrelevant, I was just too lazy and gave you a non-perfect query :D
immich=# SELECT "originalPath", "resizePath", "webpPath" FROM "assets" WHERE "originalPath" LIKE '%60379%';
originalPath | resizePath | webpPath
-------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------- upload/library/admin/60379/60379-10-07/t19240512.jpg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.jpeg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.webp upload/library/admin/2017/2017-03-08/01ce67fd6460fe6f3895e6037951db0fd7a9516f3e.jpg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/37/73/3773392c-9d51-47fd-9c38-8db9276fe733.jpeg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/37/73/3773392c-9d51-47fd-9c38-8db9276fe733.webp (2 rows)
-------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------- upload/library/admin/60379/60379-10-07/t19240512.jpg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.jpeg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.webp upload/library/admin/2017/2017-03-08/01ce67fd6460fe6f3895e6037951db0fd7a9516f3e.jpg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/37/73/3773392c-9d51-47fd-9c38-8db9276fe733.jpeg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/37/73/3773392c-9d51-47fd-9c38-8db9276fe733.webp (2 rows)
SELECT "originalPath", "resizePath", "webpPath" FROM "assets" WHERE "originalPath" LIKE '%60379/%';
This should fix it and only return one roworiginalPath | resizePath | webpPath
------------------------------------------------------+----------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------- upload/library/admin/60379/60379-10-07/t19240512.jpg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.jpeg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.webp (1 row)
------------------------------------------------------+----------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------- upload/library/admin/60379/60379-10-07/t19240512.jpg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.jpeg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.webp (1 row)
So those are all files that belong to the asset we're about to delete
You can remove all those three files from the file system to avoid orphans
You can also remove the parent directories if they are empty
So e.g.
upload/library/admin/60379/ is gonna be empty after you delete the jpg. within, so you can delete the folder as well2 secs - phone call
ok back..
so we know the file paths
and i need to remove the assets so they are not orphaned.. I got that.. But dont we also need to remvoe relevant records from the db?
@Daniel are you still there?
do i run a command to simply delete that row?
Sorry, I'm back now
Yes, I just wanted to do it in small steps :)
ok so should i remove the assets first?
have not done that yet
I would recommend that, yes.
Because after we've deleted the rows, we have no (easy) way to find the orphaned files anymore
ok those three assets have been removed from teh file system
You can also remove directories that have become empty. Did you do that as well?
yes
Alright, then let's run
SELECT "originalPath", "resizePath", "webpPath" FROM "assets" WHERE "originalPath" LIKE '%60379/%'; one more time to ensure that it really only selects that one assetside note - MacOS is very annoying
how people use it I will never know
Haha, what's wrong? I mean I use macos myself and I'm annoyed by it sometimes too, but it isn't too bad either
just file management is dumb compared to windows... cant easily see and copy paths for example
cant alt-tab between file explorer windows becuase finder is just one application
Oh wait, you prefer windows over macos? Windows is awful tbh
windows has come a long way in the past few years
Yeah ok, there are apps for that. Or just use a shell lol
anyway lets remove that entry from teh db
However, I do agree that Windows has upsides compared to other OS, I just hate that it doesn't have a proper shell and stuff
immich=# SELECT "originalPath", "resizePath", "webpPath" FROM "assets" WHERE "originalPath" LIKE '%60379/%';
originalPath | resizePath | webpPath
------------------------------------------------------+----------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------- upload/library/admin/60379/60379-10-07/t19240512.jpg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.jpeg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.webp (1 row) immich=#
------------------------------------------------------+----------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------- upload/library/admin/60379/60379-10-07/t19240512.jpg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.jpeg | upload/thumbs/27e86adf-a7a0-4a87-88b1-b341f23d504e/57/94/57946aba-da5a-4e57-aad6-5118f0f61603.webp (1 row) immich=#
You can do this:
DELETE FROM "assets" WHERE "originalPath" LIKE '%60379/%';yeah but putty is for that
oh you mean for its self - yeah well its got powershell?
I know it got powershell... And I feel like it isn't that bad. It just isn't as good as a bash or zsh imo
Disclaimer: I've never really used current powershell version though
immich=# DELETE FROM "assets" WHERE "originalPath" LIKE '%60379/%';
DELETE 1
immich=#
seemd to work 🙂
Nice one!
so now - i guess lets see if the front end is reporting errors still
Yep
Maybe you need do restart the containers again, let's just see...
Can't really tell how broken it has gotten lol
winning.
no errors!!
Oh!
Nice!!
Well done! :)
Looks really good!
Containers are also happy now?
"no issues" in console
let me see
how do i exit the postrges session
ctlr c not working
Ctrl d
only error i see now in server log is [Nest] 7 - 10/15/2023, 11:25:53 AM ERROR [ExpressAdapter] Premature close
i'll do a doan and up
Yeah those are ok
down and up command
Sure, go ahead
However I really think we've done it!
hey @Daniel thank you so much for your help and you too @bo0tzz
No problem :)
this is why open source is so great
teamwork and accessabiltiy to information to fix shit
thanks @Alex for this project
We should think about support contracts in a oracle fashion :P
haha
that would ahve been $5k thanks
check is in the mail
🚀
To be serious though, always glad to help people getting started with and using Immich!
Feel free to reach out if you should run into any other issues in the future!
cool - can I ask you - do you know if resizable thumbs is in the road map
What do you mean by that?
like a thumb size selector - small medium large
Oh, there already is
You can do that in the admin settings
i run a 42" OLED display and the thumbs are just huge
Oh, you mean the grid size in the ui?
i want them about 50% of the size they are
yea
Not sure if that can be changed
grid size in ui
That's a question for @Alex I think
here it is in another app - see the slider in the top right
showing small to large variance
ok @Alex will see this question here I guess 🙂
Three pings now... poor Alex :D
ok i shall now resume uplaoding the remaining 40,000 assets to my library
haha
thanks again guys
Sounds great. Enjoy Immich!