Errors after the latest update
Setup: Docker Swarm on multiple nodes (including a Synology NAS)
EError log from immich_server
Emessage.txt3.98KB
Emessage.txt4.16KB
ACan you post your setup? Also if you bring down and bring up the whole stack again, would it help?
EI tried that multiple times, after prune buit that didn't seem to help
AAh it seems to have an issue with migrations
A@Zack @jrasm91 do you guys have any pointers?
ZThat's very odd
ZThe database is somehow in an unexpected state, where shared_links doesn't exist where it is expected to
ZWait... That's saying the entire shared_links table doesn't exist
Z@ExtremeInspection have you ever gone into your database and manually changed anything?
ENope. I'm not that adventurous 
ZWell I'm really not sure how that table could be missing
ZDo you know how to connect to a database and run queries?
EYes I can do that
ZIf you could connect then run describe on the shared_links table
ZI believe is shorthand for that which should work
ZMake sure you're in the immich database context before running it
EDid not find any relation named "shared_links".
ZYea, that makes sense...
ZHmmm
JCan you restart the stack?
ZWhat version did you just upgrade from?
ZThey've already done that, if TypeORM is trying to apply this migration, it's because it believes all previous ones have been applied correctly
JMaybe try starting only the server container
ZBut somehow the shared links table is missing
ZCould you run for me?
EI'm not too sure.. I have an auto-update set up with Shepherd.. I can look up the version with the signatures, if it is important
ZIf you've been keeping up with the versions, it's not that important. I was just wondering if it jumped a bunch of versions at once somehow
ZAlthough even that shouldn't cause a problem
Eimmich=# \d user_token
ERROR: cache lookup failed for relation 16410
ERROR: cache lookup failed for relation 16410
ZUhhh
ZThat's a scary error, lemme look up what that means
ZCan you shut down the server and microservice instances then try run the command again?
ZCould you also retrieve the postgres server logs?
ZI notice you are using NFS for your underlying filesystem. Is this setup with async or sync on fstab?
EJust tried... Same error
Eimmich=# \d user_token
ERROR: cache lookup failed for relation 16410
ERROR: cache lookup failed for relation 16410
ZFYI I have about 10 minutes before I need to head off
ZThis one is the most important question
Enfs:
driver: local
driver_opts:
type: "nfs4"
o: addr=10.20.11.5,rw,nolock
device: ":/volume1/Docker-Data/immich"
driver: local
driver_opts:
type: "nfs4"
o: addr=10.20.11.5,rw,nolock
device: ":/volume1/Docker-Data/immich"
ZHave you had any issues with the server hosting the NFS share going down, or the connection between the servers being unstable?
ZIt looks like the connection is over your local LAN so probably only the former is relevant
EDouble checked on the server side.. It is set to async
ZAh
ZAsync makes things faster but is much more prone to corruption or data loss
ZDue to it not finishing writing to the actual server before responding saying everything is done
ZBasically, what I think has happened is some kind of data corruption. I'm not sure how else the database could be in this state. Our migration software believes it has created this table and is on a later script. postgres stores tables in seperate files, so I believe it's possible that the migration table wasn't affected but the shared_links table has been deleted in the underlying storage
Emessage.txt98.86KB
ZI'm not sure how else to diagnose this really. My suggestion would be to start fresh and re-import your assets (and probably don't run postgres over NFS)