Pre-install questions
Hey folks. Looking to migrate off of Google Photos for my wife and I to a self hosted solution. I've been reading deployment docs and have a few questions.
I'll be installing on an unraid server.
I see the community apps install option and the recommended Docker Compose option. I feel confident in my ability to configure either way but wondering what the pros/cons are of each?
My primary concerns would be ease of management, I prefer the Unraid Docker GUI for managing containers vs CLI, I want to have my install be as easily updatable to the latest builds/features and easy to migrate to another server if I ever move from Unraid to something like TrueNAS
I don't mind running separate containers for Postgres and Redis, I already do for other apps. I'm running Postgres SQL 15 currently, can that be used or does it need to be 14?
Looking to maximize performance with a large library I have both an SSD cache pool and an NVME cache pool. Normally I keep Docker configs and datbases on the SSD pool, wondering if I'll see a difference running Immich on NVME? Any other tuning tips? The library will be on the main HDD array.
I plan to start with just a backup of our Google Photos but I may eventually want to import our older photo libraries from DSLR/Mirrorless (including Raw files) and other large video files, as well as scans of old film photos. Any best practices around adding these to the mix?
78 Replies
The community image is currently not maintained by us
we recommended the docker-compose approach so that everything is isolated to the Immich's configuration and setup, so migration will be easier
on top of that, with docker-compose approach you can easily scale up or down each services
although we haven't run into any bottle neck yet but it is a consideration
FYI, with docker-compose on unraid you can still have a GUI. See the guide at https://immich.app/docs/install/unraid in case you missed that
Unraid | Immich
Immich can easily be installed and updated on Unraid via:
Immich itself is pretty light weight under normal operation, only when you upload it would take the CPU cycle to perform conversion of image and video
PG 15 probably works fine, but currently we're still using 14 in our development setups and don't "officially support" 15.
Having the database on SSD should be fine, you probably don't need to involve NVMe.
thank you both. I assume thumbnails are cached?
Thumbnails are generated during import and stored on disk, and also (usually) cached clientside
Correct
so in terms of maximizing interface responsiveness and load times short of storing all the photos on SSD/NVME how much space should I anticipate for the DB/Cache? My initial import from google photos will be ~300-400gb of photos/videos, but I have terrabytes of other photos and videos I'd like to get in there eventually
We haven't calculated any performance regarding that aspect yet. Largest dataset I test was 114_000 assets and using normal HDD for file storage and running the VM on SSD, I see no issue. Since we design the app for lazy loading anyway so things can work well with limited resources machine
Of course if you are using SSD, you have better disk read/write for initial loading of the asset. After that they will be cached anyway
so for 114k assets, how large was the cache/db files?
I didn't get the measurement 😛
I was testing for performance only
gotcha
let me look at my prod instance with about 20k assets
162MB
not much
I have a user with 360 albums and that took ~10s to load each time on the albums page
thanks for checking!
is the user access your server through dns?
what is the upload speed of your server, where does the user access from?
through DNS, I'm accessing it locally (via DNS)
I'm in Vancouver with 500/500 fiber, user is in Toronto (so no routing issues)
hmm
interesting
but I'm testing this locally
I've never tested that many album 😄
so LOCALLY it takes ~10s to load 360 albums
only assets on the main timeline
😂
probably some work to optimize loading album
honestly I'm not even concerned since he has so many albums he can wait -_-
didn't mean to derail this thread
just wanted to chime in with "if you have lots of albums..."
that is very intersting to know, thank you. Although I can't say where is the problem at 😄
ok, I've got 4tb nvme cache pool (2x2tb raid 0). It is almost empty, only other thing using it is Plex DB/cache. I think I'm going to put Immich appdata and Postgres/Redis on there and let her rip
for google takeout migration this might help https://github.com/immich-app/immich/discussions/1340
GitHub
gphotos import + albums migration · Discussion #1340 · immich-app/i...
gphotos import + albums With this guide, you STILL have to add images that are in multiple albums manually (except for the first instance of that image, so hopefully this isn’t too many) can get al...
you anticipated my next question, thanks 🙂
I haven't even downloaded the google photos takeout files yet. I've been shooting JPEG+RAW for the last couple of years. Is there any trick to retaining the RAW files, how does IMMICH handle JPEG+RAW?
It accepts a certain files
we are working on accepting additional formats
if there is a jpeg and (supported) raw file with the same filename are they just displayed side by side like in google photos?
yeah since they are consider two different assets
unless in the future we have time to work on smart import of raw+jpeg to infer the similar file to only display one
do you know if there is anything in the EXIF metadata to indicate the relation between the two files?
I don't know. I will take a look once I see what Google takeout gives me
I'd love to see a way to hide the raw files somehow while retaining the asset on the server
probably a filter might be easier
* Hide raw file
😄
I expect in the future we'll have a way to filter the main library page based on properties (filetype, applied tags, etc)
in my experience with other apps RAW files take forever to load and can slow down everything but I like to keep them for printing/adjusting the image. It would be sweet if the app were smart enough to not even load the raw files unless you told it to for image editing or export or something
I'm afraid a filter would still have to load the raw files to know which ones to filter out but I'm no programmer so maybe I'm missing something
We don't really load the original files
All that info is in the database
All the loaded files are webp thumbnail
so very small size
And that yeah, even when viewing the files are not the originals
you will only get your original file when you download them 🙂
oh awsome, how it should be!
Yeah our goal is instant loading performance like GPhotos
and User Experience is very important to me personally
Speaking of which, do let us know how performance is once you get your huge amounts of assets in there!
since my wife uses the app, I don't want to hear complaint 😛
Yes please
I'm also curious to see how Google Photos/takeout is going to handle partner sharing. My wife and I have been syncing our photos to each other's libraries for a while but I'm not confident that our entire libraries are identical (not sure if stuff we shot before we turned this on is fully synced, different albums, etc) I am doing a google takeout for both of us just in case. It sounds like IMMICH has some dedup capabilities so not sure how much parsing I should do ahead of time but I will read through the Snuupy sent before I start asking questions on that 🙂
Immich won't dedup between users fwiw, just inside one user's library
As for partner sharing, I would expect one person's photos aren't actually inside the other's library when doing a takeout, but in reality I have no idea what gphotos does there 🙃
can multiple users share a library in IMMICH?
sorry that seems like a RTFM question
You can share an album to multiple users
thanks so if we just put all of our photos in one big "Shared Photos/Videos" album that would solve that, or we could both share one user account I suppose
Album viewer hasn't been optimized for a really large amount of asset yet, more than 10K. So I wouldn't recommended this yet
Thanks, Google Photos partner sharing is quite nice (setting aside the uncertainty of what files live where). It would be nice to have a setting to share entire libraries with multiple users
We are planning to add partner sharing at some point, just haven't gotten around to it
Yeah it one of the feature we would like to have as well
Since it makes sense to have facial recognition in first for that feature to work well
I will look forward to it patiently then. I appreciate all the work and support you guys are doing. Can't wait to get my hands dirty here in a bit
yeah play around with a test instance first and get a sense of how things work before moving to import a large amount of asset to it
let us know if you have any question
Yeah I've got a plenty of free storage so I'll be retaining separate copies of everything that goes in to IMMICH
the true test will be when I start dumping new photos/videos into it instead of google
but I can take my time making that transition
see how the WAF shakes out
Thank you, this will be a very valuable data point for a huge amount of assets to measure performance
🙂
yeah, I'm also hoping to test remote access/SSO via existing CF Tunnel/domain, Traefik, Authentik setup
CF has issue with upload large file
limitation of 100MB I believe
I recall seeing something about that as I was researching, Will large files wait to upload until I'm on local net? Is switching between the two intelligent within the android app?
You will need a split DNS setup to resolve that locally
The app itself doesn't do that
I believe it will hang at the large file if you are using CF Tunnel
I suppose I could just keep it on local DNS and use wireguard tunnel when we are away from home
just the wife and I using it for now
Yeah I am currently using Wireguard
I don't expose my services 😛
home is 2gb/1gb Google fiber so should be plenty fast
Oh yeah should be plenty fast
Yeah I only expose via CF tunnel/Traefik/Authentik stack for stuff like Ombi, Mealie, and a couple others I host for extended family and friends. Plex is the only one I do outside of the CF Tunnel
I've got crowdsec, and some sort of DNS adblock on my list to set up, but photos jumped the line because my wife ran out of storage 🤣
Haha
Well same reason Immich existed in the first place
😛
speaking of, I'm being summoned to help with the kids. Thanks again!
no problem, let me know if you run into any issue
This is a long conversation and i didn't read the whole thing lol, but the get albums request returns all the assets if I'm not mistaken. So it is really 360 albums and all related assets in one request.
That sounds like room for improvement 👀
Yeah, for sure. Some refactoring still to do with regards to albums. Really should have a separate response for a detail API vs list api
Looks like they might actually get sent, but they are loaded from the database always.
GraphQL when? 👀
Did you get this working? I have the same setup, minus CF tunnel. I have no problems signing in on desktop using FQDN with workflow:
But in iOS I am only able to sign in using
http://<local-ip>:<port>/api
If I try to use the domain, Immich can’t connect via user/password, and the Authentik SSO button never appears