Immich duplicate function filtering non-duplicates?
Hi, using Immich since the early days (or are this still the early days?), and a supporter since november 2024.
I have one issue: I think Immich is trying too agressively to find duplicates. It happened a few times this week: when taking multiple photos from a subject Immich doesn't upload all of them. I took 5 photos from a baby: only 3 uploaded/visible in app. Took 10 photos from an LCD displaying different menu items: 6 uploaded/visible in app. The photos are very similar but not 100% equal. What exactly is happening here and how do I prevent this from happening?
10 Replies
:wave: Hey @stijnos,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.It's a visual similarity match (sort of)
You can tune the cutoff in the admin settings
Maybe I'm off but I think he's saying that some aren't uploading even though they aren't exact matches? Which should only happen if the hash of the photos matches exactly.
Not all the photos show up in Immich on my phone, that's what I mean with 'not uploading'.
They do appear in the google photos app.
Are you over reverse proxy or local IP?
Reverse
But everything local
Try connecting over local IP and trying. Just to rule out any funky reverse proxy things.
ok
I now notice some random photos also don't show up in Immich, but which definetely are in the camera album on my phone. Maybe this doesn't have anything to do with duplicates.
No change over local IP.
In 'Backup' on phone i see: Total: 968. Backup: 970. Remainder: 0
Hmm. Strange. It should only reject byte for byte matches. You can try clearing the duplicates in the advanced settings then restarting the upload.
Just to be clear, Immich doesn’t reject or trash near-duplicates. It’ll only ever do that if their checksums are the same