Immich•2y ago

Can I download the machine learning files and host it on my pc?

Hi! I’m having an issue where the machine learning docker will refuse to start up (i’m guessing the computer i’m hosting immich on is too slow) and thus immich won’t start up either without it. Commenting it out in the yml fixes it and lets immich run, but i’m missing out on a huge part of why I wanted to get immich in the first place lol. Instead of it trying to download it from a remote server every time the docker starts up, is it possible to self-host it? Any help is appreciated, thank ya! (I’m very inexperienced with this sort of thing, so I may need to ask follow up questions!)

13 Replies

JackOP•2y ago

Side note, I tried leaving the docker on overnight to see if it would download and it was still on the same screen when I woke up, so unless it’s supposed to take forever to download, i’m stuck 😂

IVPathfinder•2y ago

while someone else might guess what's going on. It might be good to include some logs 🙂 docker logs immich_machine_learning

JackOP•2y ago

2023-07-16 05:30:22 INFO:     Started server process [1]
2023-07-16 05:30:22 INFO:     Waiting for application startup.
2023-07-16 05:30:26 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 05:43:19 INFO:     Started server process [1]
2023-07-16 05:43:19 INFO:     Waiting for application startup.
2023-07-16 05:43:27 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 05:54:32 INFO:     Started server process [1]
2023-07-16 05:54:32 INFO:     Waiting for application startup.
2023-07-16 05:54:35 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 05:59:44 INFO:     Started server process [1]
2023-07-16 05:59:44 INFO:     Waiting for application startup.
2023-07-16 05:59:47 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 06:19:18 INFO:     Started server process [1]
2023-07-16 06:19:18 INFO:     Waiting for application startup.
2023-07-16 06:19:22 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.

2023-07-16 05:30:22 INFO:     Started server process [1]
2023-07-16 05:30:22 INFO:     Waiting for application startup.
2023-07-16 05:30:26 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 05:43:19 INFO:     Started server process [1]
2023-07-16 05:43:19 INFO:     Waiting for application startup.
2023-07-16 05:43:27 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 05:54:32 INFO:     Started server process [1]
2023-07-16 05:54:32 INFO:     Waiting for application startup.
2023-07-16 05:54:35 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 05:59:44 INFO:     Started server process [1]
2023-07-16 05:59:44 INFO:     Waiting for application startup.
2023-07-16 05:59:47 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 06:19:18 INFO:     Started server process [1]
2023-07-16 06:19:18 INFO:     Waiting for application startup.
2023-07-16 06:19:22 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.

2023-07-16 06:27:22 INFO:     Started server process [1]
2023-07-16 06:27:22 INFO:     Waiting for application startup.
2023-07-16 06:27:31 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 06:59:15 /opt/venv/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:65: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
2023-07-16 06:59:15   warnings.warn(
2023-07-16 07:00:28 INFO:     Started server process [1]
2023-07-16 06:59:37 Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
2023-07-16 07:00:28 INFO:     Waiting for application startup.
2023-07-16 07:00:31 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.

2023-07-16 06:27:22 INFO:     Started server process [1]
2023-07-16 06:27:22 INFO:     Waiting for application startup.
2023-07-16 06:27:31 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
2023-07-16 06:59:15 /opt/venv/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:65: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
2023-07-16 06:59:15   warnings.warn(
2023-07-16 07:00:28 INFO:     Started server process [1]
2023-07-16 06:59:37 Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
2023-07-16 07:00:28 INFO:     Waiting for application startup.
2023-07-16 07:00:31 Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.

those are the logs for the past hour and a half, i uncommented it in the yml/esv and let it run for a bit

bo0tzz•2y ago

Those logs look pretty normal apart from the looping. Could it be that the container is being killed because of running out of RAM?

JackOP•2y ago

it could be possible, i'm allocating basically everything to docker though so i have no clue why it's refusing to load lol

bo0tzz•2y ago

How much RAM do you have?

JackOP•2y ago

4 gigs on the one i’m running it off of. its not much but since i’m not using it for anything else i have it all it can spare allocated to the docker

bo0tzz•2y ago

Hmm, that might be just low enough that it will run out while starting up :/ Try setting MACHINE_LEARNING_EAGER_STARTUP=false, that should at least let it start up Though you might still run into problems when you start running the ML jobs

Strider•2y ago

I have similar question: can i host ML container on my pc with gpu and other containers (non gpu intensive) on another machine like RPi4? So that i keep RPi4 running all the time, but fire up my pc once a day to finish processing ML jobs.

Alex Tran•2y ago

yes you can

bo0tzz•2y ago

GPU isn't supported yet, right? But the rest is possible

meh•2y ago

Check out this thread, I'm doing same, running ML on a different machine. https://discord.com/channels/979116623879368755/991643870523830382/1126919469617397880

JackOP•2y ago

this is actually very helpful, i didn’t realize you could offload the job to another pc! instead of trying to figure out how to get it working on my (relatively slow) immich setup, i’ll just send all the jobs to my main pc with 16 gigs of ram LOL thank you so much!

Gaming

Programming

Can I download the machine learning files and host it on my pc?

Did you find this page helpful?