Immich•2y ago

machine learning keeps crashing

Hello! I'm running immich in kubernetes. Today the machine learning pod keeps crashlooping. The log doesn't say much either

Defaulted container "immich-machine-learning" out of: immich-machine-learning, provide-timezone (init)
INFO:     Started server process [1]
INFO:     Waiting for application startup.
Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configurat
ion.
Downloading pytorch_model.bin:  31%|███       | 189M/605M [02:41<06:00, 1.16MB/s]⏎

Defaulted container "immich-machine-learning" out of: immich-machine-learning, provide-timezone (init)
INFO:     Started server process [1]
INFO:     Waiting for application startup.
Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configurat
ion.
Downloading pytorch_model.bin:  31%|███       | 189M/605M [02:41<06:00, 1.16MB/s]⏎

The pod restarted around this point so I don't know what I should look for. The memory limit to the pod is 4Gi and the usage tops at 850Mi before crashing so I think this is not OOMKilled.

7 Replies