Best tips for lowering SDXL text2image API startup latency?
I'm currently using the https://github.com/ashleykleynhans/runpod-worker-a1111 along with a network volume. I only use a single model with the sd text2image endpoint and I don't need the UI. Right now, I'm experiencing an 80+ second delay for cold startups on the first request. Do you have any suggestions on how to optimize this (without 1 constant active worker)? Thanks in advance!
9 Replies
Or is it better to run comfyui api, is that one faster to boot?
Maybe .. But for me I experience like 2 mins for comfyui cold start
Ok, maybe not then! 😅
It may vary depending on your custom nodes
Do you know how long it takes for it to become cold?
No time specified soo no one knows other than you with your tests
Or other people that tests (idk it might be different too from yours)
Alright, thanks!
I'm also experimenting with both as well. Some says the long latency may also caused by using network volume. I haven't really tried package models in the docker image myself as I kind of need a lot of models and loras, not sure if it's worth packaging in the docker image
Let me know if you find out something useful 🙂 For now I'm keeping the endpoint warm by sending a request every 2 minutes