automatic serverless api slow response problem
it is taking more than 2 miniutes to generate an image using this https://github.com/ashleykleynhans/runpod-worker-a1111
17 Replies
@ashleyk hi , i have tried your repo https://github.com/ashleykleynhans/runpod-worker-a1111
with stable diffusion but it takes two miniues to serverless to generate image. how can i make this faster?
starting webui is still taking lot of timeI am facing a similar issue. The request is stuck in the queue for well over a minute before it starts to process -- even when there is no other request. Any advice would be greatly appreciated.
Update. The issue is starting cold. I adjusted the idle time to 30 seconds. After the first request, the next request only stayed in the queue for 163ms.
I don't know if there is another option -- but that could get costly if you have to keep it awake for long periods of time...
Yeah, that is not suitable for me as I don't have a lot of requests coming, and waiting 30 seconds would cost too much.
I am trying this repo that has a caching system, but I am encountering a dependency errors.https://github.com/runpod-workers/worker-a1111
Yeah, if you need it up 24 hours a day, that's $17.00 a day.
I just found Flash Boot on the endpoint. Now starting almost immediately
Have you gone through this link https://github.com/ashleykleynhans/runpod-worker-a1111 to create a Docker image with network volume?
yes
please can you help me?
i get these message for sometime until finally webui run successfully then request is processed
turn flash boot on
it's already on for me
this is my settings
request are being processed after some time like this
the network volume
is prob the thing causing slowdown
also inherently a111y has overhead too
if u send three requests sequentially
the last two should be faster?
if so the first one which is loading the model up for the worker is demonstrating the initial cost to start a111y and load the models to memory
would it load faster if everything was bundled into the docker image, and not use a network volume at all?
Yes but also A1111 is a bloated monstrosity and takes a long time to start up even without a network volume, so you will save some time, but it will still be slow.
ah true, what alternative would you recommend for img2img inpainting?
diffusers is more lightweight but I don't have any experience with it and don't know whether it supports that
ok thanks