Can't setup a1111 on serverless.. Service not ready error
Hi guys, I am wondering if anyone managed to setup a1111 on serverless RunPod without network volumes. I am using the blog post https://blog.runpod.io/custom-models-with-serverless-and-automatic-stable-diffusion/ with the https://github.com/runpod-workers/worker-a1111 but I can't seem to get it to work. I managed to build the image, createthe template and endpoint but I get a Service not ready yet. Retrying... error from a Request Exception. I also mounted the image on a container locally and tried to send a request only to get an error (56) Recv failure: Connection reset by peer which seems lke the server closes the connection. Anyone had success with this?
RunPod Blog
Create a custom AUTOMATIC1111 serverless deployment with your custo...
Have you ever wanted to create your own serverless AUTOMATIC1111 endpoint with a custom model that can scale up and down? Now you can do so without much hassle by following this guide!
Pre-requisites
A computer (local or cloud) with the following:
* docker installed
* git installed
* a relatively fast upload
GitHub
GitHub - runpod-workers/worker-a1111: Automatic1111 serverless worker.
Automatic1111 serverless worker. . Contribute to runpod-workers/worker-a1111 development by creating an account on GitHub.
32 Replies
PS. my end goal is to setup a1111 for api use with my custom model saved in the docker image.
You can try this one instead:
https://github.com/ashleykleynhans/runpod-worker-a1111
GitHub
GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...
RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111
maybe you're using a wrong docker link?
for the image*
thnx will try this one as well
what do you mean?
Maybe the images broken too try that Ashley's template first
Did that recv failure error happened inside the container?
no that was the error I got when I tried to send a curl request to the container from my local ubuntu machine
Connection reset by peer shouldn't ever happen even if your endpoint isn't working properly. The requests should still go into the queue.
the serverless implementation on this is with Network Volumes... is there a way to not use Network volumes and instead include the image in the docker image?
Oh local container yeah? Maybe the port isn't open yet
the endpoint request is going into the queue but I get the Service not ready error
Have you tried the Ashley's template on endpoint?
Nope, that one only works with network volumes, unless you want to modify the code to make it work without them.
not yet, I reading through it now and it seems to be based on Network Volume which I don't want. :/
Hmm yeah try with that network volume or modify the code
You can fork it and make it work without network volumes but probably easier just to get the RunPod one working then.
I never tried local testing BTW so i can't really hekp
It comes with sdxl model isn't it
You can install whatever models you want.
There are some examples installing models, ControlNet etc
yes will look today into creating a fresh implementation without Network Volumes
The base premade template *
I've already done a Network Volume implementation previous and it works, I get problems with the non-network volume ones.. :/
anw will jump right into it
thanks for your comments
Alright
Read the Readme and the scripts on github
You'll soon figure something out for it
will do thnx
2024-05-18T12:36:03.725845977Z Service not ready yet. Retrying...
2024-05-18T12:36:03.929343220Z Service not ready yet. Retrying...
2024-05-18T12:36:04.111961339Z Traceback (most recent call last):
2024-05-18T12:36:04.112013597Z File "/stable-diffusion-webui/webui.py", line 31, in <module>
2024-05-18T12:36:04.112020304Z import gradio
2024-05-18T12:36:04.112025577Z File "/usr/local/lib/python3.10/site-packages/gradio/init.py", line 3, in <module>
2024-05-18T12:36:04.112030334Z import gradio.components as components
2024-05-18T12:36:04.112034719Z File "/usr/local/lib/python3.10/site-packages/gradio/components.py", line 55, in <module>
2024-05-18T12:36:04.112039137Z from gradio import processing_utils, utils
2024-05-18T12:36:04.112043804Z File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 517, in <module>
2024-05-18T12:36:04.112104181Z class AsyncRequest:
2024-05-18T12:36:04.112135458Z File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 536, in AsyncRequest
2024-05-18T12:36:04.112142558Z client = httpx.AsyncClient()
2024-05-18T12:36:04.112148714Z File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1397, in init
2024-05-18T12:36:04.112545061Z self._transport = self._init_transport(
2024-05-18T12:36:04.112626920Z File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1445, in _init_transport
2024-05-18T12:36:04.112766887Z return AsyncHTTPTransport(
2024-05-18T12:36:04.112789494Z File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 275, in init
2024-05-18T12:36:04.112794809Z self._pool = httpcore.AsyncConnectionPool(
2024-05-18T12:36:04.112799462Z TypeError: AsyncConnectionPool.init() got an unexpected keyword argument 'socket_options'
2024-05-18T12:36:04.132782329Z Service not ready yet. Retrying...
2024-05-18T12:36:04.336979246Z Service not ready yet. Retrying...
this is what I am getting no matter what I do... 😦
can't seem to get it to work
tried all other repos as well with network volumes
I'm about to give up on RunPod
going to replicate
Which one is this? The RunPod one or the ashleykleynhans one? Looks like its the RunPod one using an outdated version of gradio or something.
I don't think the RunPod one is maintained, so you probably need to update a few things if you want to use it.
That means the service isn't ready yet
Service will never become ready due to the gradio issue
Oh yeah is that the ashleyk's repo/template?
Probably the RunPod one
dependabot broke it
GitHub
worker-a1111/Dockerfile at main · runpod-workers/worker-a1111
Automatic1111 serverless worker. . Contribute to runpod-workers/worker-a1111 development by creating an account on GitHub.
Its fixed at a specific commit SHA, but @Merrell merged a dependabot PR to bump gradio to an incompatible version.
It will probably work if this PR is reverted:
https://github.com/runpod-workers/worker-a1111/pull/29
GitHub
Bump gradio from 3.41.2 to 4.11.0 in /builder by dependabot[bot] · ...
Bumps gradio from 3.41.2 to 4.11.0.
Changelog
Sourced from gradio's changelog.
4.11.0
Features
#6842 846d52d - Fix md highlight. Thanks @​pngwn!
#6831 f3abde8 - Add an option to enable head...
By the way @xnorcode , diffusers is actually far more performant than A1111 in serverless if you're only going to be using a single model and baking it into your image.
@xnorcode there is a PR to fix the RunPod standalone one:
https://github.com/runpod-workers/worker-a1111/pull/37
You can use the fork until its merged:
https://github.com/ashleykleynhans/worker-a1111
GitHub
Various improvements & fixes by ashleykleynhans · Pull Request #37 ...
Bump A1111 to latest 1.9.3 release.
Use A1111 installation scripts to install everything instead of installing manually.
Removed redundant code.
Properly implement libtcmalloc.
Implemented xformers...
GitHub
GitHub - ashleykleynhans/worker-a1111: Automatic1111 serverless wor...
Automatic1111 serverless worker. . Contribute to ashleykleynhans/worker-a1111 development by creating an account on GitHub.