RunPod•6mo ago

how to deploy custom image gen model on serverless?

https://blog.runpod.io/custom-models-with-serverless-and-automatic-stable-diffusion/ i read this but can i do it without the a1111? would it make it more simple to set up? i want to deploy a custom image model for image generation on serverless. i plan on just using the sdxl API.

RunPod Blog

Create a custom AUTOMATIC1111 serverless deployment with your custo...

Have you ever wanted to create your own serverless AUTOMATIC1111 endpoint with a custom model that can scale up and down? Now you can do so without much hassle by following this guide! Pre-requisites A computer (local or cloud) with the following: * docker installed * git installed * a relatively fast upload

Solution:

For A1111, this one is better, its more up-to-date: https://github.com/ashleykleynhans/runpod-worker-a1111...

GitHub

GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...

RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111

Jump to solution

40 Replies

Solution

digigoblin•6mo ago

For A1111, this one is better, its more up-to-date: https://github.com/ashleykleynhans/runpod-worker-a1111

GitHub

GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...

RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111

digigoblin•6mo ago

Otherwise for SDXL you can use this and switch out the model: https://github.com/runpod-workers/worker-sdxl

GitHub

GitHub - runpod-workers/worker-sdxl: RunPod worker for Stable Diffu...

RunPod worker for Stable Diffusion XL. Contribute to runpod-workers/worker-sdxl development by creating an account on GitHub.

nerdylive•6mo ago

Yeah both works, but I think if you wanna use loras, embedding and other things on a1111, you can't do that all in sdxl worker

digigoblin•6mo ago

Yeah it just does plain inference, not even ControlNet etc

BBAznOP•6mo ago

so if i want to do the loras and stuff through API i would have to go with the a1111 right, so i might as well just follow the tutorial?

nerdylive•6mo ago

yeah, not sure if that tutorial has updated templates better to use ashleyk's template

BBAznOP•6mo ago

ok ill check out the ashleyk github link im using ashleyk's image container and used my network volume that i installed the a1111 on but its like stuck on initializing?

BBAznOP•6mo ago

got to this final step in the jupyter notebook terminal when installing the a1111 on network volume

BBAznOP•6mo ago

so the network volume should be good?

BBAznOP•6mo ago

@digigoblin nevermind--- i got it up, just had to switch gpu to available ones does cold start time billable? seems like the cold start runs up to like 2 minutes 🥶 @digigoblin @! what does this do?

digigoblin•6mo ago

Yes it is

BBAznOP•6mo ago

is there access to webUI after deploying a1111 to serverless? or just API

digigoblin•6mo ago

Just API, Serverless is an API , its not designed for Web UI. If you want a frontend, you have to build it yourself

BBAznOP•6mo ago

okie, but i can just spin up a pod and open the webUI right i just need to check if adetailer models came installed or not

digigoblin•6mo ago

Yes

digigoblin•6mo ago

How did you install it in serverless? if you use this, it includes ADetailer: https://github.com/ashleykleynhans/runpod-worker-a1111

GitHub

GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...

RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111

BBAznOP•6mo ago

yes its included but i dont see any adetailer models in the folder leme check volume again yeah i dont see anything in the workspace/stable-diffusion-webui/models/adetailer directory, which i think is where models should be? im checking webUI

nerdylive•6mo ago

check the {YOURWEBUIPATH}/extensions/adetailer @BBAzn

BBAznOP•6mo ago

it looks like this

nerdylive•6mo ago

check adetailer folder

BBAznOP•6mo ago

nerdylive•6mo ago

is it not there in your webui

BBAznOP•6mo ago

im trying to figure out how to load my UI

nerdylive•6mo ago

hmm im not sure but it says in the ext its should be in models/adetailer

nerdylive•6mo ago

download them from https://huggingface.co/Bingsu/adetailer/tree/main

Bingsu/adetailer at main

BBAznOP•6mo ago

okie should it be in the workspace/stable-diffusion-webui/extensions/adetailer/adetailer or just workspace/stable-diffusion-webui/extensions/adetailer

nerdylive•6mo ago

models path stable-diffusion-webui/models

BBAznOP•6mo ago

workspace/stable-diffusion-webui/models/adetailer

nerdylive•6mo ago

yep should be there /workspace/stable-diffusion-webui/models/adetailer

BBAznOP•6mo ago

@nerdylive if there are two API requests going to my A1111 serverless endpoint that uses two different SD models, would it be two instances of A1111? or would the A1111 instance have to swap out models? im asking just curious whether there would be a generation speed i mean i could test it but figured id ask ;D

digigoblin•6mo ago

It works fine if you use the override to set the model in the serverless request eg:

                "override_settings":{
                    "sd_model_checkpoint":"turboDiffusionXL_v112",
                    "CLIP_stop_at_last_layers": 1
                },

                "override_settings":{
                    "sd_model_checkpoint":"turboDiffusionXL_v112",
                    "CLIP_stop_at_last_layers": 1
                },

BBAznOP•6mo ago

yes swapping seems to be no issue, but i noticed that the generation queue is lined up one by one, so when i send generation requests from VSC terminal, i would open up multiple terminals and send requests one at a time, and the serverless endpoint would generate one image, then start the other and so on i thought it was supposed to scale with the workers?

digigoblin•6mo ago

Depends on your endpoint configuration

nerdylive•6mo ago

You can set scale type in your endpoint to queue delay to be 1

BBAznOP•6mo ago

oh ok, should i change the request count?

nerdylive•6mo ago

No, just select it on queue delay

BBAznOP•6mo ago

oh ok its working

BBAznOP•5mo ago

@digigoblin hey guys so i deleted my previous volume and did a fresh reinstall but now the adetailer wont trigger through API ;-; i have the adetailer models in /workspace/stable-diffusion-webui/models/adetailer and i was able to get it working in the webui on a pod, but the API just doesnt seem to pick it up? my code is the same too ;-;

BBAznOP•5mo ago

i did a GET extensions call and i saw the adetailer extension is there so the adetailer is installed i mean i was able to get the adetailer working in the webui so its definitely installed lol nvm i figured it out, it had something to do with an adetailer config "ad_tab_enable": true, after removing that it was good

Gaming

Programming

how to deploy custom image gen model on serverless?