Jack
RRunPod
•Created by Jack on 1/25/2024 in #⚡｜serverless
How to get around the 10/20 MB payload limit?
@ashleyk That's the best solution. Thank you
7 replies
RRunPod
•Created by Jack on 1/14/2024 in #⚡｜serverless
Do you get charged whilst your request is waiting on throttled workers?
Thank you guys for clearing that up
7 replies
RRunPod
•Created by Jack on 1/14/2024 in #⚡｜serverless
Do you get charged whilst your request is waiting on throttled workers?
Is there a way to see what portion of the Delay Time was charged and what part was not?
7 replies
RRunPod
•Created by Jack on 1/9/2024 in #⚡｜serverless
ailed to load library libonnxruntime_providers_cuda.so
Yeah I gave up trying to aolve this problem. I'm just sticking to using CPU, which is a massive waste of GPU time but whatever. At least it works
10 replies
How to enable Jupyter Notebook and SSH support in a custom Docker container?
Thanks. If I only wanted to enable jupyterlabs, can I just copy over the
start_jupyter() and execute it in start.sh? Do I need to copy over the other functions like start_nginx(), execute_script(), export_env_vars?18 replies
RRunPod
•Created by Shaggbagg on 1/13/2024 in #⚡｜serverless
Unreasonably high start times on serverless workers
Hey @Shaggbagg . I am working on the exact same problem as you. I started off with installing A1111 on a Network Volumes and noticed the cold start time are extremely high, between 60-100 secs. Then @justin recommended to install everything directly on a Docker container, and skipping the Network Volume altogether. I'm currently working on doing that right now, but running into some issues. I sent a friend request, maybe we can help each other since we're working on the same thing.
40 replies
RRunPod
•Created by Jack on 1/12/2024 in #⚡｜serverless
Error whilst using Official A1111 Runpod Worker - CUDA error: an illegal instruction was encountered
@rafael21@ Sure. It depends on what kind of images you're looking to generate. But basic images, just a single one, takes only about 2 seconds (not counting cold start times) with serverless GPU endpoints
4 replies
RRunPod
•Created by foxhound on 1/9/2024 in #⚡｜serverless
[RUNPOD] Minimize Worker Load Time (Serverless)
I saw that but they use diffusers. I need to load some extensions for A1111 for my app, so I need to use A1111 as a template for my app
40 replies
RRunPod
•Created by foxhound on 1/9/2024 in #⚡｜serverless
[RUNPOD] Minimize Worker Load Time (Serverless)
Yeah using Network Volume just isn't the way to go for A1111. The cold start time is 60-70 secs for a 3 second photo generation which is too long. Loading everything onto the Docker image is 100% the way to go
40 replies
RRunPod
•Created by foxhound on 1/9/2024 in #⚡｜serverless
[RUNPOD] Minimize Worker Load Time (Serverless)
Yeah 8GB RAM on Mac using Docker is 💀 💀
40 replies
RRunPod
•Created by foxhound on 1/9/2024 in #⚡｜serverless
[RUNPOD] Minimize Worker Load Time (Serverless)
Thanks @justin for that resource. Especially thanks for mentioning depot. It's honestly a life saver for me as I'm using a Macbook for development as well, and dealing with Docker locally is a nightmare. I found an official runpod worker for A1111 by Runpod, but it's not actively maintained and getting issues. Either way it's a great starting point for using A1111 on a worker without needing Network Volumes
https://github.com/runpod-workers/worker-a1111
40 replies
RRunPod
•Created by foxhound on 1/9/2024 in #⚡｜serverless
[RUNPOD] Minimize Worker Load Time (Serverless)
Super helpful comment, thanks justin. You're right about not using the pre-made stable-diffusion-webui-docker, I was running into some trouble running it. I'm going to try the approach you mentioned and start with a basic PyTorch template and go from there, see if I can build this dockerfile one line at a time
40 replies
RRunPod
•Created by foxhound on 1/9/2024 in #⚡｜serverless
[RUNPOD] Minimize Worker Load Time (Serverless)
Wait so is it possible loading A1111 onto a Docker image, and skip using a Network Volume like @justin mentioned? I'm not too familiar with Docker but it seems like there are some github repos offering a Docker container for A1111 like this one - https://github.com/AbdBarho/stable-diffusion-webui-docker
40 replies
RRunPod
•Created by foxhound on 1/9/2024 in #⚡｜serverless
[RUNPOD] Minimize Worker Load Time (Serverless)
I'm facing a similar issue running A1111 on Serverless Endpoints, it takes about ~60-70 seconds to start up to perform a 3 second generation task.
Is it possible to bake a customized A1111 instance onto a Docker image and having the serverless endpoint loading that docker image directly, skipping the process of having the endpoint load from a Network Volume containing the A1111 instance?
40 replies
RRunPod
•Created by Jack on 1/9/2024 in #⚡｜serverless
ailed to load library libonnxruntime_providers_cuda.so
I am able to use the GPU for faceswaplab on a colab notebook and also locally, so I had assumed it was maybe an issue with Runpod. But I could be wrong.
10 replies
RRunPod
•Created by Jack on 1/9/2024 in #⚡｜serverless
ailed to load library libonnxruntime_providers_cuda.so
@ashleyk Thanks for the tip. My network volume had both
onnxruntime and onnxruntime-gpu installed. I tried to uninstall onnxruntime, but each time I run A1111, it will reinstall onnxruntime automatically since it is a requirement by faceswaplab.
P.S. I'm actually using your runpod-worker-a1111 on my Network Volume, and installed Faceswaplab on top of it.10 replies
RRunPod
•Created by Jack on 12/21/2023 in #⚡｜serverless
Is it possible to build an API for an automatic1111 extension to be used through Runpod serverless?
ReActor is a great extension and it's what I used in the beginning. However after I discovered faceswaplab, I stopped using reActor altogether because faceswaplab just provides much better results.
The issue with reActor is it only takes a single photo. Faceswaplab allows you to build a face model using as many pictures as you want. The face model blends all the images you uploaded for a more accurate result.
ReActor is indeed more actively maintained and more popular by stars on Github, but from my experience, faceswaplab is simply the superior face swap extension as of right now. Hopefully ReActor adds the ability to use multiple photos instead of just 1 photo.
Faceswaplab actually does have API with typing support in its features, I just realized. https://glucauze.github.io/sd-webui-faceswaplab/features/
10 replies