gnarley_farley. Comments - Answer Overflow

gnarley_farley.

Explore posts from servers

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

If you get the docker 5$ per month sub. (pro i think). You can automatically build the docker images when you push to github. Makes a huge difference.

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

Yeah I am just using polling currently with RUN. Works flawlessly. Not really phased about a few extra requests popping of to check if it's ready. Hardly consequential in my current pipelines.

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

What's the latest version/implementaiton of the webhooks for runpod look like? Managed to find this link https://www.answeroverflow.com/m/1206251618022981694

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

is there a serverless endpoint for cogvideo5b yet? I see camenduru has one with a gui. looking for just api service send request, receive back video . If anyone finds it please hola.

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

Sorry I don't want to hijack your thread but The issue is that you need to put everything in the docker image otherwise it takes to long to initialize in severless. I need it to be scalable from 0 serverless, because our sass infra demands that. It's quite complicated. Have tried. It might also be a bit premature I don't see any official/popular flux based workflows for inpainting up yet.

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

Thanks will check that out don't know much about Comfyui. If anyone can make an inpainting version of this worker that uses flux it would be AMAZING! I want to be able to just pass in a masked photo, with a prompt and get something back. My starlink+ dockerhub is just super slow for some reason, can't effectively push these big images.

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

OKay cool thanks. I will test that out. Where are you adding the flag?

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

My requests come back in the same time. I know those limits are fudged. I had to figure it out through trial and error

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

Set the polling to 1 second interval. It is very performant. I don't see a difference at all

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

Because it's one request at a time and comfyui doesn't use batching, you don't really gain much from the extra ram.

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

Im using 3x 4090's and ripping hard. I saw no/very little performance increase from using bigger gpus

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

And it does not effect the speed at all. I feel it is faster now. I have not done tests to verify that. But it's definitely not slower

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

It's much more performant anyways to use the polling system. And if you build an app on say severless in future it won't effect your serverless function limits. Using runsync and waiting for a bunch of requests to return is not very efficient. Modern frameworks like nextjs have a 10s limit on function timeouts and the the serverless function time is one of the biggest cost factors

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

I am using the exact same severless comfyui api to power my application. I do bulk processing of images and I ran into the exact same issue. This is how I fixed it. I can run hundreds of images in one shot now effortlessly. Even with just 3 active gpus.

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

Use a polling mechanism and check every few seconds to see if your requests are ready.

80 replies

RRunPod

•Created by SyedAliii on 9/20/2024 in #⚡｜serverless

Issue with Multiple instances of ComfyUI running simultaneously on Serverless

Here is everything your need to get your problem solved. https://docs.runpod.io/serverless/endpoints/job-operations

80 replies

RRunPod