testymctestface Comments - Answer Overflow

testymctestface

•Created by testymctestface on 12/27/2024 in #⚡｜serverless

Running worker automatically once docker image has been pulled

@3WaD you using it for text generation? If so your approach sounds ok. I'm using it for image generation and Comfyui which I don't think would work as a warm worker would need to generate an image, which takes about 15 secs.

58 replies

RRunPod

•Created by testymctestface on 12/27/2024 in #⚡｜serverless

Running worker automatically once docker image has been pulled

ok - I'm just going to use dedicated active workers for now and try to optimise my docker image. Thanks and appreciate everyone chipping in with ideas.

58 replies

RRunPod

•Created by testymctestface on 12/27/2024 in #⚡｜serverless

Running worker automatically once docker image has been pulled

yeah I just thought there might be a way of doing it automatically. Either that, or I'll have to make them permanent... but then cost goes up

58 replies

RRunPod

•Created by testymctestface on 12/27/2024 in #⚡｜serverless

Running worker automatically once docker image has been pulled

Thanks. But is there a way to auto run a job for that worker once the image has been pulled? I use it for comfyui image generation. The first run takes about 40 secs as the models need to be loaded into the GPU memory, subsequent runs are 15 seconds.

58 replies

RRunPod

•Created by testymctestface on 12/27/2024 in #⚡｜serverless

Running worker automatically once docker image has been pulled

Ideally I was looking for an automated solution. One of my workers is pretty much always being used but when additional workers are called they're sometimes slow to output a result due to not being used since the docker image was pulled.

58 replies

RRunPod

•Created by testymctestface on 12/27/2024 in #⚡｜serverless

Running worker automatically once docker image has been pulled

Exactly, I want to queue a job that loads all the models I need. I'm just unclear the best way to initiate the the job once the worker is ready. The goal is to reduce the runtime of the first job requested through my web app. The first job takes about twice as long as subsequent runs.

58 replies

RRunPod

•Created by testymctestface on 12/27/2024 in #⚡｜serverless

Running worker automatically once docker image has been pulled

got it - I just wondered if there was a simple way to queue a job automatically once the container has been setup.

58 replies

RRunPod

•Created by testymctestface on 12/27/2024 in #⚡｜serverless

Running worker automatically once docker image has been pulled

Thanks @flash-singh - I'm trying to run a job as soon as the image is loaded. When a job runs on a new worker for the first time it has to load all the models etc which adds around 30 seconds. If I do this as soon as the image is loaded then the user won't need to experience this wait time as the models will be cached

58 replies

RRunPod

•Created by testymctestface on 7/23/2024 in #⚡｜serverless

Serverless worker failing - how do I stop it

thanks! going to try and build docker image again and make it a little more robust

16 replies

RRunPod

•Created by testymctestface on 7/23/2024 in #⚡｜serverless

Serverless worker failing - how do I stop it

ok thanks for the tip on terminating workers.

16 replies

RRunPod