Ammar Ahmed
Network Volume as Storage for images
Hi, I am building an image generation application which will store images to a database for which I was thinking RunPod's network volumes attached with a CPU pod sending and fetching images from the volume. Will network volumes be worth it?
3 replies
RRunPod
•Created by Ammar Ahmed on 10/4/2024 in #⚡｜serverless
How can I make a single worker handle multiple requests concurrently before starting the next worker
Hi everyone,
I’ve deployed an image generation model using a 24GB GPU with 2 workers (1 active) on RunPod. Each image generation uses around 6-7GB of memory. My goal is to have a single worker handle multiple requests concurrently until it can’t handle the load anymore, and only then should the second worker start.
Right now, when I send two requests, the second worker starts immediately to process the second request, even though my first worker should have enough resources left to handle both requests at once.
How can I make sure that a single worker handles multiple requests concurrently before the next worker is activated? I am new to cloud deployments so If I get any terminology wrong, please guide me.
Thanks in advance!
38 replies