BadNoise
RRunPod
•Created by BadNoise on 9/19/2024 in #⚡｜serverless
Error with the pre-built serverless docker image
9 replies
RRunPod
•Created by BadNoise on 7/5/2024 in #⚡｜serverless
Pipeline is not using gpu on serverless
70 replies
RRunPod
•Created by BadNoise on 7/4/2024 in #⚡｜serverless
Deploy BART on serverless
Hi!
Does anyone know how to deploy bart-large-mnli on serverless? Been trying with the hugging face template (ghcr.io/huggingface/text-generation-inference) but I always get "Error: ShardCannotStart"
I already tried setting NUM_SHARD = 1 in env but still failing.
Repo for reference: https://huggingface.co/facebook/bart-large-mnli
Let me know if you need further details!
Thank you 🙂
10 replies
RRunPod
•Created by BadNoise on 6/6/2024 in #⚡｜serverless
Parallel processing images with different prompt
Hi!
I am running a1111 on serverless.
Is it possible to generate images in parallel with different prompt? As far as I know on sd web ui it's possible only to set the batch size, but it will use the same prompt and moreover it needs to have an external queue manager.
Is there any serverless pod or some particular sd web ui configuration that allow me to do it?
Thank you
10 replies
RRunPod
•Created by BadNoise on 5/3/2024 in #⚡｜serverless
Serverless SD concurrent requests on multiple checkpoints
Hi,
Do you know if there is a way to handle concurrent SD predictions (even 10 is fine) on different checkpoint with different prompt?
For example I want to run 5 concurrent requests on checkpoint_1 and 5 on checkpoint_2, passing the checkpoint name in the request body.
Thank you
8 replies