How much does it cost to use multi-GPU ?
I'd like to increase the number of GPUs per worker to get better performance with parallelization.
When i read this post: https://blog.runpod.io/runpod-serverless-pricing-update/, I have the impression that the cost is only linked to the “type of GPU” (16GB, 24GB, 48GB, ...) and that increasing the number of GPUs per worker doesn't increase the price per second. But that doesn't seem logical to me. Do I pay as much if I use a worker for 30s with 2 GPUs VS a worker for 30s with only 1 GPU? Or does the worker with 2 GPUs cost twice as much as the worker with a single GPU?
Also, when I read the doc: https://docs.runpod.io/serverless/references/endpoint-configurations#gpus--worker, it says that multi-GPU is only available on 48GB instances, but in the interface I get the impression that it's available on other types (the ones I'm interested in are 24GB). Is it just that the documentation isn't up to date, or is it a display problem?

4 Replies
Maybe it isn't up to date ( the docs) and about the price yes it's *2 or times whatever your gpu amount is in each worker
Ok, it seems more logical, thank you 🙂 you should put a mention in the price page and/or in the enpoint configuration form, to make things clearer
Also, your bot told me the opposite, since it's not written anywhere in the doc : https://discord.com/channels/912829806415085598/1334482909352038431
It can be misinterpreted and you users could be in for a nasty surprise when it comes time to pay the bill 😅
Can you help post this #🧐|feedback