Is there a way to scale pods?
I would like to scale up number of pods in order to meet demand. Is there a way to do that?
8 Replies
use serverless
Yep, pods can't scale up automatically, serverless is designed for scalability.
well you can create a custom code that handles scaling like kubernetes using the graphql runpod api
you can gather their load and check if you need to deploy another one
I don't recommend doing that, using serverless is better.
yeah easier too
Easier and probably cheaper too.
Problem is that serverless doesn't have H100 gpus
Yeah that's currently high in demand