Hosted Serverless
I looked at your website and I am not sure if you host models in a multi-tenant style. I want to be able to call some common models like Mixtral at low cost without paying for a dedicated server. Do you offer this?
7 Replies
Yes its serverless
Mixtral seems a huge model ah, i think its better to use pods for that
whats a multi tenant style by the way?
Mixtral can run on serverless. Takes a minute and a half to spin up though…
oh what gpu did it takes to run that model
Uhhh… I have it on a 48GB gpu
which mixtral model is it
the small one?
Ohh