Hosted Serverless

I looked at your website and I am not sure if you host models in a multi-tenant style. I want to be able to call some common models like Mixtral at low cost without paying for a dedicated server. Do you offer this?
7 Replies
nerdylive
nerdylive8mo ago
Yes its serverless Mixtral seems a huge model ah, i think its better to use pods for that whats a multi tenant style by the way?
JJonahJ
JJonahJ8mo ago
Mixtral can run on serverless. Takes a minute and a half to spin up though…
nerdylive
nerdylive8mo ago
oh what gpu did it takes to run that model
JJonahJ
JJonahJ8mo ago
Uhhh… I have it on a 48GB gpu
nerdylive
nerdylive8mo ago
which mixtral model is it the small one?
nerdylive
nerdylive8mo ago
Ohh
Want results from more Discord servers?
Add your server