Is Quick Deploy (Serverless) possible for this RoBERTa model?
I want to run this model:
https://huggingface.co/MoritzLaurer/roberta-large-zeroshot-v2.0-c
as Serverless. And Quick Deploy (under https://www.runpod.io/console/serverless) shows multiple option, what should I choose? Should it be "Serverless vLLM" or something else? NOTE: this model is NOT a LLM. It is based on the RoBERTa architecture. And RoBERTa is not directly listed as a supported model architecture in vLLM
as Serverless. And Quick Deploy (under https://www.runpod.io/console/serverless) shows multiple option, what should I choose? Should it be "Serverless vLLM" or something else? NOTE: this model is NOT a LLM. It is based on the RoBERTa architecture. And RoBERTa is not directly listed as a supported model architecture in vLLM
1 Reply
I suggest that you take the example listed on the Huggingface page for this model and write a custom handler for running it. Here is a tutorial to get you started: https://trapdoor.cloud/getting-started-with-runpod-serverless/
Ashley's Blog
Getting Started with RunPod Serverless
This post helps you to get started with RunPod Serverless for hosting and scaling your AI applications in the cloud.