is there any method to deploy bert architecture models serverlessly?
is there any method to deploy bert architecture models serverlessly?
9 Replies
Hi
There is using huggingface's transformers
you can cache the model first in some path, then load it using relative / absolute path.
put the model in the image / use a network volume
Hi, thanks for help
Is there any template for this?
Or I should write the handler by myself?
yes
there is
use VLLM template
Solution
If you need help with configuring that template theres a setup menu after you click on it
Yeah I have try this
oh wait bert isnt compatible with that isnt it?
But it doesn't supports bart models
Exactly
Hmm then you'll have to write an custom handler first for beert
using transformers pipeline works too