Is the vLLM worker updated for LLaMA3.1 yet?

If not, is anyone aware of a good serverless container that does support it?

2 Replies

nerdylive•7mo ago

Not yet lets wait vllm-worker from runpod to be updated i guess it'll be in a few more days or less

NERDDISCO•6mo ago

It is updated, you can use it!

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

15KMembers