R
RunPod•5mo ago
Data_Warrior

serverless deployment

i want to deploy my llm on serverless endpoint, how can i do that?
Solution:
https://github.com/runpod-workers/worker-vllm
GitHub
GitHub - runpod-workers/worker-vllm: The RunPod worker template for...
The RunPod worker template for serving our large language model endpoints. Powered by vLLM. - runpod-workers/worker-vllm
Jump to solution
3 Replies
Solution
ashleyk
ashleyk•5mo ago
https://github.com/runpod-workers/worker-vllm
GitHub
GitHub - runpod-workers/worker-vllm: The RunPod worker template for...
The RunPod worker template for serving our large language model endpoints. Powered by vLLM. - runpod-workers/worker-vllm
ashleyk
ashleyk•5mo ago
Please ask in #🎤｜general , this is for support with problems with serverless not for asking basic questions that the community can help with.
haris
haris•5mo ago
This channel is fine, they asked a question about serverless in the serverless channel, doesn't matter if they are new to the platform and asking basic questions.
Want results from more Discord servers?
Add your server
More Posts
Can i still access the data of my GPU pod once my account run out of fundsI have a telegram bot running in a GPU pod. It has a postgres database container, it stores all the How to know when request is failedHello, everyone I am using webhook to be notified for job completion. I wondering if this webhook isIN-QUEUE IndefinitelyI am attempting to deploy a model from HF Spaces in runpod serverless - using the ByteDance/SDXL-LigCan I Sync Contabo storageContabo has a S3 compatible Object Storage same as Amazon S3. https://docs.contabo.com/docs/productsSave docker sessionHey so if i start a docker container as a GPU pod and then install something can i save the pod statFrequent GPU problem with H100Hello, I've seen that 9 times out of 10, I would get an H100 (PCIe) machine where in Cuda won't worCosting for Serverless pods without GPUI can't see any documentations about costing of serverless pods without any GPUMigrating from Banana.devHello. I'm trying to migrate a project over. My previous flow was, whenever I wanted to deploy a chaPost seems to have lost internet.Pod ID is pj1vaht5iuipfbhow to deploy suno bark tts model using runpod serverless endpointshi i want to deploy the opensource tts model called suno bark on runpod and i want to create an api