Sven
RRunPod
•Created by zeeb0t on 11/12/2024 in #⚡｜serverless
Mounting network storage at runtime - serverless
I have a simmilar Issue, where i want to store my llm models on the network drives, but then i am locked on one region, would be nice to be able to add a range of network drives (one for each region) in the serverless GUI, so multiple locations can be selected for a single endpoint.
11 replies
RRunPod
•Created by Sven on 10/2/2024 in #⚡｜serverless
Serverless vllm - lora
The Pull Request was merged. Does someone know when the new Versions appears for Runpod? (also the vllm worker with the 1.7.4 SDK?)
12 replies
RRunPod
•Created by hhsynalv on 9/27/2024 in #⚡｜serverless
serverless error
any news on this, i got the same problem?
3 replies
RRunPod
•Created by Sven on 10/2/2024 in #⚡｜serverless
Serverless vllm - lora
For the runpod vllm serverless worker, these are the ENVs, which are available: https://docs.runpod.io/serverless/workers/vllm/environment-variables
12 replies
RRunPod
•Created by Sven on 10/2/2024 in #⚡｜serverless
Serverless vllm - lora
there are only the options for enabling it and so on, the option for adding the url is missing
12 replies