Data_Warrior
RRunPod
•Created by Data_Warrior on 3/12/2024 in #⚡｜serverless
Serverless cost
i want to deploy my model on serverless, how it is costing?
3 replies
error in pod
"2024-03-12T08:39:43.053682465Z /usr/bin/python3: Error while finding module specification for 'vllm.entrypoints.openai.api_server' (ModuleNotFoundError: No module named 'vllm')"
i always run this in A6000, but it is getting error, why this is happening??
23 replies
RRunPod
•Created by Data_Warrior on 3/5/2024 in #⚡｜serverless
serverless deployment
i want to deploy my llm on serverless endpoint, how can i do that?
6 replies
Error while using vLLm in RTX A6000
2024-02-22T11:19:46.009303238Z /usr/bin/python3: Error while finding module specification for 'vllm.entrypoints.openai.api_server' (ModuleNotFoundError: No module named 'vllm')
Using RTX A600, i'm using this gpu from last 4-5days, not getting any error,
but today i'm facing this issue, could anyone help me out with this, why it is happening?
3 replies
RRunPod
•Created by Data_Warrior on 2/13/2024 in #⚡｜serverless
Issue in pod
i'm facing issue from last 2 days, sometime RTX 4090 generates 60 token/second and sometime it 30-20 token/second to generate same response.
don't know what is behind this ????
1 replies