james3000
RRunPod
•Created by james3000 on 8/19/2024 in #⚡｜serverless
With LLM on runpod is there a cost like other providers like tokens and if its serverless
yes this might be the issue not sure yet! I tried running meta-llama-Meta-Llama-3.1-8B but its just not working? has anyone else got this one working?
7 replies
RRunPod
•Created by james3000 on 8/19/2024 in #⚡｜serverless
With LLM on runpod is there a cost like other providers like tokens and if its serverless
Thanks buddy! We were running a serveless for transcription using Whisper but it takes a while to start up .. ill deploy a LLM in serverless and see how it goes! But thanks again for your advice. I like that the SL doesn't cost us anything for idle time as these GPUs currently are super expensive!!!
7 replies
RRunPod
•Created by james3000 on 8/7/2024 in #⚡｜serverless
Hello
Thanks guys I will contact them and then also check with some lawyers.
4 replies
RRunPod
•Created by james3000 on 5/30/2024 in #⚡｜serverless
is anyone experiencing a massive delay time when sending jobs to GPUs on serverless?
Thanks everyone its seems to work better with more max workers!
7 replies