VidimusWolf
RRunPod
•Created by VidimusWolf on 10/14/2024 in #⚡｜serverless
Streaming LLM output via a Google Cloud Function
Just in case anyone looks for this in the future, it is possible to do it using python's requests library.
2 replies
RRunPod
•Created by Ethan Blake on 10/10/2024 in #⚡｜serverless
Why too long delay time even if I have active worker ?
do we need to do anything to update it or is it automatic for our endpoints?
9 replies
RRunPod
•Created by VidimusWolf on 10/9/2024 in #⚡｜serverless
Keeping Flashboot active?
I see, thanks for the information! Of course the hope is to get enough traffic to make the active server worth it 🙂
9 replies
RRunPod
•Created by VidimusWolf on 10/9/2024 in #⚡｜serverless
Keeping Flashboot active?
I see, so it really isn't designed to be reliable?
9 replies
RRunPod
•Created by VidimusWolf on 10/9/2024 in #⚡｜serverless
Keeping Flashboot active?
Because the main selling point of this service for us was precisely flash boot, so I was hoping to have more information on its reliability
9 replies
RRunPod
•Created by VidimusWolf on 10/9/2024 in #⚡｜serverless
Keeping Flashboot active?
Or can it literally also be disabled 5 seconds after the last request?...
9 replies
RRunPod
•Created by VidimusWolf on 10/9/2024 in #⚡｜serverless
Keeping Flashboot active?
Is there a minimum time before which flash boot is always guaranteed?
9 replies
RRunPod
•Created by VidimusWolf on 10/9/2024 in #⚡｜serverless
Hugging face token not working
It seems it now works, perhaps it just needed time I don't know. Thanks anyway for the help!
7 replies
RRunPod
•Created by VidimusWolf on 10/9/2024 in #⚡｜serverless
Hugging face token not working
As I specified, I do actually already use these models locally. The tokens 100% work and I 100% have access
7 replies
RRunPod
•Created by VidimusWolf on 10/9/2024 in #⚡｜serverless
Hugging face token not working
Hello! Has anyone had issues getting their hugging face token to work on a serverless vLLM instance? I have used hugging face before and their tokens work for me locally, but I keep getting access denied log entries on the console logs when trying to send a request even though I give it the token key...
7 replies