Builderman
RRunPod
•Created by Builderman on 10/7/2024 in #⚡｜serverless
Serverless Streaming Documentation
I'm using the runpod github template for my model and it's working- but how would I set it up and make my model stream to runpod?
2 replies
RRunPod
•Created by Builderman on 2/19/2024 in #⚡｜serverless
Mixtral Possible?
Wondering if it's possible to run AWQ mixtral on serverless with good speed
21 replies
RRunPod
•Created by Builderman on 1/13/2024 in #⚡｜serverless
streaming
i'm used to using openai to stream text all in one request that is done async. why does runpod just spam requests for the /stream endpoint instead of making it one request and feeding data as it's generated?
5 replies