Serverless Streaming Documentation
I'm using the runpod github template for my model and it's working- but how would I set it up and make my model stream to runpod?
1 Reply
What do you mean stream to runpod, what model is it? Basically you'll have to create a docker image with your model and do inference in the handler.py... Then return the output