iNF3Rnus
RRunPod
•Created by iNF3Rnus on 3/6/2024 in #⚡｜serverless
Are stream endpoints not working?
Potentially nevermind, I think I understand now... You start an async job with run, then get the id from that, and retrieve the results with stream/{job_id}
Based off of testing this in Postman, I don't think this supports actual streaming, or am I mistaken?
I need these endpoints for LLMs for use with a FE application and I want to literally stream tokens
text/event-stream style, is that supported?3 replies