How to get `/stream` serverless endpoint to "stream"?
Example from official documentation: https://docs.runpod.io/sdks/javascript/endpoints#stream
The example suggests that the
/stream
endpoint should return the entire input, one letter at a time. However, when I try it it only sends the first letter back, followed by the http connection closing.
Is /stream
intended to be streamed from start to finish, or polled repeatedly?Endpoints | RunPod Documentation
Learn how to interact with RunPod's endpoints using the JavaScript SDK, including synchronous and asynchronous execution methods, status checks, and job cancellation. Discover how to set timeouts, execute policies, and purge queues.
6 Replies
* Looking at
js-sdk
source: https://github.com/runpod/js-sdk/blob/main/src/index.ts#L203 stream()
doesn't pass {responseType: 'stream'}
to xior.get
so it can't be streaming
* Running the runpod UI with network tab open: (https://www.runpod.io/console/serverless/user/endpoint/$endpointId
)
* It polls /stream
every few seconds, not doing streaming.yes it doesn't actually "stream"
its mimicking stream like behaviour if you have generator handler (view docs), and it is like pull based queue system
so once you hit that, the data got removed and you can poll that many times during the job processing to retrieve new messages
Is it true only for the js client or the serverless runtime entirely?
it doesn't actually "stream"If so, it really must be documented because to all of us 'stream' means 'chunked transfer encoding', not the general category of usage encompassing polling. I was suspicious why they don't have
-N
/--no-buffer
flag in their curl example for the stream endpoint but now have figured out why.. https://docs.runpod.io/serverless/endpoints/job-operations#stream-resultsJob operations | RunPod Documentation
Learn how to use the Runpod Endpoint to manage job operations, including running, checking status, purging queues, and streaming results, with cURL and SDK examples.
Yes it must use polling to update
thank you! I got it working with stream via polling :scarlettHeart:
Your welcome!