Streaming support in local mode
Is there a way to enable streaming on a local development env ?
If I'm not mistaken it doesn't seem possible, which makes developping on runpod very tedious since the behavior on the local server and on the serverless platform different.
See the code here: https://github.com/runpod/runpod-python/blob/a433a296dbb903f448f7c3e9a275e960812fb60b/runpod/serverless/modules/rp_fastapi.py#L330
Streaming from FastApi is possible, cf. https://fastapi.tiangolo.com/advanced/custom-response/#streamingresponse.
It just looks like the person implementing this was in a hurry and/or a tiny winy bit lazy.
Sidenote: There hasn't been any serious commit/issue/PR activity on the runpod-python since February. Are you planning to drop support ?
Custom Response - HTML, Stream, File, others - FastAPI
FastAPI framework, high performance, easy to learn, fast to code, ready for production
GitHub
runpod-python/runpod/serverless/modules/rp_fastapi.py at a433a296db...
🐍 | Python library for RunPod API and serverless worker SDK. - runpod/runpod-python
3 Replies