Lta
RRunPod
•Created by Lta on 9/10/2024 in #⚡｜serverless
Streaming support in local mode
Is there a way to enable streaming on a local development env ?
If I'm not mistaken it doesn't seem possible, which makes developping on runpod very tedious since the behavior on the local server and on the serverless platform different.
See the code here: https://github.com/runpod/runpod-python/blob/a433a296dbb903f448f7c3e9a275e960812fb60b/runpod/serverless/modules/rp_fastapi.py#L330
Streaming from FastApi is possible, cf. https://fastapi.tiangolo.com/advanced/custom-response/#streamingresponse.
It just looks like the person implementing this was in a hurry and/or a tiny winy bit lazy.
Sidenote: There hasn't been any serious commit/issue/PR activity on the runpod-python since February. Are you planning to drop support ?
4 replies