serverless socket.io support
Hello, I want to use socket.io-based serverless endpoint using RunPod serverless.
I'm curious if this is possible. When I create a serverless API and connect to that API address via socket.io, which internal instance exactly does it connect to? Because I want to connect to an instance where the queue_length is less than 30 in the serverless instance. However, looking at serverless, this seems impossible. Is this possible?
4 Replies
using tcp connections? or http?
i have never used socket,io setup in runpod serverless, but I believe it is possible by exposing ports in endpoints
i had a post here somewhere around making web sockets work with your workers, much of the work is on you but it is possible, we dont natively support it, you would have to run your own web socket server with exposed ports
1. Use job to control lifecycle of the ports / service running on the port
2. In handler, when job runs, run the webserver on the port exposed in the template.
3. Take env variables for public ip and port map env and use progress hook
RUNPOD_PUBLIC_IP RUNPOD_TCP_PORT_8888 if port exposed is 8888
4. On client-side, read the status, it will have all the progress metadata.
5. Do whatever you need while job is in progress to communicate with WS and run any workload you want.
6. Somehow signal the WS or job to mark itself done, the handler should close to websocket server at this point.
So the ws should be able to communicate the handler to return in the handler
Maybe by Handler busy-waiting for the ws right