How to WebSocket to Serverless Pods
@Merrell / @flash-singh Wondering if I can get code-pointers on how to use the API to expose ports on serverless programatically, so that I can do stuff like web sockets?
22 Replies
Wow if this is possible then it's great
But I think they'll allow some ports for you to use rather than allowing you to control the open ports on serverless ..
they used to be able to have in the UI the ability to open up ports just like GPU Pod, but they said they removed it to not cause customer confusion / etc. But supposedly, since it's just a GPU Pod under the hood of serverless, anything should be possible on serverless that is on GPU Pod.
Well either way, even if its just a dedicated port on serverless for web sockets, that still nice, to be able to stream data / files
Ooh that's weird customer confusion? So, Now they simplify it by automating the assignment of ports ye
Yeah I'm interested in this too 👌
I think there is no ports exposed at all right now, just a simple handler function; except for certain customers i guess of theirs are using it, but not widely known knowledge
Oh yea for gpu pods* I mean tcp ports are randomly assigned right
you would need to use our graphql api to add ports to a serverless template, from there the workers will have ports, by default gui will not let you add ports
tcp ports are randomly assigned but we do expose them in env variables, so technically you can expose that port back to your infra using progress updates within a job
heres the flow that someone else used
- run a job
- worker will launch and env is populated, in progress of the job share back port / ip details
- catch port / ip details in your app using /status or /stream, do whatever you want with the port / ip
- when your done using the port / ip, send message through that port / ip service to mark the job done on worker side
- once job is done, the worker will turn off automatically if there are no more jobs
Thank you! ❤️ Appreciate it! ❤️
Oh is it already possible by now and documented in the reference?
Which env variable BTW?
on any pod, you can get env variables and ports are exposed as part of env, ill have to check where we document it
is it possible to stream with this? websockets are definitely more efficient
Yeah oh really? We sockets more efficient?
considering my method of streaming yes
we just spam the request to get all chunks
websockets should work the same since your using ports, also you can use server-sent events, SSE
https://medium.com/deliveryherotechhub/what-is-server-sent-events-sse-and-how-to-implement-it-904938bffd73
Hey @flash-singh have you got the env variable yet?
Please send the env name for exposed ports if there is
heres an example, if you expose 22 port in your template, env will show this
RUNPOD_TCP_PORT_22=22097
that is saying port 22 is mapped to 22097 externally to usersOhh it works on pods and serverless? ok thanks
Oh and will i be able to get the RUNPOD_PUBLIC_IP everytime?
for serverless
yes, serverless runs on secure cloud and all secure cloud pods have public ip, just keep in mind ip and port can be different for every job you run in serverless
i did this on dockerfile:
EXPOSE 22/tcp
I did the graphql port assignment too on tcpdid it work?
Nah as I said it returned only none object on python
Why is that?
Do you wanna check out my template to see if the ports is there?
yeah sure
@flash-singh 3pgp4kglgq heres the template id
sorry for the wait lol
can you make sure if its already right
and any updates on the public ip env or the port env?
or any alternatives for websockets in serverless? ( it needs ports right? which defaults to http or https )
i should've used the stream instead for now yeah