RunPod•4mo ago

Exposing HTTP services in Endpoints through GraphQL

Hello! I was wondering how we could expose the opened HTTP service URLs in the Serverless Endpoints as they keep changing each time an Endpoint is set to zero workers. Thanks!

3 Replies

Encyrption•4mo ago

The RunPod proxy URL is based upon the POD ID so it will be unique. Cannot get around that. The way I handle that is I call the RUN (asynch) endpoint and then start checking the STATUS endpoint.
On the worker, I send out a progress update with the URL:

POD_ID = os.getenv('RUNPOD_POD_ID')
WS_PORT = 8888
WS_URL = f'wss://{POD_ID}-{WS_PORT}.proxy.runpod.net/ws'

runpod.serverless.progress_update(job, {'WS_URL': WS_URL})

POD_ID = os.getenv('RUNPOD_POD_ID')
WS_PORT = 8888
WS_URL = f'wss://{POD_ID}-{WS_PORT}.proxy.runpod.net/ws'

runpod.serverless.progress_update(job, {'WS_URL': WS_URL})

The URL will be accessible in the status response:

{
  "id": "your-unique-id-will-be-here",
  "output": {
    "WS_URL": "wss://you_pod_id-8888.proxy.runpod.net/ws"
  },
  "status": "IN_PROGRESS"
}

{
  "id": "your-unique-id-will-be-here",
  "output": {
    "WS_URL": "wss://you_pod_id-8888.proxy.runpod.net/ws"
  },
  "status": "IN_PROGRESS"
}

In this example, I am providing a web socket URL that is passed to the browser, where it creates a web socket connection to the serverless worker. You can expose a TCP port instead and this will allow you to do most anything but you have to run your own code for running HTTP/S server, SSL certificate (if HTTPS required), and some kind of dynamic DNS. Even in that case each system will have to have it's own unique DNS name.

inc3pt.ioOP•4mo ago

Thank you @Encyrption for your continuous help! You pointed to me that the proxy URL was based under the pod ID. So with that info, I can now access the ID with the following GraphQL request:

query Endpoints { myself { endpoints { pods { machine { podHostId } } }}}

query Endpoints { myself { endpoints { pods { machine { podHostId } } }}}

Even though your idea of fetching the progress status can fill the need, I needed to get the information without changing the serverless handler logic. Thanks again!

Encyrption•4mo ago

@inc3pt.io That's excellent! I am just now dipping my toes into the GraphQL water for the 1st time. I need to figure out how to query for the following, given a pod id.

PodTelemetry
    cpuUtilization
    
PodTelemetry
    individualGpuMetrics
        memoryUtilization

PodTelemetry
    cpuUtilization
    
PodTelemetry
    individualGpuMetrics
        memoryUtilization

Gaming

Programming

Exposing HTTP services in Endpoints through GraphQL