R
RunPod•2mo ago
octopus

Huggingface space on Serverless. How to get the Gradio API string which is the same as Worker ID?

I deployed Huggingface Space which use Gradio. If I have worker ID then I can connect to the worker usually like https://${workerID}-proxy.runpod.net/ How can I either the available workerIDs or forward my request from serverless endpoint to Gradio API which uses something like: 
import { Client } from "@gradio/client";

const response_0 = await fetch("https://raw.githubusercontent.com/gradio-app/gradio/main/test/test_files/bus.png");
const exampleImage = await response_0.blob();
                        
const client = await Client.connect("https://${workerID}-7860.proxy.runpod.net/");
const result = await client.predict("/stream_chat", { 
                input_image: exampleImage, 
});

console.log(result.data);
import { Client } from "@gradio/client";

const response_0 = await fetch("https://raw.githubusercontent.com/gradio-app/gradio/main/test/test_files/bus.png");
const exampleImage = await response_0.blob();
                        
const client = await Client.connect("https://${workerID}-7860.proxy.runpod.net/");
const result = await client.predict("/stream_chat", { 
                input_image: exampleImage, 
});

console.log(result.data);
5 Replies
nerdylive
nerdylive•2mo ago
you want a way to pass on the workerID from the worker?
nerdylive
nerdylive•2mo ago
maybe you return it using https://docs.runpod.io/serverless/workers/handlers/handler-generator and use the /stream in the endpoint to collect the result that you return from worker, without ending the job
Generator Handler | RunPod Documentation
RunPod offers real-time streaming for Language Model tasks, providing users with instant updates on job outputs. Two types of generator functions are supported, including regular and async generators, with the option to enable aggregate streaming for seamless access to results.
nerdylive
nerdylive•2mo ago
or with like 
runpod.serverless.progress_update(job, {'POD_ID': POD_ID})
runpod.serverless.progress_update(job, {'POD_ID': POD_ID})
from https://discord.com/channels/912829806415085598/1238153273077141574
niroknox
niroknox•2mo ago
When you queue the job, you can get the workerID checking the queued request, and perhaps 'push' it somewhere like a txt file or even serve it via rest api and the worker can fetch it. also worth checking the environment it might be there?
nerdylive
nerdylive•2mo ago
You hit /run, then hit /status so you can check Worker can fetch it? Wait you're retreving the worker Id inside the worker? You can read the env variables for that

Did you find this page helpful?