How do I expose my api key and use CORS instead?
I want to make it so that all requests from a domain to my serverless endpoint are allowed. I suppose I don't mind exposing my api key if I can make it so that only requests from a certain domain are allowed, right?
How would I do this? I want to serve a Comfy workflow on a serverless endpoint and I think I can use https://github.com/blib-la/runpod-worker-comfy to set up the endpoint itself. It would be really helpful if a) someone could let me know if this is possible, and if so b) outline the general steps I need to do to accomplish it.
GitHub
GitHub - blib-la/runpod-worker-comfy: ComfyUI as a serverless API o...
ComfyUI as a serverless API on RunPod. Contribute to blib-la/runpod-worker-comfy development by creating an account on GitHub.
37 Replies
Wah this is an awesome repository
Tldr though if you launch this on runpod serverless, runpod serverless itself managed an api endpoint to trigger
Overview | RunPod Documentation
An overview to Serverless GPU computing for AI inference and training.
Here is their doc
the main idea behind serverless is you define a python function such as:
And you can invoke it through curl ; their python sdk; or python request module:
request runpodurl/<your serverless id>/run for example, and it will trigger that python function
I guess the tldr im trying to tell u is that, if this is a repo for comfy ui serverless then its already basically already a ready to go api that when u make a request to runpod need to do so with an auth key anyways
@justin hmm is this function you're talking about something I can change? Because if so, I coul just change it to be like "if the request is coming from domain x, then allow. Otherwise, reject"
Does something as simple as that work? I think this is in the handler.py code (for this repository specifically or really any other worker repo for runpod)
Yes u can define the handler.py to respond however u would like
but u dont need to change it to do a domain authorization
to make a request to the api u need an authentication key
can read more on serverless for that information in the docs
so ur domain essentially on its requests just needs to pass the auth key with its request
u dont need to specifically define by domains
this serverless btw is to spin up gpus dynamically up and down to respond to requests
rather than keeping a gpu pod instance open 24/7 which is what gpu pod does
gpu pod is more of renting a linux server with a GPU for manual usage; usually; vs serverless is more of an api sort of usage
Okay but I guess what I'm asking is I have to make this request from the frontend (because of some weird shopify stuff) and so I would have to expose the api key. Thus, would want to restrsict it to just a certain domain
or some other form of protection
Got it. You could probably have the shopify make a request to a django / flask server u host on fly.io or something
and then proxy that request through to runpod 🙂
is my recommendation
but that is up to u ultimately
ok cool that makes more sense. I guess, just wanted your opinion--is that safer because then my proxy server can figure out if the request is legit or not? Just from the ip the request is coming form?
vs I can't do that if I call the runpod handler directly
Well, I guess your proxy server can do anything haha
But yes
I mean ppl can fake whatever request structure they want to a backend api
but i guess, u can force a shopify button maybe to pass some token or something
and rate limit it etc
on the proxy etc
If ur worried about security
But yea
why wouldn't you recommend trying to do that in the runpod handler itself? Like why not just add some python to restrict the handler.py file to only run if it's from a legit domain and theres a rate limit?
that way I don't have to setup a proxy server
Bc by exposing ur authentication key they can do a lot of stuff on runpod
like spin up pods themselves
modify ur endpoint
etc
oh okay got it
makes more sense now
any pointers on where to look just for some basic information on proxy servers with runpod? like a guide or even just what search term you'd recommend
thank you so much btw, this is really helpful
Hm. Essentially I think first, just learn how to call the serverless function on runpod.
https://discord.com/channels/912829806415085598/1194695853026328626/1194998123781685258
You can refer to this hello world example of serverless on runpod + also the docs:
https://docs.runpod.io/serverless/overview
Overview | RunPod Documentation
An overview to Serverless GPU computing for AI inference and training.
You can deploy one + also just learn how to call it
that will give you an idea of how to write a basic python function to call ur api
from there, all you need is a hello world flask application, to deploy on fly.io
You can ask chatgpt + fly.io has great documentation on a basic flask server setup
got it. Would I send a request from my client to the flask server, the flask server would send a request to my serverless endpoint with some sort of address of the client to send the image back to, and then the serverless endpoint would create the image and send it to the address the flask server gave it
?
So then essentially now your architecture looks like:
1) Some frontend
2) Pings a scalable backend flask server (which you can do rate limiting, ip address tracking, user sessions etc, ping out to any database whatever)
3) This can ping to serverless endpoint
Exactly
Or would the serverless endpoint return image to the flask server first, and then the flask server returns to the client
Runpod if you use /run which is async will actually store it for 30 mins the server side; which u can check through a /status endpoint
so flask will get a job_id, and u can return this back to the client
who can start polling against runpod directly on the /status endpoint
if u want
or u can have runpod use a /webhook
which can ping ur server on flask, to do something else when the job is done and give you the output; if u wanna do logging or something; or store into a database etc
SO u can also do stuff like that
or actually sorry, the client cannot check b/c of auth keys issue haha
so i guess flask
1) Sorry:
a) Send a request to runpod on the /run which ull get a job id
b) U can poll this job id on the /status for the job output
or c) you can also provide a webhook to ping for when the job is done
-> which then u can respond back to the client from flask
so flow looks like: client makes request; flask server receives and does some auth, rate limiting, etc then sends to serverless, serverless completes the job and gives flask back a job_id, flask server gives the job_id to client, and then the client can use the job_id since that's public to see the image?
oh wait, let me also look at your most recent messages
Yeah sorry just that last part is slightly off, the client prob starts polling against ur flask server
which proxies the request over to runpod to check the /status endpoint
does my client get the image directly from the endpoint or does it need to be piped through the flask server
Probably pipped through the flask server.
Usually runpod accepts inputs and outputs only in json text, so the image would be in base64, so the client would just get a base64 that they decode and render out as an image
Or depending on what you do on runpod, for ex. i on runpod upload to a file storage on google firebase, where I can get a public URL
so that lets me just have my clients usually directly render out that image / media / or download directly
But everything i guess end of the day is pipped through flask
Since flask has the auth key
and the client polls the flask server to understand when the image has been uploaded to firebase?
Yeah
[deleted] my last msgs bc i feel could be bad info actually
@justin nw, one more question if you have a second--I know in the request to my endpoint I'd typically send my runpod api key. Where is that api key authenticated? Like in a woker repo I could use for my own project (like the one from earlier https://github.com/blib-la/runpod-worker-comfy), is authentication with api key typically handled inside of that code? I looked at that code and couldn't see where it handles the runpod api auth
Invoke a Job | RunPod Documentation
Run Endpoints in RunPod.
Here in the header u would pass in the authorization key
in the header
I guess I'm more so asking where is the actual authentication completed? Like can I see the literal function that does the authentication and modify something/modify the key before I let it get authenticated?
Ah the auth is done on runpod's end
when u make the request to runpod it will try to read ur authentication header, and if it is correct will pass it through
Ok I figured. But can I modify the api key in my worker handler or something before runpod authenticates is?
*it
ah no
the worker handler doesnt know anything
other than ur input
body
It only knows:
{"input": {"prompt": "Your prompt"}}
id be curious why u wanna change the auth key
as that is static
if u want it to know the client who sent the request, u can in the body just add a custom token
that ties back to a different database of urs tho
ok okay. So basically, when I make, say, a curl request, it first goes to runpod auth, runpod verifies that the api key is authentic for someone trying to access the endpoint given in the POST request, and then only does runpod trigger the worker to complete the thing in the body?
Yes accurate
Ok cool. I think I understand everything now and will also look more thoroughly through the serverless docs to understand better. Yeah, I was just interested in passing in some data but could just use a custom token as you said.
I guess my general question is, is there any faster way to send a response straight from the client and get a response, other than to set up a proxy server? Or is that the only way to do it without having a true backend with environment variables
I dont think there is a faster way
Cause clientside authentication is just a dangerous thing in general and runpod doesn't have scoped api keys yet
Got it Justin, again, thanks so much. I'll go ahead and set up a proxy server