runpod nginx networking
hi all!
Just started working with runpod and im quite impressed with the affordability but im having some trouble now that im building an application that uses nginx to send users down different ports
so this works perfectly locally, i have an app that has services running on multiple ports and the user connects to just one port then goes to various /paths and gets connected to the different services using nginx's proxy. The idea is to just have one port open and filter access through it in nginx (because runpod also doesnt have a firewall feature it seems)
The problem is that when i go to the runpodurl/path i get redirected to an ip then that path, which times out since its incorrect. If i expose the port then i can visit the service normally through the different port's url, but that's not what i want, the idea is to just have a single port. I really want to get this running in runpod, in aws this works very seemlessly since the networking is simple and what you expect of public cloud, but its a lot more expensive. It would suck if I cant deploy my application in runpod just because of this. I feel that firewalls between the compute im paying for and the internet is a bare minimum for a cloud provider to have. Whatever solution I would have to implement now would seem very hacky and i dont see why that's neccesary, so some feedback on the service there. Other than that im really happy with the simplicity so appreciate you guys making it!
Has anyone else experienced this issue this issue and can suggest a solution? Thanks in advance!
12 Replies
server {
listen 6969;
# Frontend — proxy all requests to port 3000
location / {
proxy_pass http://127.0.0.1:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
# Stable Diffusion WebUI
location /sd/ {
proxy_pass http://127.0.0.1:7860/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
# LLM server
location /llm/ {
proxy_pass http://127.0.0.1:1337/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
# API
location /api/ {
proxy_pass http://127.0.0.1:8000/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
heres my nginx conf, the frontend works fine interestingly
so i want to connect over all of these paths/ports by exposing only one port
I’m guessing you exposed http port 6969 through our UI, and when you try to send a request to portid-6969.proxy.runpod.net/sd/, it should be forwarding the traffic to Stable Diffusion server, but right now that’s not working?
Hi! Thanks for the swift reaponse, yep thats exactly the case
does your nginx log the request?
if the frontend works, maybe the other services arent up? or inaccessible by nginx?
Inside the container i could curl localhost:1337 and got what i expected when its up
Not sure how to test if its accessible but i had nothing in the error log
And no access on that url in the access log, like i see it when i test locally
So it seems that the runpod proxy somehow causes issues when reverse proxying like this
Like the url/path doesnt actually go there
I see yeah probably
I'd suggest you use tcp ports instead of http then
But you'll use a public ip instead
Lmk if that works
maybe you can do sth like port 6969/* -> localhost:3000
and then curl random address (sth like https://proxy.runpod.net/yourpath/somepath)
and then see the logs for the actual url
Did you mean adding * in the location (nginx config)
i mean forward everything to some dummy server and then see the request logs
then we can know where the proxy is requesting to
just tested using TCP instead, still it creates an ip:port and works similar to HTTPS; if i just have one port then go to the /path for the other one it times out, if i expose that port then i can access it on its newly created ip and port
perhaps the nginx routing should use the runpod ports though? i think thats whats going wrong
That's what people usually do I think. Not sure why path based routing like the config used above doesn't work
yeah even if i use the runpod ports i get i have the same issue, it doesnt seem to be possible to use nginx to route traffic to different pods using runpod and the main culprit seems to be this proxy on runpod's side, both for http and tcp
as a workaround i just implemented separate IP whitelists for the other ports
so this solves my issue but imo it would be nice to be able to opt out of this proxying and just get a public IP that i can control myself
thanks for your help though!
im gonna mark this as resolved since i dont think theres anything that can be done to fix it but perhaps some feedback to make this more robust for service deployments like mine 🙂