Wildcard to nginx proxy in cf tunel doesn't work with HTTPS

I have my cf domain which is set to: @ my public ip * cf tunnel In cf tunnel public hostnames I have added *.example.org which points at NGINX using HTTPS and I get gateway error 502, but when I add subdomain one by one it works. I'm trying to not do double work and just use nginx for sub domain config not both of them. HTTP works fine when I disable Force SSL in nginx, but I would like to keep encryption on everywhere Let me know if you need any more details or have ideas
14 Replies
Chaika
Chaika3mo ago
502 means tunnel can't connect to origin. If you add a wildcard public hostname it tells you it won't make a record for it, unless you add it on a single subdomain and then change that to wildcard there would be nothing.
HTTP works fine when I disable Force SSL in nginx, but I would like to keep encryption on everywhere
If the tunnel is running on the same host as nginx there's no reason to have it use https/worry about encryption, as it's User -> Encrypted over Internet -> CF -> Encrypted Over Internet -> Tunnel (cloudflared) running on host -> Unencrypted (same device) -> nginx
iMordo
iMordoOP3mo ago
Yes, I saw the info that DNS record won't be created and I added it manually. Since writing this post I figured out that it works when I put into originServerName any of nginx configured hosts. But then for example uptime kuma decides to not show data after first F5 and then I get the same 502 error. and then the next refresh it works again. But when I use only NGINX or only a tunnel it works fine - can't wrap my head around it. I have considered moving tunnel to nginx host as you suggested but when I did it HTTP stopped working completely with too many redirects error - probably some miscofiguration on my side but I did not dig deeper yet.
Chaika
Chaika3mo ago
I have considered moving tunnel to nginx host as you suggested but when I did it HTTP stopped working completely with too many redirects error - probably some miscofiguration on my side but I did not dig deeper yet.
That's the classic your origin/end service is requiring https and the tunnel host just proxies the redirect back and then tries http again on the next request
Since writing this post I figured out that it works when I put into originServerName any of nginx configured hosts.
so it's an https issue, probably that your cert isn't wildcard/doesn't contain everything. When you get 502 errors like that from the tunnel the tunnel logs show always show more info about the failure to connect to origin, ex: journalctl -u cloudflared -f --lines=100 if on systemd linux
But then for example uptime kuma decides to not show data after first F5 and then I get the same 502 error. and then the next refresh it works again. But when I use only NGINX or only a tunnel it works fine - can't wrap my head around it.
You're not running more then one cloudflared connector right? You mentioned you moved the tunnel, make sure you removed/disconnected the old one or it'll randomly pick one to use and could result in behavior like that. In the Cloudflare Zero Trust dashboard under Network -> Tunnels if you click on your tunnel name, it should expand a card from the right, and under "Connectors" there should only be one
iMordo
iMordoOP3mo ago
That's the classic your origin/end service is requiring https and the tunnel host just proxies the redirect back and then tries http again on the next request
Um, I bet there is a setting in cloudflare that would repair that? I tried disabling "Always use HTTPS" and "Automatic HTTPS Rewrites" but that did not do it
so it's an https issue, probably that your cert isn't wildcard/doesn't contain everything. When you get 502 errors like that from the tunnel the tunnel logs show always show more info about the failure to connect to origin, ex: journalctl -u cloudflared -f --lines=100 if on systemd linux
I will paste log in the next one as it's too long.
You're not running more then one cloudflared connector right? You mentioned you moved the tunnel, make sure you removed/disconnected the old one or it'll randomly pick one to use and could result in behavior like that. In the Cloudflare Zero Trust dashboard under Network -> Tunnels if you click on your tunnel name, it should expand a card from the right, and under "Connectors" there should only be one
nope, I shut down the system with old one
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR error="unexpected EOF" connIndex=2 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR Request failed error="unexpected EOF" connIndex=2 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.200.43 type=http
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR error="unexpected EOF" connIndex=0 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR Request failed error="unexpected EOF" connIndex=0 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.192.227 type=http
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR error="unexpected EOF" connIndex=1 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR Request failed error="unexpected EOF" connIndex=1 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.200.23 type=http
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR error="unexpected EOF" connIndex=3 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR Request failed error="unexpected EOF" connIndex=3 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.192.27 type=http
Aug 24 18:08:29 nginx cloudflared[161]: 2024-08-24T18:08:29Z ERR error="unexpected EOF" connIndex=2 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:29 nginx cloudflared[161]: 2024-08-24T18:08:29Z ERR Request failed error="unexpected EOF" connIndex=2 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.200.43 type=http
Aug 24 18:08:29 nginx cloudflared[161]: 2024-08-24T18:08:29Z ERR error="unexpected EOF" connIndex=0 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR error="unexpected EOF" connIndex=2 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR Request failed error="unexpected EOF" connIndex=2 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.200.43 type=http
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR error="unexpected EOF" connIndex=0 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR Request failed error="unexpected EOF" connIndex=0 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.192.227 type=http
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR error="unexpected EOF" connIndex=1 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR Request failed error="unexpected EOF" connIndex=1 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.200.23 type=http
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR error="unexpected EOF" connIndex=3 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:27 nginx cloudflared[161]: 2024-08-24T18:08:27Z ERR Request failed error="unexpected EOF" connIndex=3 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.192.27 type=http
Aug 24 18:08:29 nginx cloudflared[161]: 2024-08-24T18:08:29Z ERR error="unexpected EOF" connIndex=2 event=1 ingressRule=0 originService=https://127.0.0.1
Aug 24 18:08:29 nginx cloudflared[161]: 2024-08-24T18:08:29Z ERR Request failed error="unexpected EOF" connIndex=2 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.200.43 type=http
Aug 24 18:08:29 nginx cloudflared[161]: 2024-08-24T18:08:29Z ERR error="unexpected EOF" connIndex=0 event=1 ingressRule=0 originService=https://127.0.0.1
By the way thank you very much for trying to help
Chaika
Chaika3mo ago
Um, I bet there is a setting in cloudflare that would repair that? I tried disabling "Always use HTTPS" and "Automatic HTTPS Rewrites" but that did not do it
No that issue is purely with your origin. You'd have to get it to stop redirecting http -> https (or connect over https from the tunnel service but as discussed above that has its own issues/isn't needed if the tunnel is local)
Aug 24 18:08:29 nginx cloudflared[161]: 2024-08-24T18:08:29Z ERR Request failed error="unexpected EOF" connIndex=2 dest=https://status.imordo.com/api/status-page/heartbeat/all event=0 ip=198.41.200.43 type=http Aug 24 18:08:29 nginx cloudflared[161]: 2024-08-24T18:08:29Z ERR error="unexpected EOF" connIndex=0 event=1 ingressRule=0 originService=https://127.0.0.1
iirc you can get that error if you're trying to connect to an http service but thinking it's https if you connect locally via curl does it work?
By the way thank you very much for trying to help
sure
iMordo
iMordoOP3mo ago
iirc you can get that error if you're trying to connect to an http service but thinking it's https if you connect locally via curl does it work?
yes curl is fine a local connection to NGINX using local DNS is also fine. Just when it goes thru cloudflare and then nginx it has issues
No description
Chaika
Chaika3mo ago
idk what configuration that screenshot is from but
originService=https://127.0.0.1
this indictates that it's https, but that show http
iMordo
iMordoOP3mo ago
No that issue is purely with your origin. You'd have to get it to stop redirecting http -> https (or connect over https from the tunnel service but as discussed above that has its own issues/isn't needed if the tunnel is local)
So when I disable force SSL a tunnel should still stay at HTTPS?
No description
Chaika
Chaika3mo ago
if the tunnel and the service the tunnel is connecting to is local you can avoid the https/cert issues/extra overhead there by forgoing http entirely. Disable force ssl, make tunnel service http, make sure your origin behind nginx isn't redirecting to https either
iMordo
iMordoOP3mo ago
yes as it's configured as wildcard in cloudflare which goes to NGINX proxy well, every service is on a different host, only nginx and cloudflared are together now
Chaika
Chaika3mo ago
well even if it's within the same LAN that no one has access to doing plaintext over it is generally secure, I assume what's what nginx is doing now anyway
iMordo
iMordoOP3mo ago
With some hosts that can't do HTTPS yes, but I have some that are HTTPS only and with firewall only nginx can go to these hosts and is accesible by my LAN and tunnel I have disabled Force SSL on uptime kuma and it still has the same issue. But when I also change tunnel to http://127.0.0.1 it works and uptime kuma is behaving good
Chaika
Chaika3mo ago
yea that's what I meant so, just some issue with https
iMordo
iMordoOP3mo ago
I guess that is why I got confused because other HTTP only hosts which were configured exactly the same worked flawlessly Thank you again then as I got really lost there, consider it closed!
Want results from more Discord servers?
Add your server