Cloudflare Developers•2y ago

Reverse Proxy for blog returns 301 on custom domain, but works correct on preview url?

I've got the domain rank-hub.com added as a custom domain for a worker and a blog lives on ghost.rank-hub.com. The worker script creates a proxy request and returns its result so I can have rank-hub.com/blog proxied to ghost.rank-hub.com/blog. This is very easy with workers and I appreciate it, but I am probably missing a little something as the setup works on the preview url: https://rank-hub-ghost-proxy.divby0.workers.dev/blog but not on the live one https://rank-hub.com/blog Somehow cloudflare returns a 301 to my request when I use the apex domain (so I get redirected to blog.rank-hub.com). When I use the preview url, the content is correctly proxied. What am I missing? Worker script:

export default {
  async fetch(request, env, ctx) {
    try {
      
        const urlObject = new URL(request.url);

        // If the request is to the Ghost subdirectory
        if (/^\/blog/.test(urlObject.pathname)) {
          // Then Proxy to Ghost
          const GHOST_URL = "ghost.rank-hub.com";
          const CUSTOM_URL = "rank-hub.com"; 

          let url = new URL(request.url);

          url.hostname = GHOST_URL;

          let proxyRequest = new Request(url, request);

          proxyRequest.headers.set('Host', GHOST_URL);

          //Have an X-Forwarded-Host header that matches the custom domain in my.ghost.org.

          proxyRequest.headers.set("X-Forwarded-Host", CUSTOM_URL);

          //Include the X-Forwarded-Proto header set to https not http.

          proxyRequest.headers.set("X-Forwarded-Proto", "https");

          //Include the X-Forwarded-For header, populated with the remote IP of the original request.

          let ip = proxyRequest.headers.get("CF-Connecting-IP");

          proxyRequest.headers.set("X-Forwarded-For", ip);

          return await fetch(proxyRequest);
        }

     } catch (error) {
        return await fetch(request);
    }

    return await fetch(request);
  },
};

export default {
  async fetch(request, env, ctx) {
    try {
      
        const urlObject = new URL(request.url);

        // If the request is to the Ghost subdirectory
        if (/^\/blog/.test(urlObject.pathname)) {
          // Then Proxy to Ghost
          const GHOST_URL = "ghost.rank-hub.com";
          const CUSTOM_URL = "rank-hub.com"; 

          let url = new URL(request.url);

          url.hostname = GHOST_URL;

          let proxyRequest = new Request(url, request);

          proxyRequest.headers.set('Host', GHOST_URL);

          //Have an X-Forwarded-Host header that matches the custom domain in my.ghost.org.

          proxyRequest.headers.set("X-Forwarded-Host", CUSTOM_URL);

          //Include the X-Forwarded-Proto header set to https not http.

          proxyRequest.headers.set("X-Forwarded-Proto", "https");

          //Include the X-Forwarded-For header, populated with the remote IP of the original request.

          let ip = proxyRequest.headers.get("CF-Connecting-IP");

          proxyRequest.headers.set("X-Forwarded-For", ip);

          return await fetch(proxyRequest);
        }

     } catch (error) {
        return await fetch(request);
    }

    return await fetch(request);
  },
};

10 Replies

Erisa•2y ago

The redirect is coming from the origin server:

<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.14.0 (Ubuntu)</center>
</body>
</html>

<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.14.0 (Ubuntu)</center>
</body>
</html>

So I suppose the Nginx in front of the Ghost instance took objection to the request in some fashion

divby0OP•2y ago

Ohhhh wait, thanks a lot Erisa 🙂 I'll take a look why exactly that is, this isn't anything with cloudflare then, probably, I'll report back if it is! I didn't came to that conclusion myself because the network tab in chrome and firefox both won't show a 301 response's body. What's weirder, the reponse headers suggest that the 301 redirect is served by cloudflare (which technicall it is of course) and the real content on the subdomain is then served by nginx. That confused me into thinking cloudflare was the problem

Erisa•2y ago

Fair enough yeah - I got that by running curl -v https://rank-hub.com/blog which showed the normal headers and then also the body, which included nginx/1.14.0 (Ubuntu) indicating it didnt come from Cloudflare curl is great for debugging these oddities, though it would have been harder if the origin hadnt given a body

divby0OP•2y ago

Okay, I've debugged around for the last 3 hours and I've found something that I want to call to Cloudflare's attention as it could help other people as well. I'd just like to know opinion on if this is correct or a bug or if I just don't see something that's obvious. Disclaimer because english is not my native lang: I am very thankful for the dev platform that cloudflare provides, I am genuinely curious and want to help, I don't want this to sound like aggressive critique or something. The reason why the redirect was appearing is because this very 301 that you pasted here @Erisa | Support Engineer has been created to upgrade http -> https (nginx does this). Port 80 comes in, redirect to https. But because the https://ghost.rank-hub.com request that the worker sends to my backend has been made on port 80 even though it's https. I didn't notice at first, because the protocol was indeed https, it's just been made on port 80. This is why my redirect nginx block triggered instead of the actual block that serves the blog. Cloudflare didn't do this when I use the preview url. After I researched more, I found the setting inside my zone's SSL/TLS -> Overview -> encryption mode. It was set to flexible by default. After putting it to full (strict), everything works fine. The issue I have with this, is that when I ask the worker to make a https request on port 443 (explicitly setting port), why does cloudflare change this by default: Minimized snippet:

const GHOST_URL = "ghost.rank-hub.com";

let url = new URL(request.url);

url.hostname = GHOST_URL;
url.port = "443";

let proxyRequest = new Request(url, request);

const GHOST_URL = "ghost.rank-hub.com";

let url = new URL(request.url);

url.hostname = GHOST_URL;
url.port = "443";

let proxyRequest = new Request(url, request);

This will send on port 80 if you don't manually change the default flexible to full (strict).

Erisa•2y ago

Thanks for the detailed explanation! I remember seeing an issue like this before but wasn't sure what happened to it - @Vero 🐙 are you able to raise this^ feedback with the Workers team?

divby0OP•2y ago

I appreciate it 🙂 I'll stay tuned to see what happens with this. If I can somehow help or debug something, just ask away in my case there is no real origin, I am just making a random fetch request with javascript for a DNS that is not proxied by CloudFlare (meaning this could easily be an external API call)

Erisa•2y ago

In this case "preview" is referring to workers.dev, which sends the fetch as a cross-zone subrequest and hits the origin as normal, whereas the one on the zone will do a same-zone subrequest that goes straight to the origin And when going straight to the origin that behaviour may be expected to flexiblle, but I can't deny it's confusing since you explicitly specify https and port 443 That said, I haven't looked extensively into it yet, so I could be making some rash assumptions

divby0OP•2y ago

ahhhh wait so even though the ghost.ranke-hub.com domain is NOT proxied via cloudflare (and doesn't point to cloudflare), it is assumed "same-origin"?

Vero•2y ago

sure, sorry I didn't see this, did anyone raise it already? Or can I raise it now

Erisa•2y ago

I didn't

Gaming

Programming

Reverse Proxy for blog returns 301 on custom domain, but works correct on preview url?