Cloudflare Developers•6mo ago

Specify host header in fetch request

I'm trying to health-check load balanced hosts, for which I need to fetch using IP address + Host header. In curl it's super simple, like this:

curl -H "Host: direct.openfreemap.org" -I http://144.76.168.195/styles/liberty

curl -H "Host: direct.openfreemap.org" -I http://144.76.168.195/styles/liberty

However I cannot replicate the same in a Worker JS environment. For example this does not work (network connection lost):

    const resp = await fetch('http://144.76.168.195/styles/liberty', {
      headers: {
        Host: 'direct.openfreemap.org',
      },
    })

    const resp = await fetch('http://144.76.168.195/styles/liberty', {
      headers: {
        Host: 'direct.openfreemap.org',
      },
    })

How can I make either fetch work with Host headers, or use a lower level library for that?

8 Replies

Chaika•6mo ago

it's a security thing, you mostly can't. You can't fetch raw IPs either. Workers do have resolveOverride but the host header would still match the URL and both the url and host in resolveOverride would have be in the same zone/domain the worker is running on. https://developers.cloudflare.com/workers/runtime-apis/request/#incomingrequestcfproperties You could hack something together with TCP Sockets yea: https://developers.cloudflare.com/workers/runtime-apis/tcp-sockets/, I know I've heard of people messing around with that sort of idea before, I don't know of any exact examples though, and TCP Sockets use WARP IPs (undocumented), not the normal CDN IPs Fetch() goes through the entire cdn flow sort of like a normal request, which makes it special

hyperknotOP•6mo ago

I see. It's a bit weird, as the non-JS runtimes can support anything I guess. Like if I write it in Python then I'm allowed to do whatever I want, but if I choose the native JS runtime then I'm in this security sandbox kind of thing?

Chaika•6mo ago

In Cloudflare Workers, everything goes through the JS layer to some extent There is only one runtime supported, and it's V8. Python Workers run via WASM You could check workerd to see how they patch into the javascript side of things, but I imagine they just use fetch with the same restrictions too. It's either that or TCP Sockets

hyperknotOP•6mo ago

I see, thanks for the help!

Chaika•6mo ago

Sure yea looks like both the patches for aiohttp and httpx just use JS Fetch: https://github.com/cloudflare/workerd/blob/c5ebbe29e3b3bcc5b2009b399074fdaca703e865/src/pyodide/internal/patches/httpx.py, should be all the same limitations

grouch•6mo ago

@hyperknot did you ever find a solution to this? i'm doing something similar right now using origin rules but the number of rules i have is starting to become unmanagable.. wanted to move everything to a worker, but it's proving to be more difficult than expected. with origin rules i can set a separate host header and dns address, but i haven't been able to reproduce that functionality in a worker.

hyperknotOP•6mo ago

I moved to Python and host it on a VM, gave up on CF workers for this.

grouch•6mo ago

haha ok i suppose that's one way to do it, not really the way i'm looking to do thanks for the update

Gaming

Programming

Specify host header in fetch request