Cloudflare Developers•2y ago

Page breaks when fetched/returned by worker

I have this public share link: https://view.monday.com/3776000595-36cfd13f4d0b049a330a3952bdae0ff9 and this bit of worker code I copied from the examples:

export default {
  async fetch(request) {
    const url = 'https://view.monday.com/3776000595-36cfd13f4d0b049a330a3952bdae0ff9';

    async function gatherResponse(response) {
      const { headers } = response;
      const contentType = headers.get('content-type') || '';
      if (contentType.includes('application/json')) {
        return JSON.stringify(await response.json());
      } else if (contentType.includes('application/text')) {
        return response.text();
      } else if (contentType.includes('text/html')) {
        return response.text();
      } else {
        return response.text();
      }
    }

    const init = {
      headers: {
        'content-type': 'text/html;charset=UTF-8',
      },
    };

    const response = await fetch(url, request);
    const results = await gatherResponse(response);
    return new Response(results, init);
  },
};

export default {
  async fetch(request) {
    const url = 'https://view.monday.com/3776000595-36cfd13f4d0b049a330a3952bdae0ff9';

    async function gatherResponse(response) {
      const { headers } = response;
      const contentType = headers.get('content-type') || '';
      if (contentType.includes('application/json')) {
        return JSON.stringify(await response.json());
      } else if (contentType.includes('application/text')) {
        return response.text();
      } else if (contentType.includes('text/html')) {
        return response.text();
      } else {
        return response.text();
      }
    }

    const init = {
      headers: {
        'content-type': 'text/html;charset=UTF-8',
      },
    };

    const response = await fetch(url, request);
    const results = await gatherResponse(response);
    return new Response(results, init);
  },
};

If I go directly to the link in my browser the page loads like I expect but if I instead fetch the same page with a worker and return it, then the page is broken. Anyone have any idea what I can do to get this page working?

13 Replies

SpunkieOP•2y ago

Here's an example worker that is running that code above so you can see what I mean by "broken". https://soft-glitter-fa33.polished-geek.workers.dev/

Hello, I’m Allie!•2y ago

What is this Worker supposed to do? Is it just a proxy?

SpunkieOP•2y ago

well eventually the worker should to be able to edit/inject html/styling into the page but for now I was just getting started so this was just a basic test of fetching and returning it.

Hello, I’m Allie!•2y ago

Does this look right? https://mondev.goalastair.com/

SpunkieOP•2y ago

Ya, that is working

Hello, I’m Allie!•2y ago

First big issue, your setup means every asset is returned with the text/html content-type. This works fine if it is HTML, but if it isn't, it tries to interpret JS/CSS/JSON as HTML, which causes problems. Easy to just make it simple:

const hostname = "view.monday.com";
const rootPage = "/3776000595-36cfd13f4d0b049a330a3952bdae0ff9";
export default {
  async fetch(req) {
    const url = new URL(req.url);
    url.hostname = hostname;
    if(url.pathname === "/") {
      url.pathname = rootPage;
    }
    console.log(url.toString());
    return fetch(url, req);
  },
};

const hostname = "view.monday.com";
const rootPage = "/3776000595-36cfd13f4d0b049a330a3952bdae0ff9";
export default {
  async fetch(req) {
    const url = new URL(req.url);
    url.hostname = hostname;
    if(url.pathname === "/") {
      url.pathname = rootPage;
    }
    console.log(url.toString());
    return fetch(url, req);
  },
};

Right now it just returns directly, but you could also intercept the response later on, to add your own stuff to it.

SpunkieOP•2y ago

Ya, I started with this much simpler bit of code before I copied that block from the CF example docs.

export default {
    async fetch(request) {
        const response = await fetch(`https://view.monday.com/3776000595-36cfd13f4d0b049a330a3952bdae0ff9`, request);
        console.log(response);
        return response;
    }
}

export default {
    async fetch(request) {
        const response = await fetch(`https://view.monday.com/3776000595-36cfd13f4d0b049a330a3952bdae0ff9`, request);
        console.log(response);
        return response;
    }
}

But I didn't realize I need to fiddle with the hostname to match. Thanks for the help @HardAtWork MeowHeartCloudflare

Hello, I’m Allie!•2y ago

No problem!

SpunkieOP•2y ago

I feel like none of the snippets I copy from CF examples ever actually work for me thisisfine

const hostname = "view.monday.com";
const path = "/3776000595-36cfd13f4d0b049a330a3952bdae0ff9";

export default {
    async fetch(request) {
        const url = new URL(request.url);
        url.hostname = hostname;
        if(url.pathname === "/") {
            url.pathname = path;
        }
        const response = await fetch(url, request);
        console.log(response);

        const OLD_URL = 'monday.com';
        const NEW_URL = 'tuesday.com';

        class AttributeRewriter {
            constructor(attributeName) {
                this.attributeName = attributeName;
            }
            element(element) {
                const attribute = element.getAttribute(this.attributeName);
                if (attribute) {
                    element.setAttribute(this.attributeName, attribute.replace(OLD_URL, NEW_URL));
                }
            }
        }

        const rewriter = new HTMLRewriter().on('a', new AttributeRewriter('href'));

        const contentType = response.headers.get('Content-Type');

        // If the response is HTML, it can be transformed with
        // HTMLRewriter -- otherwise, it should pass through
        if (contentType.startsWith('text/html')) {
            return rewriter.transform(response);
        } else {
            return response;
        }
    }
}

const hostname = "view.monday.com";
const path = "/3776000595-36cfd13f4d0b049a330a3952bdae0ff9";

export default {
    async fetch(request) {
        const url = new URL(request.url);
        url.hostname = hostname;
        if(url.pathname === "/") {
            url.pathname = path;
        }
        const response = await fetch(url, request);
        console.log(response);

        const OLD_URL = 'monday.com';
        const NEW_URL = 'tuesday.com';

        class AttributeRewriter {
            constructor(attributeName) {
                this.attributeName = attributeName;
            }
            element(element) {
                const attribute = element.getAttribute(this.attributeName);
                if (attribute) {
                    element.setAttribute(this.attributeName, attribute.replace(OLD_URL, NEW_URL));
                }
            }
        }

        const rewriter = new HTMLRewriter().on('a', new AttributeRewriter('href'));

        const contentType = response.headers.get('Content-Type');

        // If the response is HTML, it can be transformed with
        // HTMLRewriter -- otherwise, it should pass through
        if (contentType.startsWith('text/html')) {
            return rewriter.transform(response);
        } else {
            return response;
        }
    }
}

I went and added the example code for HTMLRewriter() but it doesn't seem to do anything. And in general the docs for HTMLRewriter() feels incomprehensible compared to something old school like jQuery I'm used too. @HardAtWork When searching the discord history, it looked like you help a few other people with HTMLRewriter() in the past so I was wondering if you have an idea before I made a whole new thread about it.

Hello, I’m Allie!•2y ago

Let me see...

SpunkieOP•2y ago

Again this was just the barebones test based off the example, where I'm trying the rewrite the "Powered by..." link at the top right to be tuesday.com instead of monday.com.
https://soft-glitter-adfgadfgadfgadf.polished-geek.workers.dev/

Hello, I’m Allie!•2y ago

That may be a bit more difficult. Monday seems to be using Client-Side React, which means the initial HTML the Worker receives doesn't actually contain the Powered by Monday.com button. Rather it is probably included somewhere within one of the JS bundles that the HTML later requests.

SpunkieOP•2y ago

I don't think there is a way to render the page within the worker, and even if there was it might not be very preformant or cpu time efficient.... So I guess instead of messing with the html I need to instead inject some javascript that will wait until the page is rendered to start messing with thisisfine

Gaming

Programming

Page breaks when fetched/returned by worker