Does browslerless support headful mode?

I followed the docs, implemented this and still struggling:

import puppeteer from "puppeteer-core";

const launchArgs = JSON.stringify({
  args: [`--window-size=1920,1080`, `--user-data-dir=/tmp/chrome/data-dir`],
  headless: false,
  stealth: true,
  timeout: 5000,
});

const browser = await puppeteer.connect({
  browserWSEndpoint: `wss://production-sfo.browserless.io/?token=GOES-HERE&launch=${launchArgs}`,
});

import puppeteer from "puppeteer-core";

const launchArgs = JSON.stringify({
  args: [`--window-size=1920,1080`, `--user-data-dir=/tmp/chrome/data-dir`],
  headless: false,
  stealth: true,
  timeout: 5000,
});

const browser = await puppeteer.connect({
  browserWSEndpoint: `wss://production-sfo.browserless.io/?token=GOES-HERE&launch=${launchArgs}`,
});

I'm getting: Error: Unexpected server response: 500

Solution:

it works now, and all i did was switch to browserless v1. stock template, no xvfb....

Jump to solution

100 Replies

Percy•3mo ago

Project ID: d2a990ba-9a9a-40aa-9414-f3d1ae9e016f

AshwinOP•3mo ago

d2a990ba-9a9a-40aa-9414-f3d1ae9e016f Note: the above is an example from the docs and not my code (links not configured etc)

Brody•3mo ago

railway is a headless environment, so I'd say no its not going to be supported

AshwinOP•3mo ago

Oh I see, thanks Any idea if I can load a stream from a wss in headless mode?

Brody•3mo ago

I don't see why not, I also don't see how headless or not would come into play here

AshwinOP•3mo ago

The stream I'm trying to load is from my own app - and can change the page I'm visiting with puppeteer Hm I see, I can't get the wss to load for some reason How I know the stream isn't loading

const targetWsUrl = "wss://url/rtc";
  let wsConnected = false;

  page.on("request", (request) => {
    console.log("Request:", request.url());
    const url = request.url();
    if (url.startsWith(targetWsUrl)) {
      wsConnected = true;

const targetWsUrl = "wss://url/rtc";
  let wsConnected = false;

  page.on("request", (request) => {
    console.log("Request:", request.url());
    const url = request.url();
    if (url.startsWith(targetWsUrl)) {
      wsConnected = true;

Brody•3mo ago

hmmm, I don't really know what to tell you here

AshwinOP•3mo ago

Any args for me to run that you think might help maybe?

Brody•3mo ago

sorry, I don't have any real experience with puppeteer

AshwinOP•3mo ago

Okie, thanks anyways :) @Brody is there any other way you know of to run Browserless in headful mode?

Brody•3mo ago

well does something error out when you tell it to run with a head?

AshwinOP•3mo ago

It works fine in headful, only issue is with headless Which it turns out WebRTC streams aren't supported in headless mode

Brody•3mo ago

right but can you not run it in headfull mode?

AshwinOP•3mo ago

Correct

Brody•3mo ago

then there you have it I also noticed you are using browserless.io, instead of self hosting browserless on railway? not that it would fix anything

AshwinOP•3mo ago

Oh I'm using the Railway instance - I just copied that off their docs

Brody•3mo ago

ah okay cool

AshwinOP•3mo ago

Out of curiosity, why doesn't Railway support headful mode?

Brody•3mo ago

it's a docker environment, there are no heads, nothing specific to railway

AshwinOP•3mo ago

Oh I see, so if I added something like x11 into the container it would work?

Brody•3mo ago

yeah, you'd need to emulate something like that if you give me several hours, i may be able to implement that for you

AshwinOP•3mo ago

Oh!! I would super super appreciate that

Brody•3mo ago

id just need you to provide me with a minimal reducible example that i can run so that tells me / shows me that ive started chrome in headfull mode, of course this example should also provide some indictor or error if i try to start chrome in headfull mode and browserless isnt supporting that, know what i mean?

AshwinOP•3mo ago

Sure let me work on that

Brody•3mo ago

i say several hours because im just about to head out, so ill work on it when im back but that gives you time to work on the example!

AshwinOP•3mo ago

Works perfectly, thanks a bunch :)

Brody•3mo ago

no problem!

AshwinOP•3mo ago

"use server";

import puppeteer from "puppeteer-core";

export const captureThumbnail = async (username: string) => {
  const launchArgs = JSON.stringify({
    stealth: true,
    args: [
      `--use-fake-device-for-media-stream`,
      `--use-fake-ui-for-media-stream`,
      `--no-sandbox`,
    ],
  });

  console.log("Capturing thumbnail for:", username);
  const browser = await puppeteer.connect({
    browserWSEndpoint: `${process.env.BROWSER_WS_ENDPOINT}&launch=${launchArgs}`,
  });

  const page = (await browser.pages())[0];

  await page.setViewport({ width: 640, height: 360 });

  const targetWsUrl = "wss://";
  let wsConnected = false;

  page.on("request", (request) => {
    console.log("Request:", request.url());
    const url = request.url();
    if (url.startsWith(targetWsUrl)) {
      wsConnected = true;
      console.log("WebSocket connection established:", url);

      const screenshot = async () => {

        const screenshot =   await page.screenshot({
          type: "png",
          fullPage: true,
        });

       console.log("Screenshot succesful")


      };
      procedure();
    }
  });

  await page.goto(`${page_url}`);

  while (!wsConnected) {
    console.log("Waiting for WebSocket connection...");
    await new Promise((resolve) => setTimeout(resolve, 1000));
  }
};

"use server";

import puppeteer from "puppeteer-core";

export const captureThumbnail = async (username: string) => {
  const launchArgs = JSON.stringify({
    stealth: true,
    args: [
      `--use-fake-device-for-media-stream`,
      `--use-fake-ui-for-media-stream`,
      `--no-sandbox`,
    ],
  });

  console.log("Capturing thumbnail for:", username);
  const browser = await puppeteer.connect({
    browserWSEndpoint: `${process.env.BROWSER_WS_ENDPOINT}&launch=${launchArgs}`,
  });

  const page = (await browser.pages())[0];

  await page.setViewport({ width: 640, height: 360 });

  const targetWsUrl = "wss://";
  let wsConnected = false;

  page.on("request", (request) => {
    console.log("Request:", request.url());
    const url = request.url();
    if (url.startsWith(targetWsUrl)) {
      wsConnected = true;
      console.log("WebSocket connection established:", url);

      const screenshot = async () => {

        const screenshot =   await page.screenshot({
          type: "png",
          fullPage: true,
        });

       console.log("Screenshot succesful")


      };
      procedure();
    }
  });

  await page.goto(`${page_url}`);

  while (!wsConnected) {
    console.log("Waiting for WebSocket connection...");
    await new Promise((resolve) => setTimeout(resolve, 1000));
  }
};

Brody•3mo ago

mind throwing that in a repo so I can simply run it in a service?

AshwinOP•3mo ago

Sure!

Brody•3mo ago

full repo that has everything I'd need to trigger that code to run and do it's thing

AshwinOP•3mo ago

What's your github?

Brody•3mo ago

brody192

AshwinOP•3mo ago

Done, just sent over the repo to you

Brody•3mo ago

it doesn't look very minimal

AshwinOP•3mo ago

I guess it does crash when headful is unsupported while requesting it in the .connect Will update Done, just pushed the simplified version

Brody•3mo ago

haha I think you are missing the point of minimal

AshwinOP•3mo ago

Ok I might be 😭 What do you mean by it?

Brody•3mo ago

not a full project, just like a single script that proves the issue / proves it's fixed

AshwinOP•3mo ago

Oh, I thought of doing that but wasn't sure how it would be trigerred on Railway easily so I resorted to the existing

Brody•3mo ago

you'd have your package.json setup properly, right now you have a full app

AshwinOP•3mo ago

Sorry about that, want me to rewrite it?

Brody•3mo ago

I'd just like a minimal reproducible example, emphasis on minimal

AshwinOP•3mo ago

Done, just pushed it

Brody•3mo ago

okay perfect, now that's minimal one question, how have you been able to verify that this code works? run it with your local chrome install instead of browserless?

AshwinOP•3mo ago

Yes, I wrote it with normal puppeteer initially and then modified it for browserless Will test it out again with the minimal version

Brody•3mo ago

okay perfect

AshwinOP•3mo ago

Seems like there's an issue with some newer updates I made to it, let me fix and push Okay fixed - when you run local.ts it uses normal puppeteer, when you run index.ts it uses browserless You'll be able to see the difference it output between the two (and sorry for the hassle - I should have tested the newest version with all the new stuff I added on while debugging) (Headless always returns a blank screen)

Brody•3mo ago

okay cool, what version is going to run by default when deployed to railway?

AshwinOP•3mo ago

The browserless version

Brody•3mo ago

perf

Brody•3mo ago

i dont have bun locally (windows without WSL) so how can i verify that this is not working?

Brody•3mo ago

how fast could you add in a web server to serve that image?

AshwinOP•3mo ago

I can start the stream, and if the file size increases we'll know it's working

Brody•3mo ago

Uint8Array(4190) would increase?

AshwinOP•3mo ago

Just a rough guess - let me get back to you on this Yes it would

Brody•3mo ago

might help to have a web server to serve that image?

AshwinOP•3mo ago

Okay let me see

Brody•3mo ago

i know i said minimal but id still need a way to verify my findings since i cant run this locally

AshwinOP•3mo ago

Ofcourse - working on a webserver to serve the image

Brody•3mo ago

web server or convert to plain old node, whatever is easier

AshwinOP•3mo ago

Pushed

Brody•3mo ago

sweet

AshwinOP•3mo ago

http://{domain}/thumbnail/the_winzy

Brody•3mo ago

this was such a good feature

AshwinOP•3mo ago

Oh I didn't even know that was a feature W feature

Brody•3mo ago

yeah i mean in this scenario i trust what you are pushing but its a good security feature to have

AshwinOP•3mo ago

Yeah 100%

Brody•3mo ago

and you said if the image is black its not working?

AshwinOP•3mo ago

Correct If it's just all black and no text/icons

Brody•3mo ago

gotcha

AshwinOP•3mo ago

Uint8Array(4190) should mean it's working as far as I can tell But that's just a terrible way to check lol

Brody•3mo ago

thats funny since i didnt implement the virtual display server yet

AshwinOP•3mo ago

Oh??

Brody•3mo ago

are you using browserless v1 or v2

AshwinOP•3mo ago

Brody•3mo ago

same

AshwinOP•3mo ago

Ok we should probs see the result from the endpoint

Brody•3mo ago

waiting for dns to update so i can see the image

AshwinOP•3mo ago

Nvm, I got mixed up - I have the stream running now so it should be easier to see if it works or not

Brody•3mo ago

dns still isnt updated yet lol

AshwinOP•3mo ago

You'll see my screen if it's working or a black screen Oh xD

Brody•3mo ago

it is indeed black https://headful-test-v2-production.up.railway.app/thumbnail/the_winzy

AshwinOP•3mo ago

I rlly hope X11 fixes this

Brody•3mo ago

me too

AshwinOP•3mo ago

:prayge: Thanks again for all the effort, super appreciate it

Brody•3mo ago

of course! you are running it in headless mode? i thought you wanted headless false? https://github.com/ashwwwin/headful-test/blob/main/index.ts#L19

AshwinOP•3mo ago

Ooops I forgot to change that back Pushed

Brody•3mo ago

awsome is the stream running?

AshwinOP•3mo ago

Yep

Brody•3mo ago

still black even with xvfb running

AshwinOP•3mo ago

:( The browserless instance doesn't crash though while connecting with headless: false?

Brody•3mo ago

correct

AshwinOP•3mo ago

sighs will have to dig in further tomorrow when I get up, almost 9 am for me

Brody•3mo ago

great sleep schedule lol can you leave the stream running?

AshwinOP•3mo ago

Could you share an easy way for me to replicate the instance with headful mode on? For sure

Brody•3mo ago

this is what ive done so far https://github.com/railwayapp-templates/browserless-v2/tree/xvfb

Solution

Brody•3mo ago

it works now, and all i did was switch to browserless v1. stock template, no xvfb.

AshwinOP•3mo ago

Interesting - no idea why that works but, thank you so so much :patrikdab: Literally saved me from another few days of debugging :)

Brody•3mo ago

no problem!

Gaming

Programming

Does browslerless support headful mode?