Railway•2y ago

How to get chrome in a railway deployment?

I want to deploy a small nodejs cron job which has a dependancy on chrome. But I do not want to use docker because external applications need to use the files produced by the script. What could be the solution?

79 Replies

Percy•2y ago

Project ID: N/A

ErbiumOP•2y ago

N/A

Brody•2y ago

does your app use puppeteer?

ErbiumOP•2y ago

yes

Brody•2y ago

puppeteer should install chrome itself what issues are you running into?

ErbiumOP•2y ago

it kept showing chrome not found

Brody•2y ago

send your package.json please

ErbiumOP•2y ago

Should I send the deploy logs?

Brody•2y ago

package.json first

ErbiumOP•2y ago

{
  "devDependencies": {
    "ts-node": "^10.9.1",
    "typescript": "^5.0.4"
  },
  "dependencies": {
    "webdriverio": "^8.10.5"
  },
  "scripts": {
    "start": "npx ts-node dailyrunner.ts"
  }
}

{
  "devDependencies": {
    "ts-node": "^10.9.1",
    "typescript": "^5.0.4"
  },
  "dependencies": {
    "webdriverio": "^8.10.5"
  },
  "scripts": {
    "start": "npx ts-node dailyrunner.ts"
  }
}

Brody•2y ago

does your app use puppeteer?

ErbiumOP•2y ago

webdriverio uses puppeteer to launch chrome

Brody•2y ago

not quite, it uses puppeteer-core, the core variant does not install chrome

ErbiumOP•2y ago

Brody•2y ago

you might be able to get away with chucking puppeteer in as a dep

ErbiumOP•2y ago

so basically adding puppeteer in the dependancies? but wouldnt webdriverio still use puppeteer-core

Brody•2y ago

yeah but during the install of puppeteer it will install chrome

ErbiumOP•2y ago

alright then

Brody•2y ago

just an idea, untested of course

ErbiumOP•2y ago

@Brody Do I need to configure it to launch chrome in headless mode or graphical mode will work?

Brody•2y ago

headless

ErbiumOP•2y ago

it still show the same error

Brody•2y ago

show error please

ErbiumOP•2y ago

at new LauncherError (/app/node_modules/chrome-launcher/src/utils.ts:31:18)
at new ChromePathNotSetError (/app/node_modules/chrome-launcher/dist/utils.js:33:9)
at Object.linux (/app/node_modules/chrome-launcher/src/chrome-finder.ts:153:11)
at Function.getFirstInstallation (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:192:61)
at Launcher.launch (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:266:37)
at launch (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:73:18)
at launchChrome (file:///app/node_modules/devtools/build/launcher.js:89:26)
at launch (file:///app/node_modules/devtools/build/launcher.js:199:16)
at Function.newSession (file:///app/node_modules/devtools/build/index.js:34:31)
at remote (file:///app/node_modules/webdriverio/build/index.js:48:43) {
message: 'The CHROME_PATH environment variable must be set to a Chrome/Chromium executable no older than Chrome stable.',
code: 'ERR_LAUNCHER_PATH_NOT_SET' }

at new LauncherError (/app/node_modules/chrome-launcher/src/utils.ts:31:18)
at new ChromePathNotSetError (/app/node_modules/chrome-launcher/dist/utils.js:33:9)
at Object.linux (/app/node_modules/chrome-launcher/src/chrome-finder.ts:153:11)
at Function.getFirstInstallation (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:192:61)
at Launcher.launch (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:266:37)
at launch (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:73:18)
at launchChrome (file:///app/node_modules/devtools/build/launcher.js:89:26)
at launch (file:///app/node_modules/devtools/build/launcher.js:199:16)
at Function.newSession (file:///app/node_modules/devtools/build/index.js:34:31)
at remote (file:///app/node_modules/webdriverio/build/index.js:48:43) {
message: 'The CHROME_PATH environment variable must be set to a Chrome/Chromium executable no older than Chrome stable.',
code: 'ERR_LAUNCHER_PATH_NOT_SET' }

Brody•2y ago

before we go any further, can you tell us what website you are trying to scrape?

ErbiumOP•2y ago

bseindia.com

Brody•2y ago

do they allow web scraping?

ErbiumOP•2y ago

it works when done manually

Brody•2y ago

not what I asked unfortunately

ErbiumOP•2y ago

I ran the code like 6 times on pc before trying to deploy I worded it wrong basically what I mean is "it works on my machine"

Brody•2y ago

again, not what i asked

ErbiumOP•2y ago

then?

Brody•2y ago

do they allow web scraping?

ErbiumOP•2y ago

yeah

Brody•2y ago

okay cool, can you show me where they give the user permission to scrape their site

ErbiumOP•2y ago

What do you mean by scraping though in this instance?

Brody•2y ago

pulling data from the sites html

ErbiumOP•2y ago

The website does not allow it but I am not doing that I am downloading files which the website provides an interface for so they wouldnt disallow something they literally have made

Brody•2y ago

oh so they have an offical api?

ErbiumOP•2y ago

yeah

Brody•2y ago

and you are using it?

ErbiumOP•2y ago

yes

Brody•2y ago

okay then you dont even need chrome

ErbiumOP•2y ago

I do because I could only figure out some parts of the api

Brody•2y ago

so then you are scraping data

ErbiumOP•2y ago

I do not think you understood

ErbiumOP•2y ago

The code fills in the data clicks submit and then clicks the download button at the top right corner

ErbiumOP•2y ago

how is this scraping

Brody•2y ago

that counts as web scraping, or even botting

ErbiumOP•2y ago

then why are they providing this?

Brody•2y ago

providing what

ErbiumOP•2y ago

this interface

Brody•2y ago

what are you talking about either way, this is clearly breaking railway's TOS please remove this app from railway

ErbiumOP•2y ago

It didnt even deploy

Brody•2y ago

https://railway.app/legal/terms

ErbiumOP•2y ago

Alright so I removed the webscraping part But it still has some dataprocessing that will remain when I try to deploy it says failed to upload code

Brody•2y ago

as long as it uses the official api, without chrome, you are good

ErbiumOP•2y ago

yeah removed the scraping but this is the problem now

Brody•2y ago

show me your package.json please

ErbiumOP•2y ago

{
  "devDependencies": {
    "ts-node": "^10.9.1",
    "typescript": "^5.0.4"
  },
  "scripts": {
    "start": "npx ts-node dailyrunner.ts"
  }
}

{
  "devDependencies": {
    "ts-node": "^10.9.1",
    "typescript": "^5.0.4"
  },
  "scripts": {
    "start": "npx ts-node dailyrunner.ts"
  }
}

Brody•2y ago

are you deploying from github or cli?

ErbiumOP•2y ago

cli

Brody•2y ago

show me the error

ErbiumOP•2y ago

[erbium@lallantop dailyrunner]$ railway up
  Indexed                                                                                                                                                Compressed [====================] 100%                                                                                                                 Failed                                                                                                                                               Failed to upload code

[erbium@lallantop dailyrunner]$ railway up
  Indexed                                                                                                                                                Compressed [====================] 100%                                                                                                                 Failed                                                                                                                                               Failed to upload code

Brody•2y ago

what railway version? railway --version

ErbiumOP•2y ago

railwayapp 3.3.1

Brody•2y ago

what is the total size of your project folder (minus node_modules)

ErbiumOP•2y ago

821 MB

Brody•2y ago

without node_modules!

ErbiumOP•2y ago

yeah without node_modules

Brody•2y ago

what file(s) is making it so big

ErbiumOP•2y ago

as I said it is a data processing application

Brody•2y ago

fair, the upload limit on the cli is 50mb

ErbiumOP•2y ago

Brody•2y ago

you will want to store the big files in something like cloudflare r2, and download them at runtime on railway

ErbiumOP•2y ago

If I want to use a container, I just need to write a dockerfile right no need to build the image?

Brody•2y ago

i dont see what a dockerfile has to do with this

ErbiumOP•2y ago

its unrelated

Brody•2y ago

okay then i also dont understand your question

Gaming

Programming

How to get chrome in a railway deployment?