How to get chrome in a railway deployment?

I want to deploy a small nodejs cron job which has a dependancy on chrome. But I do not want to use docker because external applications need to use the files produced by the script. What could be the solution?
79 Replies
Percy
Percy2y ago
Project ID: N/A
Erbium
Erbium2y ago
N/A
Brody
Brody2y ago
does your app use puppeteer?
Erbium
Erbium2y ago
yes
Brody
Brody2y ago
puppeteer should install chrome itself what issues are you running into?
Erbium
Erbium2y ago
it kept showing chrome not found
Brody
Brody2y ago
send your package.json please
Erbium
Erbium2y ago
Should I send the deploy logs?
Brody
Brody2y ago
package.json first
Erbium
Erbium2y ago
{
"devDependencies": {
"ts-node": "^10.9.1",
"typescript": "^5.0.4"
},
"dependencies": {
"webdriverio": "^8.10.5"
},
"scripts": {
"start": "npx ts-node dailyrunner.ts"
}
}
{
"devDependencies": {
"ts-node": "^10.9.1",
"typescript": "^5.0.4"
},
"dependencies": {
"webdriverio": "^8.10.5"
},
"scripts": {
"start": "npx ts-node dailyrunner.ts"
}
}
Brody
Brody2y ago
does your app use puppeteer?
Erbium
Erbium2y ago
webdriverio uses puppeteer to launch chrome
Brody
Brody2y ago
not quite, it uses puppeteer-core, the core variant does not install chrome
Erbium
Erbium2y ago
oh
Brody
Brody2y ago
you might be able to get away with chucking puppeteer in as a dep
Erbium
Erbium2y ago
so basically adding puppeteer in the dependancies? but wouldnt webdriverio still use puppeteer-core
Brody
Brody2y ago
yeah but during the install of puppeteer it will install chrome
Erbium
Erbium2y ago
alright then
Brody
Brody2y ago
just an idea, untested of course
Erbium
Erbium2y ago
@Brody Do I need to configure it to launch chrome in headless mode or graphical mode will work?
Brody
Brody2y ago
headless
Erbium
Erbium2y ago
it still show the same error
Brody
Brody2y ago
show error please
Erbium
Erbium2y ago
at new LauncherError (/app/node_modules/chrome-launcher/src/utils.ts:31:18)
at new ChromePathNotSetError (/app/node_modules/chrome-launcher/dist/utils.js:33:9)
at Object.linux (/app/node_modules/chrome-launcher/src/chrome-finder.ts:153:11)
at Function.getFirstInstallation (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:192:61)
at Launcher.launch (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:266:37)
at launch (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:73:18)
at launchChrome (file:///app/node_modules/devtools/build/launcher.js:89:26)
at launch (file:///app/node_modules/devtools/build/launcher.js:199:16)
at Function.newSession (file:///app/node_modules/devtools/build/index.js:34:31)
at remote (file:///app/node_modules/webdriverio/build/index.js:48:43) {
message: 'The CHROME_PATH environment variable must be set to a Chrome/Chromium executable no older than Chrome stable.',
code: 'ERR_LAUNCHER_PATH_NOT_SET' }
at new LauncherError (/app/node_modules/chrome-launcher/src/utils.ts:31:18)
at new ChromePathNotSetError (/app/node_modules/chrome-launcher/dist/utils.js:33:9)
at Object.linux (/app/node_modules/chrome-launcher/src/chrome-finder.ts:153:11)
at Function.getFirstInstallation (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:192:61)
at Launcher.launch (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:266:37)
at launch (/app/node_modules/chrome-launcher/src/chrome-launcher.ts:73:18)
at launchChrome (file:///app/node_modules/devtools/build/launcher.js:89:26)
at launch (file:///app/node_modules/devtools/build/launcher.js:199:16)
at Function.newSession (file:///app/node_modules/devtools/build/index.js:34:31)
at remote (file:///app/node_modules/webdriverio/build/index.js:48:43) {
message: 'The CHROME_PATH environment variable must be set to a Chrome/Chromium executable no older than Chrome stable.',
code: 'ERR_LAUNCHER_PATH_NOT_SET' }
Brody
Brody2y ago
before we go any further, can you tell us what website you are trying to scrape?
Erbium
Erbium2y ago
bseindia.com
Brody
Brody2y ago
do they allow web scraping?
Erbium
Erbium2y ago
it works when done manually
Brody
Brody2y ago
not what I asked unfortunately
Erbium
Erbium2y ago
I ran the code like 6 times on pc before trying to deploy I worded it wrong basically what I mean is "it works on my machine"
Brody
Brody2y ago
again, not what i asked
Erbium
Erbium2y ago
then?
Brody
Brody2y ago
do they allow web scraping?
Erbium
Erbium2y ago
yeah
Brody
Brody2y ago
okay cool, can you show me where they give the user permission to scrape their site
Erbium
Erbium2y ago
What do you mean by scraping though in this instance?
Brody
Brody2y ago
pulling data from the sites html
Erbium
Erbium2y ago
The website does not allow it but I am not doing that I am downloading files which the website provides an interface for so they wouldnt disallow something they literally have made
Brody
Brody2y ago
oh so they have an offical api?
Erbium
Erbium2y ago
yeah
Brody
Brody2y ago
and you are using it?
Erbium
Erbium2y ago
yes
Brody
Brody2y ago
okay then you dont even need chrome
Erbium
Erbium2y ago
I do because I could only figure out some parts of the api
Brody
Brody2y ago
so then you are scraping data
Erbium
Erbium2y ago
I do not think you understood
Erbium
Erbium2y ago
The code fills in the data clicks submit and then clicks the download button at the top right corner
Erbium
Erbium2y ago
how is this scraping
Brody
Brody2y ago
that counts as web scraping, or even botting
Erbium
Erbium2y ago
then why are they providing this?
Brody
Brody2y ago
providing what
Erbium
Erbium2y ago
this interface
Brody
Brody2y ago
what are you talking about either way, this is clearly breaking railway's TOS please remove this app from railway
Erbium
Erbium2y ago
It didnt even deploy
Erbium
Erbium2y ago
Alright so I removed the webscraping part But it still has some dataprocessing that will remain when I try to deploy it says failed to upload code
Brody
Brody2y ago
as long as it uses the official api, without chrome, you are good
Erbium
Erbium2y ago
yeah removed the scraping but this is the problem now
Brody
Brody2y ago
show me your package.json please
Erbium
Erbium2y ago
{
"devDependencies": {
"ts-node": "^10.9.1",
"typescript": "^5.0.4"
},
"scripts": {
"start": "npx ts-node dailyrunner.ts"
}
}
{
"devDependencies": {
"ts-node": "^10.9.1",
"typescript": "^5.0.4"
},
"scripts": {
"start": "npx ts-node dailyrunner.ts"
}
}
Brody
Brody2y ago
are you deploying from github or cli?
Erbium
Erbium2y ago
cli
Brody
Brody2y ago
show me the error
Erbium
Erbium2y ago
[erbium@lallantop dailyrunner]$ railway up
Indexed Compressed [====================] 100% Failed Failed to upload code
[erbium@lallantop dailyrunner]$ railway up
Indexed Compressed [====================] 100% Failed Failed to upload code
Brody
Brody2y ago
what railway version? railway --version
Erbium
Erbium2y ago
railwayapp 3.3.1
Brody
Brody2y ago
what is the total size of your project folder (minus node_modules)
Erbium
Erbium2y ago
821 MB
Brody
Brody2y ago
without node_modules!
Erbium
Erbium2y ago
yeah without node_modules
Brody
Brody2y ago
what file(s) is making it so big
Erbium
Erbium2y ago
as I said it is a data processing application
Brody
Brody2y ago
fair, the upload limit on the cli is 50mb
Erbium
Erbium2y ago
oh
Brody
Brody2y ago
you will want to store the big files in something like cloudflare r2, and download them at runtime on railway
Erbium
Erbium2y ago
If I want to use a container, I just need to write a dockerfile right no need to build the image?
Brody
Brody2y ago
i dont see what a dockerfile has to do with this
Erbium
Erbium2y ago
its unrelated
Brody
Brody2y ago
okay then i also dont understand your question
Want results from more Discord servers?
Add your server