Puppeteer Hangs
Hi! I have Pupeteer up and running with node on railway, but for some reason, my job just hangs when I open too many (10-20) tabs. I automatically kill any worker that is idle for 10 minutes and almost all of them end up idle and killed and restarted! Looks like Puppeteer just doesn't load pages on Railway. 10-20 tabs really is not that much and there are no resource spikes in the Railway console. Why else would this job get killed?
This job runs fine locally.
20 Replies
Project ID:
98f66160-ce86-4b15-a73b-cab912256208Railway
404 - Page not found
Railway is an infrastructure platform where you can provision infrastructure, develop with that infrastructure locally, and then deploy to the cloud.
have you tried using the browserless template instead of running chrome in your container?
I do not want to pay for a browserless subscription and was hoping to host all on Railway. Browserless is quite pricy I believe
you can self host it on railway with the ready made browserless template
Solution
Yes, but you need a Browserless subscription to use this
no you don't
I do not want to purchase Browserless as I need to do mass amounts of scraping. The job runs locally using Pupeteer, so why would I need to purchase Browserless?
oh
Perhaps I'm confused what this does
but you should know scraping is in railways tos, so unless you have permission to scrape what you are scraping, it wouldn't be allowed on Railway
Yes I do
then you're all set
So how can I use this? I can look around, but i seems like you put the browserless url from this instance into your pupeteer initializer?
please see this example - https://github.com/brody192/puppeteer-example
So does Railway pay to host Browserless then?
and this is something we get to use
Thank you, can try it out
you don't need to pay anything to self host browserless, I'm not sure where you are getting that from
they have a license required on their site to self-host, so i assumed you had to pay
hmm
are you using it for commercial use?
not quite? we use it scrape information that powers our product, but its not like we sell the scraping as a service
sounds good to me