Node.js 18 and node-gyp compatibility
Howdy folks. I am loving Railway, the interface and overall experience is fantastic. I am trying to deploy an API project which uses NestJS and running into some issues with some Node packages.
The first is an issue with a package I am using
langchain which throws a warning because it requires Node.js 18 and above. Is there a way I can tell my Railway app to use the latest LTS version of Node? Version 16 is quite old now.
Secondly, the dreaded node-gyp error with another package.
These issues are preventing me from deploying my project. I would hate to have to use another platform if I can't find a solution to these issues. Thank you for your time.Solution:Jump to solution
```json
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",...
87 Replies
Project ID:
dc2d2825-bef5-4c42-8d8b-646d7f1898f9dc2d2825-bef5-4c42-8d8b-646d7f1898f9
in your package.json set
engines.node = 18
as for the gyp issue, send me your railway.json file and i will try to modify it to stop installing gyp@Brody That was an incredibly fast response. Thank you. I'll see if bumping Node.js version to 18 maybe fixes the issue with the package.
i doubt it will fix the issue with gyp
Yeah, you were right. Gyp issue still prevalent. I don't seem to have a
railway.json file because I created the project through the web interface and then linked it using railway link in my project.every deployment gets an auto generated railway.json file, you can copy it from the latest deployment under the details tab
Ah, gotcha. I didn't realise that. This is my
railway.json file:
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"buildCommand": "npm run build"
},
"deploy": {
"numReplicas": 1,
"startCommand": "npm start",
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}fun fact, you dont need to define those build and start commands yourself if you already have build and start scripts in your package.json file
Ah, cool. Always good to not have to configure things.
so now you can create a railway.json file in your project folder, and use this
i have not tested this personally, but this is what has worked for another user
https://discord.com/channels/713503345364697088/1106436043827335169/1110792220627451994
I thought maybe something good was happening, but alas:
send your package.json?
would you happen to know what package is using node-gyp as a dep?
I do actually. It's the
hnswlib-node dependency.oh so you actually do need python then?
I think it uses Python to build it or something. You've given me an idea to see if there is a way to specify a non-source version of it actually.
i can also give you a railway.json file that will install python too?
Oh, that would be awesome if that's possible.
Installing Python would sort it I think.
If this all works out, do you know if there is a way I can contribute back to Railway docs and write up some docs on dealing with Node.js and node-gyp issues?
i honestly cant think of a good place for you to write docs for this, we need a community blog lol
Haha, yes. A community blog or wiki would be great.
Okay, that made some progress. Man,
node-gyp really is a stain on the Node.js ecosystem, huh? I have no idea what any of this even means, haha.
fun
Solution
you know what to do lol
Haha
let me know how that goes
Oh shit. I think this worked.
Brody, you're a genius.
woohoo
Okay. So, what I am going to do is write up a Markdown document. And then, if you's want to take it and put it somewhere in the docs on the site, might save you a tonne of time in the future, because I know
node-gyp is a pain for Node.js users.i can put it into the project help section of https://help.railway.app/
at least i can write my own guides there, i will ask angelo if i could put a guide you authored there too
Sounds great. Thank you for not only your fast response but your patience in getting this sorted out as well.
no problem, happy to help 🙂
Just for good measure: I also ran into the same error ('gyp ERR! find Python). This solved it. Thanks.
Well, I am back. Initially, adding "python38" worked. Later, I got the same (as mentioned in this thread) the missing "gcc" error. "gcc" added, but now deployments fail due to missing "libvips".
I have been trying to add all the suggested libvips (i.e. "dev", "tools") to "nixPkgs", but Nix just seems to not recognise the packages. All help is highly appreciated.
This is my railway.json:
send the json file in text please
Of course. Railway:
... and if needed; package.json and pnpm-lock.yaml.
Package.json:
pnpm-lock.yaml:
try adding
vips to your nix packages arrayI think I have tried, but I gave it a go again - failed, unfortunately:
My railway.json with vips:
try this, but if this doesnt work we'd need to move to a custom dockerfile
Thanks, but unfortunately it didn't work. Still the exact same error - "fatal error: vips/vips8: No such file or directory".
I am looking forward to look into a custom dockerfile, though I must admit that I don't really have any experience with dockerfiles. I'm fine with Linux, Bash and CLI (apart from web dev) - so let's bring it on. Thanks. 🙂
if you can share your repo with me it would help me write the dockerfile
Sure:
GitHub
GitHub - Zvezke/TreKor
Contribute to Zvezke/TreKor development by creating an account on GitHub.
can you also send me your latest build logs https://bookmarklets.up.railway.app/log-downloader/
This should be it.
thank you, dont know when ill be able to write the dockerfile, railway is having a bit of an issue right now
Sure, no stress. Take your time ... we can wait a couple of days if needed. 🙂
haha it wont be a couple of days
Ok, just don't stress. 😉
And hey - thanks a lot. I really appreciate it.
hey @Zvezke - give this a try, no clue if it will work first try, just copy this into a
Dockerfile in your project, push the changes and railway will build with it instead of nixpacks
Thanks! Unfortunately, it errored out immediatly:
Should I try with NPM instead, i.e.:
well I assume you'd want to use pnpm, but you wouldn't need to install npm because it comes with the node image
Yeah, of course - I don't need to install npm.
I'm not sure whether I use pnpm or npm matters with regards to the docker file. Does it matter? I am using pnpm locally - but isn't the docker file only for deployment on Railway?
it's always good to have the deployment use the same tooling as you use locally
Ok, I'll just give it a try with npm - or no?
sure why not, but remove npm from the apk add line
Yeah, I did. 🙂
keep me updated then!
Well, it didn't crash immediately - it started generated the pages. The log doesn't show any fatal errors, but a bunch of error about missing env vars - 'Error: Your project's URL and Key are required to create a Supabase client!'.
The docker file hos only 'NEXT_PUBLIC_RAILWAY_URL' whereas four in all:
... can I just add the three missing the docker file (ARG)?
yes! i didnt know what other variables your build would need so i only used what i found in the .env.development file
Oh, of course. How should I do it syntactically? All in one line?
... or a new line for each:
ARG ...
ARG ...
i'm pretty sure either works, so up to you
whatever you think looks best
😄
yes, just checked, either way works the same
Hey! It's alive! 🙂
Thanks a billion - I'll buy you a beer next time you're in Copenhagen, Denmark - Carlsberg, of course.
so do you want to get pnpm working on the dockerfile?
If possible, yeah. I assume using the same pkg-manager is less prone to bugs.
can you send me your current dockerfile?
give it a try
Thanks. I'll be back i five.
It works - thanks. I really appreciate it.
I'll buy you two beers next time you're in Copenhagen - two Carlsbergs, of course. And I'll introduce you to Scandinavian women as well.
lmfao thanks
... so actually - running 'npm install -g pnpm' separately did it (compared to you first suggestion).
Btw, do you know if this is a bug in Railway?
its not, can you tell me why it didnt work? error message you got?
Initially, I got "gyp ERR! find Python" - adding "python38" to railway.json worked. Then, I got the same (as mentioned in this thread) the missing "gcc" error. "gcc" added, but deployments started failing due to missing "libvips". Adding libvips (in all different variations) didn't work - pkg not recognized.
Then, I asked for help, and here we are ...
not too sure what any of that has to do with the dockerfile we are now using?
Well, adding packages (libvips in different variations) to nixPkgs didn't help - finally, you suggested a docker file.
you have significantly confused me
does this dockerfile, as is, not work?
or in other words, did you have to modify it, for it to work?
The dockerfile works - all is fine again.
Initially, you suggested 'RUN apk ... pnpm ... ', but that didn't work. The file works now. I am just wondering what is causing a normal deployment to fail.
It's just my curiosity - don't worry about it. Thanks, much appreciated - I'll see you in the threads. And I'll get back if things fail again. 🙂
ohhh you asked why apk add couldn’t install pnpm.
apk is alpines package manager, like debian's apt, but better.
pnpm is a package in alpines package manager, but its only in the testing repository and apk doesn’t pull from that by default so it couldn’t install it, i overlooked that.
instead of switching what repository apk pulls the packages from, i opted to just have npm install pnpm.