Railway•2y ago

Node build failing

Created a project today - I initially had a Dockerfile then removed it to try and use the default build. Project is KoaJS on Node 14. The build errors with:

#11 19.44 gyp ERR! find Python   (accepted by both node-gyp and npm)
#11 19.44 gyp ERR! find Python - Set the environment variable PYTHON
#11 19.44 gyp ERR! find Python - Set the npm configuration variable python:
#11 19.44 gyp ERR! find Python   npm config set python "/path/to/pythonexecutable"
#11 19.44 gyp ERR! find Python For more information consult the documentation at:
#11 19.44 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
#11 19.44 gyp ERR! find Python **********************************************************
#11 19.44 gyp ERR! find Python
#11 19.44 gyp ERR! configure error
#11 19.45 gyp ERR! stack Error: Could not find any Python installation to use
#11 19.45 gyp ERR! stack     at PythonFinder.fail (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
#11 19.45 gyp ERR! stack     at PythonFinder.runChecks (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)
#11 19.45 gyp ERR! stack     at PythonFinder.<anonymous> (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:179:16)
#11 19.45 gyp ERR! stack     at PythonFinder.execFileCallback (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:271:16)
#11 19.45 gyp ERR! stack     at exithandler (child_process.js:390:5)

#11 19.44 gyp ERR! find Python   (accepted by both node-gyp and npm)
#11 19.44 gyp ERR! find Python - Set the environment variable PYTHON
#11 19.44 gyp ERR! find Python - Set the npm configuration variable python:
#11 19.44 gyp ERR! find Python   npm config set python "/path/to/pythonexecutable"
#11 19.44 gyp ERR! find Python For more information consult the documentation at:
#11 19.44 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
#11 19.44 gyp ERR! find Python **********************************************************
#11 19.44 gyp ERR! find Python
#11 19.44 gyp ERR! configure error
#11 19.45 gyp ERR! stack Error: Could not find any Python installation to use
#11 19.45 gyp ERR! stack     at PythonFinder.fail (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
#11 19.45 gyp ERR! stack     at PythonFinder.runChecks (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)
#11 19.45 gyp ERR! stack     at PythonFinder.<anonymous> (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:179:16)
#11 19.45 gyp ERR! stack     at PythonFinder.execFileCallback (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:271:16)
#11 19.45 gyp ERR! stack     at exithandler (child_process.js:390:5)

Unsure if this is a Nixpacks issue or something else?

46 Replies

Percy•2y ago

Project ID: 18c9abfc-def5-4aa2-8518-6b3d9567c97c

GreyOP•2y ago

18c9abfc-def5-4aa2-8518-6b3d9567c97c nixpacks info:

==============
Using Nixpacks
==============
context: 6e6ea8f73c1638910f6b4660b31a5cea
 
╔═════════ Nixpacks v1.5.1 ═════════╗
║ setup      │ nodejs-14_x, npm-6_x ║
║───────────────────────────────────║
║ install    │ npm ci               ║
║───────────────────────────────────║
║ start      │ npm run start        ║
╚═══════════════════════════════════╝

==============
Using Nixpacks
==============
context: 6e6ea8f73c1638910f6b4660b31a5cea
 
╔═════════ Nixpacks v1.5.1 ═════════╗
║ setup      │ nodejs-14_x, npm-6_x ║
║───────────────────────────────────║
║ install    │ npm ci               ║
║───────────────────────────────────║
║ start      │ npm run start        ║
╚═══════════════════════════════════╝

Brody•2y ago

did the dockerfile you had work?

GreyOP•2y ago

no, it was something I set up before and relied on old conventions it built but wasn't meant for this usecase

Brody•2y ago

okay fair, so your node project needs python installed is what I get from the logs?

GreyOP•2y ago

it seems like that's because node-gyp needs it, but node-gyp is only imported as a devDependency for the packages I've installed

Brody•2y ago

ah i see, can you send me your railway.json file from your most recent deployment? this can be found in the details tab

GreyOP•2y ago

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS"
  },
  "deploy": {
    "numReplicas": 1,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS"
  },
  "deploy": {
    "numReplicas": 1,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

Brody•2y ago

but also "When your NODE_ENV environment variable is set to production, using npm ci will not install devDependencies" railway sets that to production, are you perhaps overwriting it somehow

GreyOP•2y ago

yep that's a good point just checked env vars in railway and it's not being overridden there not sure how else it would be getting changed since this is the build stage

Brody•2y ago

strange, try this though

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "nixpacksPlan": {
      "phases": {
        "install": {
          "dependsOn": ["setup"],
          "cmds": ["npm ci --omit=dev"]
        }
      }
    }
  },
  "deploy": {
    "numReplicas": 1,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "nixpacksPlan": {
      "phases": {
        "install": {
          "dependsOn": ["setup"],
          "cmds": ["npm ci --omit=dev"]
        }
      }
    }
  },
  "deploy": {
    "numReplicas": 1,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

i think ci will accept the omit flag, but im not too sure, im not a js dev but that at least should give you an idea on how to overwrite the install command and you can play around with that now

GreyOP•2y ago

ahh found the issue maybe https://github.com/nodejs/node-gyp/issues/2110#issuecomment-622326221

fsevents is macOS only. It specifies an os field in its package.json but this is apparently ignored by npm ci. fsevents used to provide prebuilt binaries up to [email protected] which meant for those versions it didn't attempt to compile

bumping fsevents to >= v1.2.13 should fix the issue

Brody•2y ago

let me know how that goes, ive seen this issue asked here before, where ci would install a dev dep that needed python

GreyOP•2y ago

Updating that skipped the fsevents package, but another devDependency was still being installed that required node-gyp under the hood. Spent a bunch of time trying things locally and saw that npm ci would try to install these packages unless either NODE_ENV=production was set, or the --production flag was passed in updated the command in railway.json to npm ci --production and it built fine so it must be that NODE_ENV=production isn't set at build time?

Brody•2y ago

--omit=dev didn't work? yeah must not be for some reason

GreyOP•2y ago

nah, wasn't recognized maybe it's bc some people want to do dev steps when they build

Brody•2y ago

interesting, well I will keep that in mind and recommend the --production flag thank you for this information

GreyOP•2y ago

np! thanks for the assist

Brody•2y ago

no prob 🙂

GreyOP•2y ago

I'm back 😄 Same problem, different node version updated to node 18 and am starting to see the same issue again #11 0.820 npm WARN config production Use --omit=dev instead. ^ I started to see this appear before the error about gdal/fsevents appearing the command I use is now npm ci --omit=dev --omit=optional in railway.json I've tried a bunch of different things but am still getting config production Use --omit=dev instead. errors separately, I moved the offending packages to optional and it still tries to install them (although fsevents does get skipped this way) "overrides": { "gdal": "^0.11.0" }, "optionalDependencies": { "fsevents": "^1.2.13", "gdal": "^0.11.0" }

Brody•2y ago

can I see your railway.json file?

GreyOP•2y ago

sure thing @Brody !

railway.json

Brody•2y ago

and have you tried npm ci --production ?

GreyOP•2y ago

Yep I tried that too, same warning build failed too, can't remember which error but I think fsevents was causing it

Brody•2y ago

fsevents sure is causing a lot of troubles, how about just removing it all together? last release was Feb 2021, surely there's an equivalent more up to date package without these issues?

GreyOP•2y ago

Yeah I can start to go that route. It's required by something else further down the pipe, but should be able to figure it out I was able to resolve the fs events issue, but the gval package then became the issue. It really still feels like it's trying to install Dev dependencies, that's the part that I can't get around and feels like I need to solve first @Brody confirmed that it's still trying to install stuff like chokidar, which is a devdependency tried using npm install --production and npm install --omit=dev too

GreyOP•2y ago

https://stackoverflow.com/questions/72776325/npm-warning-config-production-use-omit-dev-instead this mentioned to change the .npmrc file to get rid of that warning

Stack Overflow

npm Warning: config production; Use --omit=dev Instead

Every time I run any kind of npm command, I get the following error before the actual output: npm WARN config production Use --omit=dev instead. Of course I've come across similar issues regardi...

GreyOP•2y ago

do you know how I could debug this further?

Brody•2y ago

check the NODE_ENV variable right before install with echo $NODE_ENV && npm install --omit=dev

GreyOP•2y ago

ooh I think I got something

GreyOP•2y ago

https://nixpacks.com/docs/providers/node

Node | Nixpacks

App source + Nix packages + Docker = Image

GreyOP•2y ago

The Node provider sets the following environment variables: CI=true NODE_ENV=production NPM_CONFIG_PRODUCTION=false: Ensure that dev deps are always installed

Brody•2y ago

now why tf would that be forced

GreyOP•2y ago

😄

Brody•2y ago

well try setting NPM_CONFIG_PRODUCTION = true

GreyOP•2y ago

tried to set at the service level didn't work. going to echo out a few variables to confirm what is sticking

Brody•2y ago

try setting it before the install command or i should say, set it inline with the install command

GreyOP•2y ago

ok, I think that setting it at the service level actually did stop it from trying dev deps because logs don't show a warning about chokidar anymore

Brody•2y ago

progress!

GreyOP•2y ago

yep! still getting stuff like

#11 8.744 npm ERR! code 1
#11 8.744 npm ERR! path /app/node_modules/gdal
#11 8.746 npm ERR! command failed
#11 8.746 npm ERR! command sh -c -- node-pre-gyp install --fallback-to-build
#11 8.746 npm ERR! Failed to execute '/nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/bin/node /nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/node_modules/gdal/lib/binding/gdal.node --module_name=gdal --module_path=/app/node_modules/gdal/lib/binding --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v108' (1)
#11 8.747 npm ERR! node-pre-gyp info it worked if it ends with ok
 
#11 8.747 npm ERR! node-pre-gyp info using [email protected]
#11 8.747 npm ERR! node-pre-gyp info using [email protected] | linux | x64
#11 8.747 npm ERR! node-pre-gyp info check checked for "/app/node_modules/gdal/lib/binding/gdal.node" (not found)
#11 8.747 npm ERR! node-pre-gyp http GET https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp WARN Pre-built binaries not installable for [email protected] and [email protected] (node-v108 ABI, glibc) (falling back to source compile with node-gyp)
#11 8.747 npm ERR! node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! gyp info it worked if it ends with ok

#11 8.744 npm ERR! code 1
#11 8.744 npm ERR! path /app/node_modules/gdal
#11 8.746 npm ERR! command failed
#11 8.746 npm ERR! command sh -c -- node-pre-gyp install --fallback-to-build
#11 8.746 npm ERR! Failed to execute '/nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/bin/node /nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/node_modules/gdal/lib/binding/gdal.node --module_name=gdal --module_path=/app/node_modules/gdal/lib/binding --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v108' (1)
#11 8.747 npm ERR! node-pre-gyp info it worked if it ends with ok
 
#11 8.747 npm ERR! node-pre-gyp info using [email protected]
#11 8.747 npm ERR! node-pre-gyp info using [email protected] | linux | x64
#11 8.747 npm ERR! node-pre-gyp info check checked for "/app/node_modules/gdal/lib/binding/gdal.node" (not found)
#11 8.747 npm ERR! node-pre-gyp http GET https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp WARN Pre-built binaries not installable for [email protected] and [email protected] (node-v108 ABI, glibc) (falling back to source compile with node-gyp)
#11 8.747 npm ERR! node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! gyp info it worked if it ends with ok

but that may be unrelated to this/something that has to do with a preinstall script or similar

Brody•2y ago

what is node-pre-gyp

GreyOP•2y ago

it's used to compile C++/other add-ons for node packages it looks like the line that triggers the error comes from "install": "node-pre-gyp install --fallback-to-build", in the package.json of gdal, which is a dev dependency of my project but maybe the install script gets triggered no matter what/isn't getting ignored will see if I can find more info on that ok got it this did the trick NPM_CONFIG_PRODUCTION= npm ci --omit=dev --omit=optional needed to unset the NPM_CONFIG_PRODUCTION variable, and pass --omit=dev will try without the optional to see if that changes anything nope it needed the optional flag too craziness

Brody•2y ago

thats very odd, but im glad you got to the solution in the end, and sorry i couldn't have helped more

GreyOP•2y ago

yea, I blame needing the flag on my setup, but the NPM_CONFIG_PRODUCTION thing was definitely the way to go don't know why it wasn't an issue before with Node 14 🤷‍♂️

GreyOP•2y ago

definitely helped to talk through it though! thanks a ton

Brody•2y ago

no problem! 🙂

Gaming

Programming

Node build failing