Railway•3y ago

Nixpacks Conda Package Issues

Hi, I'm moving my app from Heroku to Railway, which is a NodeJS application which utilizes Python as its subprocess for plotting. NodeJS packages are working correctly, and it appears all the Python packages I've specified in NIXPACKS_PKGS seem to be working correctly, which is

gdal geos proj python39 python39Packages.numpy python39Packages.matplotlib python39Packages.requests python39Packages.cartopy python39Packages.pandas

. When I build the application, and the subprocess in NodeJS runs, it throws an error like this

PythonShellError: ModuleNotFoundError: No module named 'matplotlib'
at PythonShell.parseError (/app/node_modules/python-shell/index.js:295:21)
at terminateIfNeeded (/app/node_modules/python-shell/index.js:190:32)
at ChildProcess.<anonymous> (/app/node_modules/python-shell/index.js:182:13)
at ChildProcess.emit (node:events:513:28)
at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
----- Python Traceback -----
File "/app/model/storms/fix/fixplot.py", line 6, in <module>
from matplotlib import pyplot, dates, style {
traceback: 'Traceback (most recent call last):\n' +
'  File "/app/model/storms/fix/fixplot.py", line 6, in <module>\n' +
'    from matplotlib import pyplot, dates, style\n' +
"ModuleNotFoundError: No module named 'matplotlib'\n",
executable: 'python3',
options: null,
script: 'model/storms/fix/fixplot.py',
args: null,
exitCode: 1
}

PythonShellError: ModuleNotFoundError: No module named 'matplotlib'
at PythonShell.parseError (/app/node_modules/python-shell/index.js:295:21)
at terminateIfNeeded (/app/node_modules/python-shell/index.js:190:32)
at ChildProcess.<anonymous> (/app/node_modules/python-shell/index.js:182:13)
at ChildProcess.emit (node:events:513:28)
at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
----- Python Traceback -----
File "/app/model/storms/fix/fixplot.py", line 6, in <module>
from matplotlib import pyplot, dates, style {
traceback: 'Traceback (most recent call last):\n' +
'  File "/app/model/storms/fix/fixplot.py", line 6, in <module>\n' +
'    from matplotlib import pyplot, dates, style\n' +
"ModuleNotFoundError: No module named 'matplotlib'\n",
executable: 'python3',
options: null,
script: 'model/storms/fix/fixplot.py',
args: null,
exitCode: 1
}

18 Replies

RyanKnackOP•3y ago

When I change the NIXPACKS_PKGS to replace Python39 with conda, i.e.

gdal geos proj conda python39Packages.numpy python39Packages.matplotlib python39Packages.requests python39Packages.cartopy python39Packages.pandas

, a new error pops out and states

node:events:505
throw er; // Unhandled 'error' event
^
Error: spawn python3 ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:478:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
Emitted 'error' event on PythonShell instance at:
at ChildProcess.<anonymous> (/app/node_modules/python-shell/index.js:177:18)
at ChildProcess.emit (node:events:527:28)
at Process.ChildProcess._handle.onexit (node:internal/child_process:289:12)
at onErrorNT (node:internal/child_process:478:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn python3',
path: 'python3',
spawnargs: [ 'model/storms/fix/fixplot.py' ]
}

node:events:505
throw er; // Unhandled 'error' event
^
Error: spawn python3 ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:478:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
Emitted 'error' event on PythonShell instance at:
at ChildProcess.<anonymous> (/app/node_modules/python-shell/index.js:177:18)
at ChildProcess.emit (node:events:527:28)
at Process.ChildProcess._handle.onexit (node:internal/child_process:289:12)
at onErrorNT (node:internal/child_process:478:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn python3',
path: 'python3',
spawnargs: [ 'model/storms/fix/fixplot.py' ]
}

Project ID: 393340a8-9e6c-46b1-b8fe-1854ed590d1c

JustJake•3y ago

I can't seem to find anything at that projectId You sure it's correct?

RyanKnackOP•3y ago

Oop, how about this d2281d4c-ae97-47ba-a59c-311473bb0baa Pardon, I will have to return to you at a later time

JustJake•3y ago

Th'ats more correct Have you tried having both? Cause ya looks like it just can't find the executable

RyanKnackOP•3y ago

Yeah, having both, the issue goes back to "no module found 'matplotlib'"

JustJake•3y ago

That sounds like you also need to install requirements.txt or something Lemmse ee Yea. Defs need to override that install or build or start command

RyanKnackOP•3y ago

There is a library I'm using up there that cannot be properly installed with pip, that's Cartopy, hence I opted for the conda option as well Alright I'll see what I can do later

JustJake•3y ago

JustJake•3y ago

All good. You can override those settings using either environment variables or in the Settings

JustJake•3y ago

https://nixpacks.com/docs/config

Configuration | Nixpacks

App source + Nix packages + Docker = Image

JustJake•3y ago

JustJake•3y ago

We'll uhh, merge them at some point sorry for the split brain annoyance lol

RyanKnackOP•3y ago

Lol no worries

AceyGoesBoom•3y ago

why cant you install it with pip?

RyanKnackOP•3y ago

I went through a rabbit hole when I tried installing it with pip (through requirements.txt) when I was on Heroku. It constantly throws out an error stating that numpy was not installed and could not run setup.py, despite numpy being the first things installed Cartopy requires you to install all prerequisites before installing Cartopy itself, which I have all went through many many times but still no luck with pip. Only anaconda did the job. https://scitools.org.uk/cartopy/docs/latest/installing.html#building-from-source Apologies, but I would need some examples on how to install the dependencies using requirements.txt using either the env vars or through the settings page. I tried pip install -r requirements.txt in NIXPACKS_INSTALL_CMD env var and NIXPACKS_BUILD_CMD env var but it says pip is not found lol

JustJake•3y ago

Ah you're gonna need to actually make sure you install pip on NIx NIXPACKS_PKGS=pip Actually hmm pip isn't in nix...

Unknown User•3y ago

Message Not Public

RyanKnackOP•3y ago

Yeah python39 is part of the env vars I've stated in NIXPACKS_PKGS Ive additionally tried the env var NIXPACKS_APT_PKGS and the value python3-matplotlib and after indicating that matplotlib is downloaded (supposedly), and the subprocess runs, it still says matplotlib is not found Update: After trying to fiddle with the install and packages commands, this has now worked for me with the following settings, where pip and matplotlib is now successfully installed, on top of NodeJS: NIXPACKS_APT_PKGS=python3-pip NIXPACKS_INSTALL_CMD=npm i && pip install -r requirements.txt NIXPACKS_PKGS=gdal geos proj python39Packages.cartopy (Not sure if this works yet but still trying) requirements.txt:

numpy==1.21.5
matplotlib==3.5.2
requests==2.27.1
pandas==1.4.2

numpy==1.21.5
matplotlib==3.5.2
requests==2.27.1
pandas==1.4.2

Hi, after trying out the cartopy install for the past 2 months, I am afriad it does not work. I tried using the nixpacks conda pack but am unsure of how i works and how to install libraries with it. Is there an example to this? (I've also tried with Dockerfile but Im having loads of folder installation issues as well when installing miniconda)

Gaming

Programming

Nixpacks Conda Package Issues