Railway•6mo ago

Build Possibly OOM on Builder V2

For customer NextBanana, the build suddenly fails: https://railway.app/project/a0dd0432-b630-4b33-9235-b63593fd7389/service/f5d52ff3-914f-4b41-b5af-281306fd040b/settings

Railway

404 - Page not found

Railway is an infrastructure platform where you can provision infrastructure, develop with that infrastructure locally, and then deploy to the cloud.

47 Replies

Percy•6mo ago

Project ID: a0dd0432-b630-4b33-9235-b63593fd7389,f5d52ff3-914f-4b41-b5af-281306fd040b

angeloOP•6mo ago

cc @Kosai106 👋

Kosai106•6mo ago

the build suddenly fails

To be fair, the build never worked lol

angeloOP•6mo ago

;-;

Kosai106•6mo ago

Happy to provide any information needed to help debug this 😬 Would love to try out your service to get this project deployed

angeloOP•6mo ago

Anyway flagged to the team, I do have to dip for a call but we are looking into it. (Unless you have time in an hour when I can go deep into this.)

Kosai106•6mo ago

I'll be around, thanks for the help so far!

angeloOP•6mo ago

Hey there @Kosai106 - so good-ish news, it's not a total platform issue, the bad news is that it may still be our fault here since we aren't properly detecting Turbo repo.

Kosai106•6mo ago

Oh! That's good-ish news

angeloOP•6mo ago

The errors that I am seeing now are much more dephicerable, its says:

WARN config production Use `--omit=dev` instead.

#13 0.566

#13 0.566 > build

#13 0.566 > turbo run build

#13 0.566

#13 0.570 sh: 1: turbo: not found

WARN config production Use `--omit=dev` instead.

#13 0.566

#13 0.566 > build

#13 0.566 > turbo run build

#13 0.566

#13 0.570 sh: 1: turbo: not found

Kosai106•6mo ago

Ah, ,we're back to that error again. That's nice haha

angeloOP•6mo ago

I switched your workloads back over to the legacy system just so we can grok whats going on, I do suspect that you were running into platform related messes by way of what our build engineer was mentioning to us- however, Turbo availability is a tale as the end of time.

Kosai106•6mo ago

Okay, do what you must :salute: I just noticed that the build is installing dependencies with yarn install --frozen-lockfile but I'm running the scripts with npm run xxx which could explain why the error I'm now getting the error turbo: not found But I'm not sure where the yarn command is defined 🤔 Only instances I'm seeing in the code is my GitHub actions but I'm not sure those are being used here

Brody•6mo ago

nixpacks will use yarn if it finds a yarn lock file, this would take priority over a package lock file

Kosai106•6mo ago

Ahhh, okay that's why. I use yarn myself. Good to know!

Brody•6mo ago

but as for turbo not being found, is it in your deps / dev deps? maybe you have it only installed locally globally

Kosai106•6mo ago

🤦‍♂️ I do have it installed, but it's as a dev dependency... That would explain it

Brody•6mo ago

afaik dev deps should be installed but swap it over just to be sure

Kosai106•6mo ago

Yeah trying that now Yeah that worked lol, I mean the build is still failing but it did find turbo now

Brody•6mo ago

progress! whats the new error?

Kosai106•6mo ago

It's failing to build one of the websites and not the one I'm specifing the build for using --filter='@nextbanana/web' it's trying to also build @nextbanana/landing for some reason

Brody•6mo ago

how are you telling railway what build command to run? by default nixpacks is going to run the build script maybe also try specifying NIXPACKS_TURBO_APP_NAME https://nixpacks.com/docs/providers/node#environment-variables

Kosai106•6mo ago

I specified the command in my railway.json

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "buildCommand": "npm run build --filter='@nextbanana/web'"
  },
  "deploy": {
    "numReplicas": 1,
    "startCommand": "npm run start --filter='@nextbanana/web'",
    "sleepApplication": false,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "buildCommand": "npm run build --filter='@nextbanana/web'"
  },
  "deploy": {
    "numReplicas": 1,
    "startCommand": "npm run start --filter='@nextbanana/web'",
    "sleepApplication": false,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

Brody•6mo ago

does the nixpacks build table state that it will be running those commands?

Kosai106•6mo ago

Brody•6mo ago

are you using node 18 to run this app locally?

Kosai106•6mo ago

Yes, I'm using Node 18.18.2 as specified in my .nvmrc Okay, instead of specifying the filter in the railway.json, I added separate npm scripts in my root package.json and used those commands instead. This seems to work, though I'm not sure why there would be a difference.

Brody•6mo ago

that is very odd indeed, im not sure why either

Kosai106•6mo ago

Geez, the built image is 4.25GB

Brody•6mo ago

nixpacks is not known for small images

Kosai106•6mo ago

Haha good to know. Honestly I'm just happy to see this step at last

Kosai106•6mo ago

:lfg:

Brody•6mo ago

but does it work

Kosai106•6mo ago

Yes and no. The app launches, though it isn't picking up one of the environment variables so I get an error immediately lol

Brody•6mo ago

you have it set as a service variable?

Kosai106•6mo ago

Yes

Kosai106•6mo ago

It's complaining about the SESSION_SECRET_KEY var

Kosai106•6mo ago

@nextbanana/web:start: Error: You must provide the SESSION_SECRET_KEY environment variable in production.

Brody•6mo ago

is this a next app?

Kosai106•6mo ago

It's a Blitz.js app (Blitz is a layer on top of Next) https://blitzjs.com/

Blitz.js - The Missing Fullstack Toolkit for Next.js

Blitz picks up where Next.js leaves off, providing battle-tested libraries and conventions for shipping and scaling world wide applications.

Kosai106•6mo ago

The build logs don't indicate an issue

Brody•6mo ago

its 32 characters right?

Kosai106•6mo ago

Yes, I just double checked Anyway, it's 2am here now, we already got over the main issue which is great! I'll figure out the rest sometime tomorrow after work. Thank you so much for your time @Brody as well as you @Angelo ! Appreciate all the effort 🙏

Brody•6mo ago

happy to help!

angeloOP•6mo ago

We need a way to surface these common issues better- anyway, appreciate you bearing with us

Kosai106•6mo ago

@Brody I had to add the SESSION_SECRET_KEY as an env string in my turbo config. App is deployed and working now And now I'll finally be able to sleep

Brody•6mo ago

ah I knew it was something like that, seen another user have to do something similar, but wasn't confident enough with the reason so I didn't say anything but glad you got it working, when you have some down time to test stuff you can drop nixpacks and write a Dockerfile so you aren't ending up having to wait for a 2.25GB image to get pushed and pulled on every deployment

Gaming

Programming

Build Possibly OOM on Builder V2