Railway•3mo ago

overwrite files during deploy

Is it possible to overwrite a file(files) during deployment? i.e. based on env variable I would like to replace some files in /public folder. /images/ /public/ /src/ package.json etc. "build": "node copyfiles.js && next build", copyfiles.js:

const fs = require('fs')
if (process.env.NEXT_PUBLIC_THEME) {
   fs.cpSync(`images/${process.env.NEXT_PUBLIC_THEME}/`, 'public/images', { recursive: true })
}

const fs = require('fs')
if (process.env.NEXT_PUBLIC_THEME) {
   fs.cpSync(`images/${process.env.NEXT_PUBLIC_THEME}/`, 'public/images', { recursive: true })
}

it does not produce any error but files are not copied. locally it works. maybe files are read only?

Solution:

you may be interested in writing a custom Dockerfile for your app, it will give you much more control over the build process

Jump to solution

21 Replies

Percy•3mo ago

Project ID: 2af97fce-ba10-44a9-aedb-a643afb83aec

whitetownOP•3mo ago

2af97fce-ba10-44a9-aedb-a643afb83aec it seems it did not work with shell command either :/ cp -r images/$NEXT_PUBLIC_THEME/ public/images ok, files are copied but after deploy they are still old. I have no idea why... in the build script I added a command to check the content of destination file

cat public/manifest.json

cat public/manifest.json

it prints the new content. but after deploy mysite.com/manifest.json - it is still old one more file I create dynamically (adding .env variables for firebase auth) the file is created and auth works somehow. but when I access it via web mysite.com/process-env.js - there is still original content. is anybody there? Hi.... I gave up.. it seems something happens behind the scene with files cp -f images/favicon.ico public/favicon.ico (public/favicon.ico exists and overwritten)
cp -f images/favicon.ico public/favicon2.ico (public/favicon2.ico did not exist and copied) next build after the deploy on the web favicon.ico file is still old 😦

Brody•3mo ago

hello, nothing is happening behind the scenes, this is something simple like you are copying files to / from the wrong location

whitetownOP•3mo ago

Okay I created another project, just for testing. feel free to change there anything. e7bf8c87-4233-41c5-bb0c-1eeccb2be302 story: 1. public/ folder contains 4 colorful png file 2. theme/ folder contains black png file 3. we use cutom build command:

./theme.sh && yarn build

./theme.sh && yarn build

./theme.sh && next build

./theme.sh && next build

4. locally all squares are black. on railway they remain colourful. 5. theme.sh:

#/bin/sh

echo "copy black files to colorful files"
cp -f theme/black.png public/red.png
cp -f theme/black.png public/green.png
cp -f theme/black.png public/blue.png
cp -f theme/black.png public/yellow.png

echo "all files below have to be equal"
ls -la theme/*.png
ls -la public/*.png

#/bin/sh

echo "copy black files to colorful files"
cp -f theme/black.png public/red.png
cp -f theme/black.png public/green.png
cp -f theme/black.png public/blue.png
cp -f theme/black.png public/yellow.png

echo "all files below have to be equal"
ls -la theme/*.png
ls -la public/*.png

in logs I see that files are copied and are equal

#12 0.103 -rw-r--r-- 1 root root 9639 Sep 12 17:19 theme/black.png
#12 0.105 -rw-r--r-- 1 root root 9639 Sep 12 17:19 public/blue.png
#12 0.105 -rw-r--r-- 1 root root 9639 Sep 12 17:19 public/green.png
#12 0.105 -rw-r--r-- 1 root root 9639 Sep 12 17:19 public/red.png
#12 0.105 -rw-r--r-- 1 root root 9639 Sep 12 17:19 public/yellow.png

#12 0.103 -rw-r--r-- 1 root root 9639 Sep 12 17:19 theme/black.png
#12 0.105 -rw-r--r-- 1 root root 9639 Sep 12 17:19 public/blue.png
#12 0.105 -rw-r--r-- 1 root root 9639 Sep 12 17:19 public/green.png
#12 0.105 -rw-r--r-- 1 root root 9639 Sep 12 17:19 public/red.png
#12 0.105 -rw-r--r-- 1 root root 9639 Sep 12 17:19 public/yellow.png

on the web they are not https://all-in-black.up.railway.app/ 😉 huh. on vercel they are black https://all-in-black.vercel.app/

Brody•3mo ago

railway does not do any web caching, we simply run your app as-is

whitetownOP•3mo ago

I do not see what exactly happening on railway, but in your logs before build #11 [stage-0 7/10] COPY . /app/. //raylway command #12 [stage-0 8/10] RUN --mount=type=cache,id=s/b5ec3316-27dc-4e02-8b03-6218db5405fe-next/cache,target=/app/.next/cache --mount=type=cache,id=s/b5ec3316-27dc-4e02-8b03-6218db5405fe-node_modules/cache,target=/app/node_modules/.cache ./theme.sh && yarn build then copiing and build is going.. after build #14 [stage-0 10/10] COPY . /app ^^^^ what is this command? maybe it restores all orifinal files?

Brody•3mo ago

set NIXPACKS_NO_CACHE=1

whitetownOP•3mo ago

I added it as env variable - deployed. it did not help. I think the issue in in this command. it's something harmful #14 [stage-0 10/10] COPY . /app I have set the Custom Start Command

ls -la public/*.png && next start

ls -la public/*.png && next start

and what I see - these files that were copied at build time and became equal are different Starting Container -rw-r--r-- 1 root root 1459 Sep 12 17:48 public/blue.png -rw-r--r-- 1 root root 534 Sep 12 17:48 public/green.png -rw-r--r-- 1 root root 5344 Sep 12 17:48 public/red.png -rw-r--r-- 1 root root 3595 Sep 12 17:48 public/yellow.png ▲ Next.js 14.2.10 - Local: http://localhost:8080 ✓ Starting... ✓ Ready in 477ms

Solution

Brody•3mo ago

you may be interested in writing a custom Dockerfile for your app, it will give you much more control over the build process

whitetownOP•3mo ago

unfortunately I am not a real backend developer and have never used Docker (only run docker desktop for some reason I do not remember). I like railway that it allows even such dumb as me to deploy everything easily. I could make a workaroud, like a special branch for deploy. but it would be nice if you fix this... well, it's not so urgent. but there is definitely something copied after deploy for some reason

Brody•3mo ago

i'm sorry but this is not a platform issue

whitetownOP•3mo ago

I would agree with you but in this case we both will be wrong 😦 my deploy command is in the middle of dosen events and since it works locally or on vercel - I assume that it works correctly. something in this chain changes files after the build

whitetownOP•3mo ago

Hi again, I have tried to deploy with Dockerfile. on one hand it worked, on another hand it added other issues. everything I run inside dockerfile does not see environment variables. I found that I can expose them as ARG RAILWAY_ENVIRONMENT but if I have a lot of them? is there a way to expose all?

Brody•3mo ago

you only need to use ARG for the variables you need to use during your build, for any variables you only access during runtime you do not need to use ARG

whitetownOP•3mo ago

the dockerfile contains a line yarn build (next build) as you may know next could need a lot of vars during build time and runtime, kind of NEXT_PUBLIC_whatever As i see if a var not added with ARG it is undefined. Actually, while i was playing with docker, i understood one thing (correct me if I’m wrong): Railway creates dockerfile based on configuration settings. Probably it also exposes all vars, then runs it step by step. So, I still think that step 10 in the railway dockerfile is unnecessary. If the project files were modified during the build, step 10 reverts all modified files back #14 [stage-0 10/10] COPY . /app OK. based on my latest research I have found very simple solution. just need to modify files again after railway restored original versions. if anybody is interested.

{
    "$schema": "https://railway.app/railway.schema.json",
    "build": {
        "buildCommand": "./modify-project-files.sh && yarn build"
    },
    { "workaroud": "let's do it again, no dockerfile is needed"},
    "deploy": {
        "startCommand": "./modify-project-files.sh && yarn start"
    }
}

{
    "$schema": "https://railway.app/railway.schema.json",
    "build": {
        "buildCommand": "./modify-project-files.sh && yarn build"
    },
    { "workaroud": "let's do it again, no dockerfile is needed"},
    "deploy": {
        "startCommand": "./modify-project-files.sh && yarn start"
    }
}

Brody•2mo ago

why are you running the script twice, run it only once during runtime

whitetownOP•2mo ago

because before the deploy command railway script replaces changed files with the original files from my repository. 1. in your deploy script (docker) there is a command

#11 [stage-0  7/10] COPY . /app/.

#11 [stage-0  7/10] COPY . /app/.

it copies files from the repository to the /app folder - it's OK, we need it 2. then it runs

"buildCommand"

"buildCommand"

this command modifies some files during build in the /public folder 3. then your script copies files again

#14 [stage-0 10/10] COPY . /app

#14 [stage-0 10/10] COPY . /app

this is the same command as in the step #11 [stage-0 7/10] whatever changes in the /app folder were made - this command overwrites modified files with originals.

Brody•2mo ago

sorry, that doesnt answer my question ill rephrase, why are you running it during build when you should only be running it during runtime? (it = the script)

whitetownOP•2mo ago

if I understand correctly, when yarn build works - it uses some variables relates to environment variables. i.e. I need to create const process = { env: { NEXT_PUBLIC_FIREBASE_apiKey: '${process.env.NEXT_PUBLIC_FIREBASE_apiKey ''}', NEXT_PUBLIC_FIREBASE_authDomain: '${process.env.NEXT_PUBLIC_FIREBASE_authDomain ''}', NEXT_PUBLIC_FIREBASE_databaseURL: '${process.env.NEXT_PUBLIC_FIREBASE_databaseURL ''}', NEXT_PUBLIC_FIREBASE_projectId: '${process.env.NEXT_PUBLIC_FIREBASE_projectId ''}', NEXT_PUBLIC_FIREBASE_storageBucket: '${process.env.NEXT_PUBLIC_FIREBASE_storageBucket ''}', NEXT_PUBLIC_FIREBASE_messagingSenderId: '${process.env.NEXT_PUBLIC_FIREBASE_messagingSenderId ''}', NEXT_PUBLIC_FIREBASE_appId: '${process.env.NEXT_PUBLIC_FIREBASE_appId ''}', NEXT_PUBLIC_FIREBASE_measurementId: '${process.env.NEXT_PUBLIC_FIREBASE_measurementId ''}', }, } I do not want to keep these variables in the repo. honestly I am not sure what will happen if I omit this command at build step. I will try

Brody•2mo ago

i dont think you are understanding the question, oh well, if it works, it works

whitetownOP•2mo ago

I did. probably, copying/modifying only at start will work.

Gaming

Programming

overwrite files during deploy