TypeORM migrations
I've installed the Railway CLI and am trying to run database migrations. This is from my
package.json on the backend service.
"migration:run:prod": "npm run typeorm migration:run",
So I'm trying to cd into the /server directory, and run
but I'm getting the error message:
All my env vars are set, so I'm not sure what else to check. Does anyone have a suggestion? 🙂61 Replies
Project ID:
152d55f3-0598-4f41-a2a7-ca60b41a626cdoes your version of typeorm not support the scram auth in use by postgres?
Project ID:
152d55f3-0598-4f41-a2a7-ca60b41a626c
That's a great question - I'm on version 1.22.21 - I'll check1.22.21? npm says typeorm is at 0.3.20
unless im looking at the wrong package? https://www.npmjs.com/package/typeorm
Ah right - sorry, that was the yarn version. I'm on [email protected]
mar 4th 2022
i forget if scram auth was the default back then
Solution
try updating the typeorm version to a more recent version?
Gotcha - I'll give that a try. Thanks [again]!
OK, it looks like I got past that error, now there's another error about missing the
dataSource that TypeORM is raising. But I should be able to fix that pretty quick.awsome!
Well, I thought that solved it but it didn't. I did upgrade to [email protected] and am still getting the error. I was getting a different error before because with the version upgrade I had to update ormconfig.ts and add an argument to the script line. But once I corrected that, the SCRAM error returned. It is working locally just fine. I've researched the issue but haven't been able to come up with a specific reason for this. These are the scripts as defined in my
package.json:
I run yarn run migration:run locally and railway run yarn run migration:run:prod for my Railway service. Any suggestions would be greatly appreciated.what version of node are you using locally and what version is railway using?
We're both on v20.11.0
I think the issue is that env vars are not loaded when I run migrations. When I run
railway run yarn run env I do not see any of the database vars that are set in the Variables dashboard of my backend service. I'm not sure why that's happening.
I figured it out - since I have a monorepo but two separate client and server services, the server service was linked to the client service; therefore it wasn't getting the server env vars. Once I linked it, it started working.well I'm glad you figured it out because there was no way I was going to come to that conclusion! 😆
Haha right @Brody ! It is worth noting, that
railway run yarn run env does not produce the same results as railway variables . railway variables is definitely the way to go.whats the difference?
When I run
railway run yarn run env gives me my local env vars.
No matter which directory of which linked service I'm in.railway variables just prints variables, so what variables are missing from railway run?May be an issue with the version of
yarn I'm running.
railway run yarn run env does not print any of the variables set in the service.what actual command is being ran by the
env script?Well, scratch what I just said - now it's working. I swear it was not working before.
env should print out the variables in the service. I'm guessing railway variables is an alias for that.How did you get past the dataSource error @Steve Cook ? I've tried everything I can think of....
But the nice thing about
railway variables is that it does not print out all the unnecessary variables - it only prints those in the service, which is very nice. And it's formatted very nicely, as well.
@MUG Knight have you double-checked that your directory is linked to the appropriate service?
The way I finally figured out my linking issue was running railway variables and seeing the wrong vars.I did link it this morning. Will double-check the vars again.
right but how does that help you get those variables loaded into your environment so that your code can use them?
It wasn't linked yesterday when I tried.
I'm loading those variables into my data source in
ormconfig - is that what you mean? I'm referencing them wherever I need them on the server side with process.env.[var]
railway variables doesn't do anything other than confirm I have them set correctly.right so what command are you now running to do the migrations locally?
I was so stuck on thinking the issue stemmed from my upgrading TypeORM that missed the obvious.
@Brody to run migration locally:
railway run yarn run migration:run:prod which is how I have the script configured in packages.json.
I did have to edit that a bit after upgrading, as it now requires the the dataSource flag, and I had to export a dataSource from ormconfig rather than just have the config and use createConnection (which is now deprecated).so the entire problem is that you where just linked to the wrong service?
Not entirely - the initial problem was my migration script was incorrect to work with [email protected]. I was having trouble determining that the cause was linking because I couldn't get migrations to run local with the production ./dist .js files. Once I was able to get it to work loading env vars with dotenv locally, it didn't take long to track down the linking issue.
gotcha, thanks!
Of course! Thank you!!
Now I just need to figure out why I'm getting a 404 when I try to load the app using the URL generated for the client service. But that's a different issue. I'm not setting the listening host because express listens on 0.0.0.0 by default. After I deploy it logs listening on http://localhost[$PORT] so I'm not sure what's mis-configured there. I'll keep digging.
seeing a 404 for the root path is very normal for many APIs
Yeah that's the client service though that I'm loading.
What error are you getting? The
dataSource error I was getting was because my migration command wasn't quite right.it keeps telling me my dataSource variable is not provided. It is defined in the package.json and I even try to pass it in manually via the CLI
Give this a shot in your packages.json scripts:
"migration:run:prod": "npx typeorm migration:run --dataSource ./dist/ormconfig.js",. I found production to be a bit more tricky than just my local migrations.
Also you need to make sure you're exporting the dataSource in your ormconfigWhen you linked, did you use the dist directory, or just the root directory of the product?
Root directory of the service, e.g.,
/server
and /clientMy migration:run looks just like that, but uses database.config.ts, which does all of the same stuff.
I have it set to use
./dist for production, I'm guessing you do, as well. So you should be linking to your .js version in /dist (your database.config.js, that is), or wherever you're exporting the data source.cool. I'll keep hacking away at it and see what happens
Are you exporting the
dataSource in database.config.ts?yep
ok cool
welp, it just started working... 🤷♂️
not sure what I did different...
Hey, @Brody - I removed
serve from my client service and copied Caddyfile and nixpacks.toml into my repo. I'm getting ready to try a deployment. I have the Dockerfile - do I not need to do anything to run it in production or will those new files do everything I need (I'm running yarn dev for my local environment)?if you are using nixpacks you would want to remove the dockerfile, they are mutually exclusive
Ah gotcha. Docker has been part of my dev environment forever - might be hard to break that one. But I'll see what I can do. Can't get it to work in prod anyway (even though the log says
listening on localhost:8002) so I guess it can't hurt to try something different.are you serving the frontend and backend separately? like two apps in a monorepo?
yes - it is a monorepo but I have them split into separate services - may need to revisit that.
and you do use a dockerfile for the frontend?
backend is working fine, just can't get the two to work together
and yes, both services have their own Dockerfile
well, can't get the client to serve - just get a 404
why not have a Dockerfile.dev and a Dockerfile.prod, forego the whole nixpacks stuff
I was able to post to the backend with curl, though
I have some conditionals in the Dockerfiles to run different commands based on the environment
fair enough, so then write the dockerfile to use caddy, still foregoing the nixpacks stuff
ah gotcha - that sounds good. great idea.
I'll read up on Caddy - I was cheating and using your files haha 😆
you would still use my caddyfile though
yep, I'll just update the Dockerfile
and nix the nixpack toml
the root path in it would likely need to be changed, i dont know your project structure
Thanks for all the help 🤘
I was talking with my buddy over coffee this morning - he's the one who recommended Railway - and he knew who you are. You're omnipresent 😂
thank you, im honored