Self-hosted migration scripts in docker
Hi all!
We have self-hosted Novu in docker (docker-compose).
In the description of version 0.18 states that we have to execute the migration script.
npm run migration:primary-provider
What is the correct way to do it in docker-compose instance of Novu.60 Replies
I have the exact same question
Did you manually change the image reference to 0.18 or you are only preparing for the update
up
I'm running with the same issue. I tried to upgrade the api service in docker-compose to 0.18, but the migration doesn't exist in that version (even in the source code that migration doesn't exist in the package.json file in the api folder.
So I checked in the 0.19 version, and there is it. So I tried to upgrade the docker image to 0.19 and run the migration inside the container but it failed because reference another file in the src folder.
So it looks like the migration only works from the source code, and not from the containers.
Lets see what Novu staff says about this.
Thanks
Hi @aegeavaz and sorry for the delayed response here, I would suggest running the migrations from source and not from the containers. (You can clone it locally or in a machine that has access to the mongo instance, and run the migrations)
Thanks. But what about the versions? because that migration is not in the v.0.18.0 tag version but it is described in the release documentation for v.0.18.0. Should we upgrade to version 0.19.0?
You can just clone the next branch, all the migrations file are there historically
Well, it works. Thank you!
@dimagrossman
@novu/api@0.19.0 migration cross-env NODE_ENV=local MIGRATION=true ts-node --transpileOnly ./migrations/expire-at/expire-at.migration.tsnode:events:491 throw er; // Unhandled 'error' event ^ Error: spawn ts-node ENOENT at ChildProcess._handle.onexit (node:internal/child_process:283:19) at onErrorNT (node:internal/child_process:476:16) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) Emitted 'error' event on ChildProcess instance at: at ChildProcess._handle.onexit (node:internal/child_process:289:12) at onErrorNT (node:internal/child_process:476:16) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { errno: -2, code: 'ENOENT', syscall: 'spawn ts-node', path: 'ts-node', spawnargs: [ '--transpileOnly', './migrations/expire-at/expire-at.migration.ts' ] } Node.js v18.7.0 npm ERR! Lifecycle script
migration failed with error:
npm ERR! Error: command failed
npm ERR! in workspace: @novu/api@0.19.0
npm ERR! at location: /home/technology/novu/apps/api
this is what I get
I am running this against the next branch
any insights to resolve this?@harrisyn based on your logs you try to run the
migration npm script, i'm not sure what version you are upgrading to, but in 0.18.0 the script is: npm run migration:primary-providerHi @dimagrossman
I am running this off the next branch, so you can see the workspace is @novu/api@0.19.0
might have been assumed, however I think node_modules have to be installed first, this environment had only docker.
What I have tried
1) Ran npm i from the apps/api directory
2) ran npm i from the base novu folder
I still end up now with
npm run migration:primary-provider
@novu/api@0.19.0 migration:primary-provider cross-env NODE_ENV=local MIGRATION=true ts-node --transpileOnly ./migrations/integration-scheme-update/add-primary-priority-migration.tsError: Cannot find module '/home/technology/novu/node_modules/@novu/shared/dist/cjs/index.js'. Please verify that the package.json has a valid "main" entry at tryPackage (node:internal/modules/cjs/loader:364:19) at Function.Module._findPath (node:internal/modules/cjs/loader:577:18) at Function.Module._resolveFilename (node:internal/modules/cjs/loader:942:27) at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/home/technology/novu/node_modules/@cspotcode/source-map-support/source-map-support.js:811:30) at Function.Module._load (node:internal/modules/cjs/loader:804:27) at Module.require (node:internal/modules/cjs/loader:1022:19) at require (node:internal/modules/cjs/helpers:102:18) at Object.<anonymous> (/home/technology/novu/apps/api/src/config/index.ts:2:1) at Module._compile (node:internal/modules/cjs/loader:1120:14) at Module.m._compile (/home/technology/novu/node_modules/ts-node/src/index.ts:1618:23) { code: 'MODULE_NOT_FOUND', path: '/home/technology/novu/node_modules/@novu/shared/package.json', requestPath: '@Novu/shared' } npm ERR! Lifecycle script
migration:primary-provider failed with error:
npm ERR! Error: command failed
npm ERR! in workspace: @novu/api@0.19.0
npm ERR! at location: /home/technology/novu/apps/api@harrisyn could you please run: pnpm install && npm run build
From the root repository folder
pnpm part is important
Hi @dimagrossman I am still not successful at this.
What I have tried
1) updated the codebase to the latest in next branch.
2) installed pnpm and node (it didn't work with the latest versions, so installed the recommended versions)
3) ran pnpm install && npm run build
- Successfully ran target prebuild for 60 projects (49s)
✖ nx run @Novu/web:build
@novu/web@0.19.0 build /home/technology/novu/apps/web cross-env NODE_OPTIONS=--max_old_space_size=4096 react-app-rewired --max_old_space_size=4096 buildCreating an optimized production build... The build failed because the process exited too early. This probably means the system ran out of memory or someone called
kill -9 on the process.
ELIFECYCLE Command failed with exit code 1.
————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————
NX Ran target build for 65 projects (38m)✔ 57/59 succeeded [0 read from cache] ✖ 2/59 targets failed, including the following: - nx run @Novu/outlook365:build - nx run @Novu/web:build this process takes waaaaay too long, should I need to increase the VM specifications to run a migration??
I think the issue is with the web projects, which in reality arent required to be built for the migrations
could you try doing: npm run build:api insteald of the global build
And by the way, i'm very sorry for the very complicated flow on this. I would really love to simplify this. Just hadn't a change to get to it yet 😦
oot@Novu:/home/technology/novu# npm run build:api
build:api nx build @Novu/api✖ 1/47 dependent project tasks failed (see below) ✔ 46/47 dependent project tasks succeeded [46 read from cache]
nx run @Novu/outlook365:build
@novu/outlook365@0.19.0 build /home/technology/novu/providers/outlook365 run-p build:*
@novu/outlook365@0.19.0 build:main /home/technology/novu/providers/outlook365 tsc -p tsconfig.json
@novu/outlook365@0.19.0 build:module /home/technology/novu/providers/outlook365 tsc -p tsconfig.module.jsonsrc/lib/outlook365.provider.ts(24,7): error TS2769: No overload matches this call. The last overload gave the following error. Argument of type '{ host: string; port: number; requireTLS: boolean; connectionTImeout: number; auth: { user: string; pass: string; }; tls: { ciphers: string; }; }' is not assignable to parameter of type 'TransportOptions | Transport<any>'. Object literal may only specify known properties, and 'host' does not exist in type 'TransportOptions | Transport<any>'. src/lib/outlook365.provider.ts(24,7): error TS2769: No overload matches this call.
NX Ran target build for project @Novu/api and 48 task(s) they depend on (11s)✖ 1/47 failed ✔ 46/47 succeeded [46 read from cache] @Dima Grossman any alternatives for me? currently stuck here
Distributed Caching and Task Execution for Nx and Lerna
Leverage the power of distribution to bring your Nx and Lerna workspaces to the next level. All nicely integrated into a single unified experience.