Twenty•2y ago

v0.20.x+ has broken docker builds

Hi team!

Seeing the following on container start:

Running database setup and migrations...
Need to install the following packages:
  ts-node@10.9.2

Running database setup and migrations...
Need to install the following packages:
  ts-node@10.9.2

This is when attempting to version bump from v0.12.2 ->v0.20.2

Skyfox675OP•7/11/24, 12:12 AM

cc @xSwampxFox

Skyfox675OP•7/11/24, 1:59 PM

Morning @charles, do you mind taking a look at this when you have a moment?

charles•7/12/24, 8:03 AM

Hi @Skyfox675, how are you setting up v0.20?

Ccharles Hi @Skyfox675, how are you setting up v0.20?

Skyfox675OP•7/12/24, 8:22 AM

Thanks for the reply! Here's my upgrade process I've been following since v0.10.0, we run Twenty on a on-prem K3s cluster:

Set twenty-server replicas to 0, wait for container to stop
Set twenty-db replicas to 0, wait for container to stop
Bump twenty-db image to new version
Bump twenty-server image to new version
Set twenty-db replicas to 1, wait for successful start
Set twenty-server replicas to 1, wait for successful start

Last bump I've tried is v0.12.2 -> v0.20.0 and I get the above error. Previously migrations have been automatic when the server container starts

All env vars, db connection string, and deployment settings are the same, only the docker image tag is being modified

charles•7/12/24, 9:19 AM

OK! It's seems related to: https://github.com/twentyhq/twenty/issues/5976

GitHub

Migrations get stuck due to a confirmation prompt · Issue #5976 · t...

Here, the script runs npx ts-node. On newer npm CLIs, this requires a --yes flag, otherwise the following prompt is generated: Running database setup and migrations... Need to install the following...

charles•7/12/24, 9:22 AM

However, you don't need to run the migration automatically at update (I would recommend to do it manually in a second step)

charles•7/12/24, 9:22 AM

To avoid this behavior, ssh into your pod and create file /app/docker-data/db_status (touch /app/docker-data/db_status)

charles•7/12/24, 9:22 AM

this will skip the auto-migration process

charles•7/12/24, 9:23 AM

It's there packages/twenty-docker/twenty/entrypoint.sh

charles•7/12/24, 9:23 AM

I'm still going to take a look at the issue as this might be tied to a newer npx version

Ccharles To avoid this behavior, ssh into your pod and create file /app/docker-data/db_s...

Skyfox675OP•7/12/24, 12:46 PM

Is this file removed after each successful migration or is the presence of the file the trigger to disable all auto migrations moving forward?

Skyfox675OP•7/12/24, 12:47 PM

Going to try this shortly! Thank you!

Skyfox675OP•7/12/24, 1:33 PM

@charles Just noticed the file path is this the path to persist data for the server?

I've been using maybe I missed this change

charles•7/12/24, 5:59 PM

True, I forgot about it, we have updated the docker-compose here: https://github.com/twentyhq/twenty/blob/main/packages/twenty-docker/docker-compose.yml

GitHub

twenty/packages/twenty-docker/docker-compose.yml at main · twentyhq...

Building a modern alternative to Salesforce, powered by the community. - twentyhq/twenty

charles•7/12/24, 5:59 PM

so you can update yours to also have a volume for this docker-data folder

Skyfox675OP•7/12/24, 8:18 PM

That worked!! Thank you!

Skyfox675OP•7/12/24, 8:19 PM

@xSwampxFox let's update the Terraform and K8s manifests to reflect this too so it doesn't catch anyone by suprise

Ccharles True, I forgot about it, we have updated the docker-compose here: https://github...

lucas lima•7/13/24, 2:07 AM

Hey @charles how could I deploy it using traefik into my docker swarm?

what labels do I have to set?
I have to set it just into server? the traefik labels?

about the envs:
Can I set
server url: api.twenty.mydomain.com
Front_url: twenty.mydomain.com?

lucas lima•7/13/24, 2:13 AM

@Skyfox675 hey buddy can you help me?

Skyfox675OP•7/13/24, 2:24 AM

I'm not very familiar with the network and intra-cluster service routing on Docker Swarm, but Terraform and K8s examples were just merged for both the Server and Postgres DB containers, as well as their services, persistant volumes and persistant volume claims!
https://github.com/twentyhq/twenty/tree/main/packages/twenty-docker/k8s

Keep in mind that this uses Cert-Manager to handle Let's Encrypt dynamic TLS certificate requests, and the Ingress-NGINX Controller for K8s.

One key point that I needed to get this working was to ensure that the "X-Forwarded-For" header existed and this required a custom configuration snippet in the Ingress-NGINX Controller.

GitHub

twenty/packages/twenty-docker/k8s at main · twentyhq/twenty

Building a modern alternative to Salesforce, powered by the community. - twentyhq/twenty

lucas lima•7/13/24, 2:27 AM

so i use traefik into my swarm

Skyfox675OP•7/13/24, 2:29 AM

For K8s the flow of traffic in my case is:

Ingress from firewall/internet (port 80/443)
Sent to 80/443 to the LAN/Host IP of Ingress-NGINX Controller
Server Name Identification (SNI) parses the inbound requested Host Header
Traffic hits the virtualHost and follows the above Ingress Spec
Ingress routes traffic to "twentycrm-server" service via port name "http-tcp"
Service twentycrm routes traffic to any available twentycrm-server Pods via 3000

I use container lables for routing of "app=twentycrm-server"

lucas lima•7/13/24, 2:30 AM

Oh thanks, in my case is a bit different

Skyfox675OP•7/13/24, 2:31 AM

I'm not sure how much of this will apply to Docker Swarm or Traefik but you can take what you need assuming that Traefik can do ProxyPass and Host header in-flight modification.

I rip Traefik out of all of my clusters and strictly use NGINX based controllers because it's the only one I've found to meet all of my use cases with minimal headaches

lucas lima•7/13/24, 2:32 AM

I see, thank you again! I use traefik successufuly! but this twenty stack is giviming headache! I think i will go for odoo that stacks easy works.

lucas lima•7/13/24, 2:33 AM

many stacks i easily deploy into my swarm

lucas lima•7/13/24, 2:33 AM

but this one is an excepcion

lucas lima•7/13/24, 2:33 AM

And no one cares about help me here

lucas lima•7/13/24, 2:33 AM

i tried to talk with a lot of people here no one wanna help

lucas lima•7/13/24, 2:34 AM

@Skyfox675

SERVER_URL=http://localhost:3000
FRONT_BASE_URL=http://twenty.mysite.com:-http://localhost:3000

lucas lima•7/13/24, 2:34 AM

its right?

Skyfox675OP•7/13/24, 2:35 AM

Did you look at the link I posed above?

Skyfox675OP•7/13/24, 2:36 AM

TwentyCRM's env var namees and value formating is the same regardless of the container orchestration engine you choose to use

lucas lima•7/13/24, 2:36 AM

yes but, my way to deploy is using the docker-compose into portainer in a easy way

Skyfox675OP•7/13/24, 2:37 AM

That's fine, you're asking are your variable names and values correct, I posted examples in the link above:
https://github.com/twentyhq/twenty/blob/12c68fd77fd7bca26f662e4193c2b7f35700d92b/packages/twenty-docker/k8s/manifests/deployment-server.yaml#L29-L61

GitHub

twenty/packages/twenty-docker/k8s/manifests/deployment-server.yaml ...

Building a modern alternative to Salesforce, powered by the community. - twentyhq/twenty

lucas lima•7/13/24, 2:39 AM

I'm using this here

https://github.com/twentyhq/twenty/blob/main/packages/twenty-docker/docker-compose.yml

GitHub

twenty/packages/twenty-docker/docker-compose.yml at main · twentyhq...

Building a modern alternative to Salesforce, powered by the community. - twentyhq/twenty

Llucas lima @Skyfox675 SERVER_URL=http://localhost:3000 FRONT_BASE_URL=http://...

Skyfox675OP•7/13/24, 2:39 AM

You asked this...

Your answer is here: https://github.com/twentyhq/twenty/blob/12c68fd77fd7bca26f662e4193c2b7f35700d92b/packages/twenty-docker/k8s/manifests/deployment-server.yaml#L29-L61

GitHub

twenty/packages/twenty-docker/k8s/manifests/deployment-server.yaml ...

Building a modern alternative to Salesforce, powered by the community. - twentyhq/twenty

lucas lima•7/13/24, 2:40 AM

yes but I can not see front_base_url there

Skyfox675OP•7/13/24, 2:40 AM

The vars I linked are the only ones required to run the application

lucas lima•7/13/24, 2:40 AM

I see

lucas lima•7/13/24, 2:40 AM

so i wiil cut ir off

lucas lima•7/13/24, 2:41 AM

and try it out again

Skyfox675OP•7/13/24, 2:42 AM

Maybe this was changed recently...

Skyfox675OP•7/13/24, 2:56 AM

You can try removing SERVER_URL and adding FRONT_BASE_URL and BACKEND_SERVER_URL and setting the value of each to

charles•7/13/24, 6:38 PM

Thanks a lot @Skyfox675 for the help! Is there any more help needed @lucas lima, we haven't change the env var recently so they should still be up-to-date

lucas lima•7/13/24, 6:40 PM

@charles I still wondering how to deploy a twenty stack at portainer. I'm working on it more than 1 week

lucas lima•7/13/24, 6:40 PM

In general I can get docker-compose.yaml from github and deploy it easily

charles•7/13/24, 6:40 PM

I have no experience with portainer but we are hosting Twenty on K8 in production

charles•7/13/24, 6:40 PM

where are you stuck?

lucas lima•7/13/24, 6:41 PM

hetzner