T
Twentyā€¢4w ago
bn

Calendar sync succeeds, while Gmail sync does not start

Hi v0.33.0 is running on docker. Following commands were executed on Worker container.
yarn command:prod cron:messaging:messages-import
yarn command:prod cron:messaging:message-list-fetch
yarn command:prod cron:calendar:calendar-event-list-fetch
yarn command:prod cron:messaging:messages-import
yarn command:prod cron:messaging:message-list-fetch
yarn command:prod cron:calendar:calendar-event-list-fetch
Gmail and Calendar access is granted through UI. Calendar sync works fine but Gmail sync does not import any message. Setting page continues to show importing Google Cloud Console shows sustained requests (1 req / min) at Calendar API, while Gmail API does not receive any request except when granted. Worker log indicates Gmail sync cron job ends fairly quickly. I guess requests are not sent.
worker-1 | MessagingMessagesImportCronJob time: 53.8ms
worker-1 | CalendarEventListFetchCronJob time: 5.988ms
worker-1 | CalendarEventListFetchJob time: 775.207ms
d
worker-1 | MessagingMessagesImportCronJob time: 53.8ms
worker-1 | CalendarEventListFetchCronJob time: 5.988ms
worker-1 | CalendarEventListFetchJob time: 775.207ms
d
How could this be fixed?
9 Replies
thomast
thomastā€¢4w ago
@RaphaĆ«l šŸ™
Raphaƫl
RaphaĆ«lā€¢4w ago
Hello @bn, could you show me your env variables and tell me what are the statuses for the messageChannel in your db? Do you see any errors in your worker's terminal? Could you look at module:messaging:messages-to-import in your redis and tell me if there are message ids there?
bn
bnOPā€¢4w ago
@Raphaƫl Thanks for following up. Hope following gives you a clue.
TAG=v0.33.0

PG_DATABASE_HOST=db:5432
REDIS_URL=redis://xxxx

SERVER_URL=https://xxxx
SIGN_IN_PREFILLED=true
STORAGE_TYPE=local
STORAGE_S3_REGION=#eu-west3
STORAGE_S3_NAME=#my-bucket
STORAGE_S3_ENDPOINT=

MESSAGE_QUEUE_TYPE=pg-boss
# === Randomly generated secrets ===
APP_SECRET=xxxx
PGPASSWORD_SUPERUSER=xxxx
ACCESS_TOKEN_SECRET=xxxx
LOGIN_TOKEN_SECRET=
REFRESH_TOKEN_SECRET=
FILE_TOKEN_SECRET=
APP_SECRET=xxxx
POSTGRES_ADMIN_PASSWORD=xxxx
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=xxxx
AUTH_GOOGLE_CLIENT_SECRET=xxxx
AUTH_GOOGLE_CALLBACK_URL=https://xxxx/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=https://xxxx/auth/google-apis/get-access-token
ACCESS_TOKEN_EXPIRES_IN=30d
LOGIN_TOKEN_EXPIRES_IN=10d
REFRESH_TOKEN_EXPIRES_IN=90d
API_TOKEN_EXPIRES_IN=1000y
LOG_LEVELS=error,warn,log
IS_SIGN_UP_DISABLED=false
AUTH_PASSWORD_ENABLED=false
TAG=v0.33.0

PG_DATABASE_HOST=db:5432
REDIS_URL=redis://xxxx

SERVER_URL=https://xxxx
SIGN_IN_PREFILLED=true
STORAGE_TYPE=local
STORAGE_S3_REGION=#eu-west3
STORAGE_S3_NAME=#my-bucket
STORAGE_S3_ENDPOINT=

MESSAGE_QUEUE_TYPE=pg-boss
# === Randomly generated secrets ===
APP_SECRET=xxxx
PGPASSWORD_SUPERUSER=xxxx
ACCESS_TOKEN_SECRET=xxxx
LOGIN_TOKEN_SECRET=
REFRESH_TOKEN_SECRET=
FILE_TOKEN_SECRET=
APP_SECRET=xxxx
POSTGRES_ADMIN_PASSWORD=xxxx
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=xxxx
AUTH_GOOGLE_CLIENT_SECRET=xxxx
AUTH_GOOGLE_CALLBACK_URL=https://xxxx/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=https://xxxx/auth/google-apis/get-access-token
ACCESS_TOKEN_EXPIRES_IN=30d
LOGIN_TOKEN_EXPIRES_IN=10d
REFRESH_TOKEN_EXPIRES_IN=90d
API_TOKEN_EXPIRES_IN=1000y
LOG_LEVELS=error,warn,log
IS_SIGN_UP_DISABLED=false
AUTH_PASSWORD_ENABLED=false
> docker compose logs | grep worker
worker-1 | CalendarEventListFetchCronJob time: 12.17ms
worker-1 | CalendarEventListFetchJob time: 537.308ms
worker-1 | MessagingMessagesImportCronJob time: 19.299ms
worker-1 | CalendarEventListFetchCronJob time: 3.7ms
worker-1 | CalendarEventListFetchJob time: 1.201s
worker-1 | MessagingMessagesImportCronJob time: 68.476ms
worker-1 | CalendarEventListFetchCronJob time: 22.42ms
worker-1 | CalendarEventListFetchJob time: 1.086s
worker-1 | MessagingMessagesImportCronJob time: 16.905ms
worker-1 | CalendarEventListFetchCronJob time: 2.522ms
worker-1 | CalendarEventListFetchJob time: 616.103ms
worker-1 | MessagingMessageListFetchCronJob time: 20.352ms
worker-1 | MessagingMessagesImportCronJob time: 1.848ms
worker-1 | CalendarEventListFetchCronJob time: 2.392ms
worker-1 | CalendarEventListFetchJob time: 611.09ms
> docker compose logs | grep worker
worker-1 | CalendarEventListFetchCronJob time: 12.17ms
worker-1 | CalendarEventListFetchJob time: 537.308ms
worker-1 | MessagingMessagesImportCronJob time: 19.299ms
worker-1 | CalendarEventListFetchCronJob time: 3.7ms
worker-1 | CalendarEventListFetchJob time: 1.201s
worker-1 | MessagingMessagesImportCronJob time: 68.476ms
worker-1 | CalendarEventListFetchCronJob time: 22.42ms
worker-1 | CalendarEventListFetchJob time: 1.086s
worker-1 | MessagingMessagesImportCronJob time: 16.905ms
worker-1 | CalendarEventListFetchCronJob time: 2.522ms
worker-1 | CalendarEventListFetchJob time: 616.103ms
worker-1 | MessagingMessageListFetchCronJob time: 20.352ms
worker-1 | MessagingMessagesImportCronJob time: 1.848ms
worker-1 | CalendarEventListFetchCronJob time: 2.392ms
worker-1 | CalendarEventListFetchJob time: 611.09ms
> redis-cli keys '*' | grep messag
bull:messaging-queue:stalled-check
bull:messaging-queue:meta
> redis-cli keys '*' | grep messag
bull:messaging-queue:stalled-check
bull:messaging-queue:meta
Raphaƫl
RaphaĆ«lā€¢4w ago
It seems that the MessagingMessagesImportCronJob are running correctly according to the logs, can you check your db to see if messages have been imported and tell me what are the statuses for the messageChannel in your db? Thank you Could you change MESSAGE_QUEUE_TYPE=bull-mq?
bn
bnOPā€¢4w ago
@Raphaƫl This is what I got from db. I also updated MESSAGE_QUEUE_TYPE but still the same situation.
SELECT * FROM "workspace_cpm71hy00wbh9oh8qzputiy9h"."messageChannel";
| id | visibility | handle | type | isContactAutoCreationEnabled | contactAutoCreationPolicy | excludeNonProfessionalEmails | excludeGroupEmails | isSyncEnabled | syncCursor | syncedAt | syncStatus | syncStage | syncStageStartedAt | throttleFailureCount | createdAt | updatedAt | deletedAt | connectedAccountId | | 510cf144-efdc-4022-a514-93d01014ab80 | SHARE_EVERYTHING | xxxx | email | t | SENT | t | t | t | 253406 | | ONGOING | MESSAGES_IMPORT_ONGOING | 2024-11-25 23:55:00.079+00 | 0 | 2024-11-25 23:54:18.740525+00 | 2024-11-25 23:55:00.09059+00 | | f3b63ce4-75a3-417a-983f-90fb3a8b92b2|
Raphaƫl
RaphaĆ«lā€¢4w ago
And you don't have anything inside the message table ? Did you maybe kill your worker while it was importing? You can run: yarn command:prod cron:messaging:ongoing-stale yarn command:prod cron:calendar:ongoing-stale This will put back your channels back to pending if for whatever reason they are stuck in ongoing Could you try removing your connected account and reconnecting it?
bn
bnOPā€¢4w ago
@Raphaƫl OMG, it started working. I am not sure what made this. What I did was 1) run your yarn commands on worker container 2) Remove connected Google account on settings UI 3) Reconnect the account on settings UI 4) run following cron job setting commands
yarn command:prod cron:messaging:messages-import
yarn command:prod cron:messaging:message-list-fetch
yarn command:prod cron:messaging:messages-import
yarn command:prod cron:messaging:message-list-fetch
I guess these could be multiple reasons. 1 could be env MESSAGE_QUEUE_TYPE and timing of setup. Anyway, thanks!
Raphaƫl
RaphaĆ«lā€¢4w ago
Great ! Happy to have helped šŸ™‚ I encourage you to also run these commands, it will prevent your channels from being stuck if your worker crashes or something else happens šŸ˜‰ yarn command:prod cron:messaging:ongoing-stale yarn command:prod cron:calendar:ongoing-stale
bn
bnOPā€¢4w ago
Will do. Appreciated
Want results from more Discord servers?
Add your server