Upgrade to v0.23.0 failing
Hi @charles hope you and team are doing well!
I'm running into issues with the v0.23.0 migration following the upgrade guide on the docs page.
-
yarn database:migrate:prod ✅
- yarn command:prod workspace:sync-metadata -f ❌
I'm receiving the following errors after running the second command:
Any help would be appreciated, thanks!102 Replies
cc @Weiko , you probably have more context on 0.23
If it helps, I'm all for wiping out the troublesome "demoUrl" attribute.
There were some other errors related to profile pictures and workspace avatars and attachments missing as well. I'm also open to resetting those to defaults and wiping all attachments to get this up and running again.
Hi @Skyfox675, that's weird, this one seems to be a custom field, the sync command should ignore those. Did you rename an existing standard-field by any chance? If we check in the field-metadata table, does demoUrl have isCustom: false or a standardId?
@Weiko sorry for the late reply, late getting home from the office..
Ok so assuming my psql game isn't too rust 😅 , it looks like
demoUrl has isCustom=f and standardId IS NULLOk so that's the issue. demoUrl is not a standard field. isCustom should be true. If you change its value to true it should be ignored by the sync command
It shouldn't be false though, I'm not sure what happened. Did you create this field via the UI? Or updated it by any chance?
I've guaranteed only made data model changes via the UI, I'm not looking for trouble 😂
I did start on v0.10 when first starting to use Twenty. I recently found a misconfiguration with my deployment that wasn't persisting .local-data correctly. So I may have missing data that may still be referenced in the database. Happy to wipe out all of those entries if needed for missing data.
I did originally start by extending the standard Company object via the UI and adding attributes specific to my business/processes there, until @charles highly recommended not doing this (thank you btw) and as of v0.20.0 I added an
Automation Custom Object that has all of these custom fields. But I don't remember if demoUrl is on the standard Company, or both (Automation and Company)
FYI I do have daily pg_dumps available should we need to restore anythingI see! Well, hard to tell, I don't see any other report / bugs related to custom fields having the wrong isCustom value. I'll keep an eye on it. Did the rest succeed for you?
We changed the model a lot these last few months but we are trying to make upgrade easier after each update.
Let me know if you still have any issue
Would you mind providing the exact psql query I should run to change the field value, I can work it out myself if it's too much, would rather be safer though if it's something easy for you to share
@Skyfox675
Should do the trick
Amazing, thank you, running now 🙏
So now if you try the command, it should not pick it up during the sync
Giving it a shot now!
Looks like it failed on another field
invoicePortalLink this did (or still does) reside on both Company and Automation. I believe though that for Automation since the introduction of Link types in 0.12 I removed the <type> from the attribute name so this might actually be the Company field
Probably the same reason, I'm guessing this field has isCustom:false
I'm really curious now how all of this happened, maybe you were on a previous version that had this bug, I can't reproduce it
The one that has false is the one on the company object maybe? Since company object is a standard object, maybe there was a bug 🤔
I'll just be honest... I'm a serious bug 🧲 like weird shit just happens around me
Like restraunts loosing internet access seconds before I tap-to-pay kind of weird shit so I'm really not suprised lol
Since there's duplicate name fields or this one, is it safe to update both to
t , assuming yesIt is yes! Both should have true anyway
done:
and another one... I'll work though this one as well really quick
Here is the list of standard fields. https://github.com/twentyhq/twenty/blob/main/packages/twenty-server/src/engine/workspace-manager/workspace-sync-metadata/constants/standard-field-ids.ts
More than a isCustom bool, we have a standard-id that we use to keep in sync those fields regardless of their names (which can change, where standard-ids won't)
So in theory, all field-metadata that don't have a standard-id, should be isCustom: true
sharepointUploadLink is fixed but hitting more weird shit, looks like a uique contstraint violation
So this query
should work
it's cut, do you have the rest?
Ah
Sorry huge output was trunacting
From which version are you upgrading?
Checking the git blame one sec
last few bumps:
0.22.0 -> 0.22.1 -> 0.23.0From here https://github.com/twentyhq/twenty/blob/main/packages/twenty-server/src/engine/workspace-manager/workspace-sync-metadata/constants/standard-field-ids.ts, do you see any of those fields that you could have been created by yourself before and that could clash with the creation of new fields?
easier with a blame https://github.com/twentyhq/twenty/blame/main/packages/twenty-server/src/engine/workspace-manager/workspace-sync-metadata/constants/standard-field-ids.ts
That's something we are not really good at yet. If we introduce a new feature that needs the creation of a new table, we need to make sure the DB does not have already one with the same name. Since we allow users to create custom objects which in result creates a table, that name could have been already taken for the table (or the field/object/api)
At least we should improve the error above. The unicity constraint you see throws an error directly from the DB/ORM, we should catch that and log the field that's being created
So this'll fix the unique constraint issue, safe to run now?
Sorry, this request was suggested because you had to update multiple fields isCustom. It seems you are done with that error now and the unicity constraint won't be fixed by this query because it's based on the ['name', 'objectMetadataId', 'workspaceId'] columns regardless of the isCustom column
So far suspect field names:
- domain
- stage
List of isCustom=TRUE:
Just as a note I use
deal(s) and lead(s) instead of opportunity(s) those do share many of the same field names as the Opportunity objectMmm, should be fine since the constraint is based on the name and the object so it won't fail if Opportunity and deal have the same fields
I don't see anything wrong with your fields. If you wait for a bit I will try to push a change that will improve the error
Absolutely, thank you for the support, I really appreciate it
@Skyfox675 do you still need support here? 🙂
Yes please, waiting on a more verbose query error on the duplicate key constraint
@charles not sure if you saw my reply, thanks!
oh yes!
I've faced this non verbose issue too (logger is a bit too silent). I'm pushing a patch in 0.23 tonight to get more verbose logs, will ping you
I did just notice that v0.23.1 was pushed like 6 days ago lol I'll wait for your message though on next steps
Logs have been improved, I'm still working on the patch, will ping you today
Thank you!!
Hi @Skyfox675 sorry for the long dealy, the patch has been published and includes more verbose logs!
Thank you! No worries 🙂 updating now and will grab the new error logs
yarn command:prod workspace:sync-metadata -f
In your DB, what does e0f94b08-826c-4382-9252-205e051de179 objectMetadata.Id corresponds to (metadata.objectMetadata table)
Looks like
auditLog
Ok! could you show me the fieldMetadata having objectMetadataId = 'e0f94b08-826c-4382-9252-205e051de179'
(the fields of auditLog)
I suspect that the standardId of the "objectMetadataId" field of auditLog is wrong
(it's confusing because the field itself is named objectMetadataId :p )
My psql is very rusty lol so it'll be a min to grab the tables
the standardId of this "objectMetadataId" field should be: '20202020-127b-409d-9864-0ec44aa9ed98'
I think it's different on your end
the standardId identifies a given standard field, during metadata-sync we use it to understand if we need to update or create a field
here we are trying to create this field so the standardId must be wrong (different than then one we have in code base)
Interesting! Ok sounds good so we need to check the fieldMetadata table where objectMetadataId = 'e0f94b08-826c-4382-9252-205e051de179'?
yes
could you change the standardId of the "objectMetadataId" field displayed on your screenshot?
on your screenshot you have two standardIds ending with 00a
the second one (objectMetadataId) should be 20202020-127b-409d-9864-0ec44aa9ed98
I think it's an error we missed in an old version of Twenty (likely 0.20)
How's this look?
UPDATE "metadata"."fieldMetadata" SET "standardId"='20202020-127b-409d-9864-0ec44aa9ed98' WHERE "name"='objectMetadataId';Looks like same error, new field:
fieldMetadata matching that key:
Weird!
It should be 20202020-d6de-4fd5-84dd-47f9e730368b !
oh ok!
UPDATE "metadata"."fieldMetadata" SET "standardId"='20202020-127b-409d-9864-0ec44aa9ed98' WHERE "name"='objectMetadataId'; ==> this is wrong, you have updated all fields having "objectMetadataId" name
that's why you are facing the new issue
UPDATE "metadata"."fieldMetadata" SET "standardId"='20202020-127b-409d-9864-0ec44aa9ed98' WHERE "name"='objectMetadataId' AND objectMetadataId = 'xxxx'
Actually, we only have two fields named "objectMetadataId", one on "AuditLog' and one on "View"
For AuditLog you are fine
For View you need to make it 20202020-d6de-4fd5-84dd-47f9e730368b as it was before your update
Sorry catching up 😅
Sorry I'm not sure I"m following
yep, update the "View target object" one to be 20202020-d6de-4fd5-84dd-47f9e730368b
This feels off, mind double checking?:
UPDATE "metadata"."fieldMetadata" SET "standardId"='20202020-127b-409d-9864-0ec44aa9ed98' WHERE "name"='objectMetadataId' AND objectMetadataId = '67dcb7ee-c385-48c5-864a-b463da21a696';same but with 20202020-d6de-4fd5-84dd-47f9e730368b
UPDATE "metadata"."fieldMetadata" SET "standardId"='20202020-d6de-4fd5-84dd-47f9e730368b' WHERE "name"='objectMetadataId' AND objectMetadataId = '67dcb7ee-c385-48c5-864a-b463da21a696';LGTM!
then, actually you should not run the sync-metadata
just do:
- yarn command:prod upgrade-0.23
(the sync-metadata is already included in this command)
Something is up, 0 rows updated
sorry!
UPDATE "metadata"."fieldMetadata" SET "standardId"='20202020-d6de-4fd5-84dd-47f9e730368b' WHERE "id" = '67dcb7ee-c385-48c5-864a-b463da21a696';
and statement is off just saw it
that did it thanks!
ok, before going further
could you check what you have in metadata workspaceMigration table?
do you have any row with applyAt = null?
(I'm making sure you don't have anything stuck as you faced issue during the migration)
If not you are good, you can run yarn command:prod upgrade-0.23 and it should work :p
Looks from from where I'm sitting lol
4-5 file not found's, looks like migration failed for workspace:
I don't need those attachments if it's easier to purge them, was doing testing with the attachments REST API
damned 😄
the issue is that you don't have the correct messageChannel_syncstatus_enum
I think it's a legacy issue on your workspace prior to 0.22
Could you run the following queries:
replace ${schema} by your workspaceSchema
Here we go 🙂
good!
and then you can retry to upgrade
Is there a missing
; on the last line?yes indeed
sweet just wanted to double check, thanks!!!!
Looks like the UPDATE had 0 items, is that ok?
yes 🙂
it was just in case!
Trying the upgrade again 🙂
Figured that was the case lol
I think we're good? lol Figuring that the files can be ignored and I can hunt them down and delete the references and readd
yes we are good!
great!
is your Twenty up and running?
Amazing thank you so much!!
Checking now
Looks like I've got a hung loading on /companies
are you using redis?
nope, just the db, server and worker so far
ok, could you share your network tab?
from your browser console
sure thing
All looks good there:
Seeing a GraphQL error in the pod logs on the server though:
mmmh oh yes
Looks like it's this query:
Ok so we have introduce new fields that are composite and composed of different subfields
and the library we use to query the data (pg_graphql) has a limitation of the number of parmeters we can send (100)
I think the issue is that you have too many fields or objects in your workspace
could you share the whole query?
query CombinedFindManyRecords...
(in your screenshot, it's cropped in the middle by ...)
The good news is that next week when we release the 0.24 we get rid of pg_graphql
It has been a 6 months effort to get rid of it but here we are!
I think I got it 🙂
So /settings is still working
if you can disable some fields there you should be good
was able to go into the Company data model and disable a bunch of fields I've moved to my new Automation object
Just loaded up!
I can break it to grab the query if you're interested still
no it's fine!
great!
0.24 will remove this limitation
happy we figured this out! I promise it will become smoother and smoother
Thank you so much, seriously couldn't have done this without both you and @Weiko really appreciate the help!
Oh quick question while I have you 🙂
Can you confirm which storage paths should be persisted across pod restarts?
-
/app/docker-data
- /app/packages/twenty-server/.local-storage
Assuming both but wanted to double checkboth yes:
- /app/docker-data: this one enables us to differentiate new setup from existing one to see if we need to initialize the database
- /app/packages/twenty-server/.local-storage: this one is the storage for Twenty for files and future serverless functions (automations) if you choose to store them locally
Amazing, thank you!
@xSwampxFox @xSwampxFox (seriously figure out your accounts lol) ^here's your answer you need for your open PR 🙂