Validation error
Hi All, we are in poc phase and trying to call import tml api to create/validate sql view, i m getting a validation error, Here is the curl from develop tab
curl -X POST \
--url 'https://syncari.thoughtspot.cloud/api/rest/2.0/metadata/tml/import' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Authorization: <token>' \
--data-raw '{
"metadata_tmls": [
"guid: 6d763960-e259-4718-af09-d4ab62100231 sql_view: name: Account_Oppty description: Account and Oppty join connection: name: thoughtspotpoc_postgres sql_query: "select o.amount, o.accountid, o.name, a.name \"accountname\" from syncari_qonzo4.syncari_qonzo4.opportunity__c AS o \ninner join syncari_qonzo4.syncari_qonzo4.account AS a on a.syncariid = o.accountid" sql_view_columns: - name: amount sql_output_column: amount properties: column_type: MEASURE aggregation: SUM index_type: DONT_INDEX - name: accountid sql_output_column: accountid properties: column_type: ATTRIBUTE index_type: DONT_INDEX - name: name sql_output_column: name properties: column_type: ATTRIBUTE index_type: DONT_INDEX - name: accountname sql_output_column: accountname properties: column_type: ATTRIBUTE index_type: DONT_INDEX"
],
"import_policy": "VALIDATE_ONLY",
"create_new": false
}
52 Replies
Can you post the response you are getting?
"root":[1 item
0:{1 item
"response":{1 item
"status":{2 items
"error_message":"Invalid YAML/JSON syntax in file."
"status_code":"ERROR"
}
}
}
]
Error i m getting ^^
Can you please update me regarding this
Looks like the TML syntax you are using in the API has invalid syntax.. Can you check the syntax once on your side or contact TS support for the same? The team should be able to help
@rohitkanchan Can you send the TML in a formatted syntax, we can check if there is anything invalid
The tml which our poc support contact gave that did not work either, the tml string which i used above curl call is from thoughtspot ui where u can edit tml
Can you please provide me valid Tml which would work
@ashish what do u mean by formatted syntax? I did add tml string in my request listed above
I mean with tabs and newlines intact. It's hard to read otherwise
I was able to solve this
Thx for the help
Seems something was wrong with tml
Hi
I m importing tml a valid yaml but i m getting invalid yaml error
How can i debug this what is wrong
I m validating a sql view
{
"metadata_tmls": [
"sql_view:\n name: testsqlviewds\n description: test\n connection:\n name: thoughtspotpoc_postgres\n sql_query: >\n SELECT Opportunity.amount Opportunity:Amount FROM syncari_qonzo4.syncari_qonzo4.opportunity Opportunity"
],
"import_policy": "VALIDATE_ONLY",
"create_new": false
}
Hi, while integrating thoughtspot api, i m getting error when trying to create connection
I m getting 401 when trying from develop tab while it should be authorized right
When i m trying to create connection using api from my java unit tests then i m getting 500
Can someone help me please to find out the issue
Following up on this, please let me know what you need from me
Can we have the curl that you are trying out? Also can you check if you have permission to create connection form the UI?
What is the 500 error you are getting?
When i tried through my java unit tests then i m seeing 500 with message internal server error
And nothing else
And yes i have permission to create
When i tried using develop tab then i got unauthorized even though bearer token is auto generated
If it is 500 we might need to check the logs. Can you log a support ticket for TS so the correct team can look into it?
Yes that is wierd, we should never get unauthorized when trying through develop tab
Can i send a request payload for 500 here
sure
And you can point me if anything wrong there
TS Payload:{"id":null,"name":"QWAK1T_CONNECTION","data_warehouse_type":"POSTGRES","data_warehouse_config":{"externalDatabases":[{"isAutoCreated":false,"schemas":[{"name":"schemaname","tables":[]}],"name":"schemaname"}],"configuration":{"password":"<dbpwd>","accountName":"Non-prod","port":"5432","host":"<hostip>","user":"<dbusername>"}},"validate":true}
And i tried validate true and false both
I do not see "id" field in the payload. Also I am not sure if "data_warehouse_config" is correct or not
But Id is not there in the payload options
Let me try to remove id
Though i think extra attribute should not cause error
Also I hope you have these Requires DATAMANAGEMENT (Can manage data) or ADMINISTRATION (Can administer ThoughtSpot) privilege.
I m using tsadmin user
Even after removing id, still same issue
Can you log a support ticket through SRE. We will get the relevant teams to look into it..
Oh, strange, i got information from TS member that ping engineering on discord, he said you can help to debug
I can log support ticket anyway
Ohh this might need a ticket. As the experts for this are not on discord..
Sure
the 401 you are getting is it for other APIs as well or only for this particular API?
Its same
Create connection api
Through UI it works fine
For other APIs it works?
Yes
It should not be happening. Can you send over the HAR file while you are trying out in develop tab?
I have discord on my mobile
I m not sure how do i send that
I have to figure out that
You can add it as part of the ticket that you log.
Sure
Logging now
I created support ticket
I logged a ticket but still waiting for resolution
Can you follow up with the SRE? It might take some time to resolve.
Hi
I have one more question while our integration is in progress
Hi, I am trying to fetch all liveboards using search metadata api. I am able to fetch all liveboards, the only issue which I am seeing is that api returns all auto created boards as well. In request I am setting false for attribute include_auto_created_objects. Can you please help me what attribute should i set false so that api does not return default liveboard and only return user define liveboards
so auto_created_objects are not the same as system defined objects. It might be a bit difficult to not return the system defined objects, but you can explicitely remove them from the list. Here is how to remove them:
1. Find objects created by user whose name is 'system' using attribute " "created_by_user_identifiers": [
"system"
]"-> these are all the system defined objects
2. Explicitely exclude those objects using "exclude_objects": [
{}
] attribute.
Okay thx, what is the use of attribute which i asked about?
I can filter after fetching but it is unnecessary fetching system liveboards when user does not need those
Auto_created liveboard are AI generated liveboards
Okay
Thx
One quick question while i m working on integration, update connection would need all table’s information again or just what are we adding to existing connection?
I think you would need all table's information again. as written in our documentation "If you are updating tables or database schema of a connection object: a. Add the updated JSON map of metadata with database, schema, and tables in data_warehouse_config"
Okay
Strange though
Hi, another quick question, how can i fetch tables guid using api, i tried search metadata api but that does not return Tables
Which tables?
Postgres connected schema tables , i want to restrict usage of tables for some groups
There should be corresponding tables in TS for those tables. Maybe you can restrict the usage of those tables? Use LOGICAL_TABLE parameter to search for them.
Okay, will try that
Hi Shikhar quick question, if i create a user in multitenant environment, i want to give access of multiple orgs to that user, how can i do that
You can provide a list of orgIds in the /user/create API
What i meant to ask, is user org specific ?
No one user can be part of multiple orgs
Okay so can i search user without passing orgid
I think we should be able to do it. You can try it out and let us know if you do face any error..
You can search for users even with name for a multi-org user
Sure will try
Hi, i m trying to create worksheet using import tml api but i m getting invalid yaml/json, can u please tell me what is wrong
I tried to create worksheet using Import TML, getting 500 back
{
"metadata_tmls": [
"worksheet": {
"name": "testdataset2WS",
"description": "testdataset2",
"tables": [
{
"name": "testdataset2"
}
],
"table_paths": [
{
"id": "testdataset2_1",
"table": "testdataset2",
"join_path": [
{}
]
}
],
"worksheet_columns": [
,{
"name": "name",
"column_id": "testdataset2::name",
"properties": {
"column_type": "ATTRIBUTE",
"index_type": "DONT_INDEX"
}
} ],
"properties": {
"is_bypass_rls": false,
"join_progressive": true
}
}
}],
"import_policy": "ALL_OR_NONE",
"create_new": true
}
I tried online tools to validate, it is showing valid
@shikharTS any thoughts ?
@rohitkanchan I am checking this. Also can you please post this in the #dev-help channel so that everyone from TS is aware and anyone can help you with the resolution?
Can you also post the error response ?
[
{
"response": {
"status": {
"error_message": "Invalid YAML/JSON syntax in file.",
"status_code": "ERROR"
}
},
"request_index": 0
}
]
This is when i use postman or develop tool of ts
I get 500 if i call this from my code
@shikharTS anything u want me to try?
??
@rohitkanchan - I have asked a dev to reach out to you on the main thread. They will reach out
Okay