I tried to create worksheet using Import
I tried to create worksheet using Import TML, getting 500 back
{
"metadata_tmls": [
"worksheet": {
"name": "testdataset2WS",
"description": "testdataset2",
"tables": [
{
"name": "testdataset2"
}
],
"table_paths": [
{
"id": "testdataset2_1",
"table": "testdataset2",
"join_path": [
{}
]
}
],
"worksheet_columns": [
,{
"name": "name",
"column_id": "testdataset2::name",
"properties": {
"column_type": "ATTRIBUTE",
"index_type": "DONT_INDEX"
}
} ],
"properties": {
"is_bypass_rls": false,
"join_progressive": true
}
}
}],
"import_policy": "ALL_OR_NONE",
"create_new": true
}
80 Replies
@rohitkanchan - Did you use V2 Playground to see if this is working or you are getting the same error ?
Yes
I created ticket too
Logged this support case Case No.
00372437
I need support for this, it is a blocker for our implementation
@rohitkanchan how did you create this worksheet? We do have a issue going around here and the workaround is not very pretty but it should help for you
I m creating worksheet using sql view, and i created this programmatically
I have hard coded tml, but replacing values with dynamic values
What issue is going on?
And what is the work around
Hi @rohitkanchan :
If you are doing this programmatically, there should be no issues. Looks like you are passing metadata_tmls : [JsonObject] which is incorrect.
Here is the sample how the values should be passed in metadata_tmls
metadata_tmls : [String]
"metadata_tmls": "["worksheet": {"name": "testdataset2WS","description": "testdataset2","tables": [{"name": "testdataset2"}],"table_paths": [{"id": "testdataset2_1","table": "testdataset2","join_path": [{}]}],"worksheet_columns": [{"name": "name","column_id": "testdataset2::name","properties": {"column_type": "ATTRIBUTE","index_type": "DONT_INDEX"}}],"properties": {"is_bypass_rls": false,"join_progressive": true}}]";
I m not passing jsonobject
Its tml which i was passing
Can we connect over zoom call to discuss this now?
Do u think ur tml would work because i can try that from develop tab
Sure
I can connect
Is there a link?
Please join this https://thoughtspot.zoom.us/j/96924491130
Zoom Video
Join our Cloud HD Video Meeting
Zoom is the leader in modern enterprise video communications, with an easy, reliable cloud platform for video and audio conferencing, chat, and webinars across mobile, desktop, and room systems. Zoom Rooms is the original software-based conference room solution used around the world in board, conference, huddle, and training rooms, as well as ex...
https://meet.google.com/wai-kstc-nct
Hi Sandeep, let me know when can u join
The json which u gave also failed
Hi @rohitkanchan :
Here is the summary for today morning's call:
- We tested the API on the playground using both YAML and JSON formats, and it worked as expected. There are no issues with the API itself.
- However, when tried integrating the API into their Java code, they encountered some parsing issues related to strings, JSON, or YAML. We spent about 15-20 minutes reviewing the code but couldn’t identify the problem, due to some errands we had to drop from the call.
@bill_da_golfer /@shikharTS : Could you please assist Rahul in the PST hours so that this is unblocked.
I am looking for support today to get unblocked on this one
Please let me know who can assist me and time as well
@shikharTS : Could you please help here
@rohitkanchan let us connect for this once you are available.
@shikharTS lets connect at 11:30 am or 12 pm PST?
Can u please share ur email, i will send u invite
Sent u invite for 12
I joined google meet @shikharTS
joining
Summary for the call in PST hours : We are able to get the code working, all the JSON formatting issues are resolved. There is one other problem though, when we are trying to use the SQL view created via TML instantaneously to create a worksheet, we are getting an error code 14516 from TS. We will resolve this internally and provide an update to the customer.
I tried even 3 seconds then also it failed
So that time is mot certainly fixed
Is there a way to know if safe to create worksheet or not
@shikharTS any update regarding this
@rohitkanchan - We have raised this to another team who works on TML. I see that you have opened a support case for this too. The respective team will reply on the ticket.
Okay sure, that team is not here in discord? Can u please tag them here
I have checked internally and they suggested to apply a tomcat change to avoid using sleep seconds. To apply this, we need 2 minutes of downtime for tomcat service restart.
Please let me know when can I apply the change? You can test once change is applied.
Please apply now
To Close the loop, this is resolved after applying the tomcat config change.
Thanks everyone for helping
Hi @shikharTS
I am again facing tml issues
This time with sql view
Response is invalid yaml
I did not change the format
Just a different sql now
I m getting again invalid yaml after formatting through library
Here is my worksheet json
TS Payload for Worksheet:{
"metadata_tmls": [
"{"worksheet": {\n "name": "testdataset2WS",\n "description": "testdataset2",\n "tables": [\n {\n "name": "testdataset2"\n }\n ],\n "table_paths": [\n {\n "id": "testdataset2_1",\n "table": "testdataset2",\n "join_path": [\n {}\n ]\n }\n ],\n "worksheet_columns": [{\n "name": "name",\n "column_id": "testdataset2_1::name",\n "properties": {\n "column_type": "ATTRIBUTE",\n "index_type": "DONT_INDEX"\n }\n },{\n "name": "amount",\n "column_id": "testdataset2_1::amount",\n "properties": {\n "column_type": "MEASURE",\n "index_type": "DONT_INDEX"\n "aggregation": "SUM"\n }\n },{\n "name": "isclosed",\n "column_id": "testdataset2_1::isclosed",\n "properties": {\n "column_type": "ATTRIBUTE",\n "index_type": "DONT_INDEX"\n }\n }],\n "properties": {\n "is_bypass_rls": false,\n "join_progressive": true\n }\n }}"
],
"import_policy": "ALL_OR_NONE",
"create_new": true,
"all_orgs_context": false
}
This is really very painful and prone to errors
Can you try the same using tml import API in playground?
Okay trying that
Same errkr
Error
When i removed MEASURE column it worked fine
I don’t see any issues in measure column as well
Can we import this TML through the TS UI? Maybe that will help identify the error?
I did try that
Same error
Invalid yaml
Lets have a call around this as well
Can we have call around 11:30?
I will send u invite
Hi, i have a quick question if i import a metadata to update a sql with name already exists and isCreate false then it should update existing sql view and should not create new one right
@shikharTS any insights, i tried this , it is creating new each time
@rohitkanchan Can you try this and make sure the GUID is the same as the target one? I think
isCreate means that if the object with the GUID exists create a new one, not don't create a new one if the old one doesn't exist, i.e. force copy. If they did have the same name and that name is unique, I would have expected an update, though.@rohitkanchan it checks according to the GUID and not the name.
I did try that
Got another error
What error did you get?
Response of SQL View creation is <200,[{"response":{"status":{"error_message":"The following operations currently unsupporColumn name: amount, sql_output_col: amount. <br/> is only removed from column list. Column name: amount, sql_output_col: amount. <br/> should be removed from SQL query to be considered deleted. <br/>Column name: closedate, sql_output_col: closedate. <br/> is only removed from column list. Column name: closedate, sql_output_col: closedate. <br/> should be removed from SQL query to be considered deleted. <br/>Column name: isclosed, sql_output_col: isclosed. <br/> is only removed isclosed. <br/> should be removed from SQL query to be considered deleted. <br/>","status_code":"ERROR","error_code":14501}},"request_index":0}],[date:"Mon, 23 Sep 2024 22:09:36 GMT", content-type:"application/json", set-cookie:"rememberMe=deleteMe; Path=/callosum; Max-Age=0; Expires=Sun, 22-Sep-2024 22:09:36 GMT; SameSite=lax; secure; SameSite=None", "clientId=1dd2df8c-497d-480f-9c75-9c9dcbaaa922; Path=/; Secure; HttpOnly; SameSite=lax; SameSite=None", "rememberMe=deleteMe; Path=/callosum; Max-Age=0; Expires=Sun, 22-Sep-2024 22:09:36 GMT; SameSite=lax; secure; SameSite=None", x-callosum-incident-id:"79af93e4-a2a7-4db3-9c2f-a057b5f375af", x-callosum-request-time-us:"212653", x-callosum-trace-id:"5a591341-292a-4859-a6fb-de80d94bce7b", x-content-type-options:"nosniff", x-ua-compatible:"IE=edge", x-xss-protection:"1; mode=block", server:"ThoughtSpot", content-security-policy:"default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://mfe-cdn.thoughtspotdev.cloud https://mfe-cdn.thoughtspotstaging.cloud https://mfe-cdn.thoughtspot.cloud https://docs.thoughtspot.com https://cdn.segment.com https://cdn.mxpnl.com https://api.segment.io https://*.intercom.io/ https://js.intercomcdn.com/ https://*.wistia.com https://*.wistia.net https://pendo-io-static.storage.googleapis.com https://cdn.pendo.io https://app.pendo.io https://data.pendo.io https://pendo-static-
I am passing updated sql
I think you need to pass the column names as well in the TML. Just updated SQL will not work
Huhhh
Why
If you download from TS UI even there it shows the column names as well, that is just how it is stored in TS..
How do i add new columns
Please go through this : https://docs.thoughtspot.com/cloud/latest/tml-sql-views
TML for SQL views | ThoughtSpot Cloud
Use TML to modify SQL views in a flat-file format. Then, migrate the view to a different cluster, or restore it to the same cluster.
Other way which i m using is create new sql view and update worksheet with new sql view
Then delete old sql view
That might be time taking. Maybe you can download the old sql view TML and edit that only to update the sql view
I went with my approach for now
The approach which you mentioned, in that i have parse sql which we are not doing today
I have another quick question, would be easy one for you guys, i have live boards and answers, i want to share those only with given groups, i want other groups wont see that
How can we do that while creating liveboard or answers
@shikharTS any thoughts for this?
After creating these liveboards/answers you can use the share API to share them just with those groups.
Okay, sure i will check with Nikhil if he call our backend api after that from UI
Quick question if we search user in embed cluster like ours, without passing orgid, would it search in all orgs?
So you search without the orgId, if you want to search across all orgs, get the token with orgId 0. Replied the same on mail
Alright thx
Hi, i created a support ticket for user which i created through ui and given administrator access of primary org, when that user is accessing other orgs through embed, they are seeing user does not have access while from ui user can access all other orgs
Support ticket response is this
Noticed that you are not using JIT user creation with the token flow, and uses secret key to retrieve the token. I noticed on the call that you mentioned you create users using the API.
Please correct me if I am wrong, I suppose that the issue was due to creds being same in embed as well as TS. And since the user is a valid user in ThoughtSpot, whenever they try to login by going to the actual ThoughtSpot instance it will work since the creds are valid. Would it work for your use case to consider changing the user password to be different for embed logins than that in the actual ThoughtSpot cluster?
Also since this is more of an implementation question/task, I would also like to request you to consider joining the TS developer discord, we have our experts there who can help you on everything related to embeddin
While initilizing in embed, we mention orgId, I think that is the org which they will have access to. How are you initializing the TS embed app. Also we are seeing very high activity for v2/auth/token in the TS cluster. Are you fetching token even when the user is successfully authenticated in TS? This will result in performance degradation @rohitkanchan
@shikharTS we are fetching token for each api call we make from backend, which should be minimum. Let me know if you can jump on a call. According to my understanding Administrator access in primary org should have access to all orgs and everything
Sure lets get on a call
I don't think we should be doing v2/auth/token call everytime
Sending you google meet invite
I joined google meet which i sent you over email
@rohitkanchan you might need to assign orgs to the users for them to see the content of other orgs. Are you doing that?
Hi quick question, during search i see only user can select one table fields and others grayed out, what is the reason for that if multi select for ans is enabled
@shikharTS any thoughts ?
This happens if thhere is no Joins between these tables
Okay thanks
Another quick question: lets say of i unshare a worksheet with a user but worksheet is still shared with the group user belongs too, user would be able to see the worksheet?
@shikharTS any thoughts
I have another question, lets say if i created a connection and i have created a liveboard using that connection, is it okay to update that connection
Is there any impact?
1st question : yes the user would be able to see the worksheet.
2nd yes it is okay to update the connection.
Also please direct these questions to your SRE and they will have a CSM help you out with these questions. We mostly support discord for queries related to embed questions and code.
Okay, last question @shikharTS , how do i add limit for the number of records returned in search
There is a back end flag for it.
Okay thx, so i need to log a support ticket?
There is no keyword like limit in sqls
yes I think there should be.
Okay
I just found top, did not find limit
Another question @shikharTS is have logged a ticket where an api is failing, ur support team is saying get help here
Case number 00374783
How can i get faster help on that
Can you post the error here?
I can help you out here
I searched a group using group name but group now found
Then code tried to create a group with same name but then got error that name already exists
Strange right
Search should have returned existing group
The group might be present in a different org. Try getting a token from that org and use that token to search for that group.
I m using same token to search n create
And it should be present in that group
Will be able to jump on a call on Monday, i can show u in our logs
Sure lets get on a call today?
Can you do it tomorrow morning around at 11 am
I think the SRE scheduled a call
@shikharTS the solution which u told regarding permission, i m facing challenge related to that
Can we jump on a call
Or anyone can help me
Question , if a user belongs to group x and creates a liveboard then it should be accessible by all user of that group
@Sandeep , @utsav.kapoor any one in IST timezone
Can you forward these questions to the CSM, they should be able to help you with these queries.. Ask on the support ticket to get you connected to a CSM..
I will ask @Nanditha internally as well
There is lot of confusion, when i open support ticket they say ask here and when i ask here you guys redirect me back there
Question i asked is supposed to happen but i m seeing that is not what is happening
I see let me try to repro this tomorrow.
Do u have few mins to jump on call again tomorrow morning around 10:30 am PST
I can send you invite
We can try to solve and brainstorm these problems together
Sure lets meet at 10:30am
Okay, let me send u invite
@rohitkanchan when did you see error last according to your logs?
I am not able to find this error in the logs for the past 4 days atleast
@rohitkanchan so I tried to repro the issue, I think I know what might be happening. You cannot create group in other orgs using the primary org token. If you are using a primary org token and the user is present in the different org, the user search will return the correct result but creating a group will give this error of .
Trying the same thing with the token of an org in which this user is present allows to create the group and does not throw an error
Can you please check your code to make sure to set the correct org while you are calling the /auth/token?
@shikharTS i have to check code but why second time it is working?
Second time same thing works fine? And why error is about user and not about authentication token
I do not think the 2nd time same thing will work. If you try to create a group for a user in a different org using a primary org token, it will always throw an error. 2nd time are you creating a group in the primary org? Only then will it work. The error is about the user because it can't find the user from the org for which you have created the token. We cannot give the error about the token to the user as it will be not secure, the token is fine by itself, but the user being searched is of a different org.
Okay thx, i think i understand why it works second time, i will change token before creating group in code, will test today
@shikharTS i think you found the root cause, i tested in non prod, it seems to be working there
Now only issue which we are looking for is permission issue which i talked about over call