Hello, we recently rolled out an
Hello, we recently rolled out an integration with our site and ThoughtSpot. We did a delayed rollout where only 10% of our clients were using it and everything was fine but now we've rolled it out to all of our clients the Fetch Permissions of Principals API endpoint (2.0/security/principals/fetch-permissions) is extremely slow for a seemingly simple request that returns no data. Specifically we're using it to get what's shared with a set of groups like this:
I've run some tests and it does run faster setting a record size, even a very large one but another weird thing I'm seeing inconsistent results with using the record size parameter- e.g. I send record_size = 5 and it returns nothing under metadata_permission_info but if I send record_size = 10 it returns results.
Should I file a ticket or are their other endpoints I could utilize?
9 Replies
@shikharTS - Can you PTAL ?
@Martin Fisher - Can you send some trace-ids so we can start debugging on our side ? Also, what is your usecase for this API, If we know the usecase, we can see if there are other APIs which we can use
Running some requests now to get trace-ids.
Our use case is that we allow users to create and then share liveboards and answers with groups via our app.
In our app we use the metadata/search API to get the items a user has access to which gives us the creator information but not the share information. We need the share information to know if the items the user has created are shared with any groups or not so we use that fetch permission API for the specific groups to determine that.
Unfortunately the API is timing out so I can't get the trace ids
^ @shikharTS @utsav.kapoor
With record_size -1 things are timing out. I put a record size of 1000 and it took 26 seconds to run - trace id: 94c6bcb6-c4b0-42ce-93f6-606d955b8e08
same exact request but with record size of 10000 took 54 seconds trace-id b92a80aa-aa61-48ac-898d-db5e30b21804
Thanks. We are checking this. Lemme also run the usecase internally to see if we can achieve this by using any other API
Also, can you give the scale idea in terms of how many groups and objects you have in your system ?
Can you also DM me your cluster and customer name, so we can fetch logs for the trace ids ?
So, just had a chat internally with one of our Specialist. Here is the recommendation:
1. Cluster Upgrade to 10.3
2. Modify the approach to the following:
It will probably be faster to do..
POST /api/rest/2.0/metadata/search
{
"include_headers": true,
"record_offset": 0,
"record_size": -1,
"created_by_user_identifiers": [
"c56267e9-4136-4424-8e2d-db31888e7623"
]
}
and feed those guids to
POST /api/rest/2.0/security/metadata/fetch-permissions
{
"metadata": [
{identifier": "guid1"},
{identifier": "guid2"}
],
"record_offset": 0,
"record_size": -1,
"permission_type": "DEFINED"
}
permission_type=DEFINED was introduced in 10.3. Default is EFFECTIVE and its actually a really expensive operation in the system.
^ @Martin Fisher @Matt
Thank you, I did notice the metadata filter and was playing around with that and it did seem faster. The permission type change also makes sense. I assume DEFINED looks for explicit shares for the group/metadata combination? That would do what we want
yes., Effective actually checks the whole graph which includes underlying worksheets, tables etc. thus becoming super expensive
Hello @utsav.kapoor @shikharTS were told by Ashley Jodry that our system would be upscaled by 11pm PST 12/5 to resolve the issue of our system being completely unusable to all our customers. This has not happened yet. I'm not sure who to reach out to at this time, but we need this to happen immediately so we can get the system back up and running for all our customers. This issue is critical.
I am in connect with Ashley Jordy. Did you implement the above changes ? Upscaling might help but we recommend you put the above changes as well
Also, I am talking internally so we can upgrade you to either 10.3 or 10.4