Justin - Spotter (embedded in a visualization) ...
Spotter (embedded in a visualization) does not use the trained responses. If I train Spotter on a question and ask the embedded version, it will not work. If I ask the stand alone version, it will work. I understand that the embedded uses the visualization as a starting point, but even when training on specifics in the visualization it does not work.
Example (images in thread)
In image 1, I show the starting point of the Liveboard visualization. It is spend over the last 2 years by spend type. In image 2, I train spotter on "how much did we spend last month". The search bar is the same, except i'm looking at last 30 days instead of previous 2 years. In image 3 you'll see the result I get when asking the embedded Spotter the question. When I ask the standalone spotter, I get the expected answer.
In this example I don't think the context changes enough to justify not using the trained response. I even tested training it with the Approved Date Monthly instead of Approved Date daily (only thing that was different was last 30 days instead of last 2 years) and it did not work.
10 Replies
Can we get har file to debug
@Mourya Balabhadra yes, it is safe to send that publically or should I dm?
DM me @Justin
Thanks @shikharTS , I sent those over to you
Hello, just following up on this. Have you had a chance to look at the HAR logs? @shikharTS
Yes I have forwarded this to the relevant team.
I would suggest to create a support ticket, so you can directly get updates from the team
Thanks! Is there an estimate on when to expect a response? I was told from support to ask in here.
Ohh I see. Ask them to refer to my message and forward the ticket to the spotter team. They should be able to help
thanks
This is something I got from the internal team :
it looks like the standalone spotter was able to give the right answer because it was pulled from cache (the answer that the user provided feedback), while the embedded spotter got wrong answer [Approved Date] = 'last month' from LLM (not from cache) even though the feedback below was provided to LLM:
{"query":"how much did we spend last month","tokens":"[Total Cost Base Currency] [Spend Type] [Approved Date] [Approved Date].monthly [Approved Date] = 'last 30 days'"}
I think this is because the user is asking about "last month" while expecting "last 30 days" and this confuses LLM.
It seems the feedback is provided, but standalone spotter's answer was pulled from cache hence the result was correct..