CosmoBean
JJanusGraph
•Created by CosmoBean on 4/1/2024 in #questions
Issues faced for consistent indexing (both Composite & Mixed) [ElasticSearch]
Really Sorry for all the trouble 😓
18 replies
JJanusGraph
•Created by CosmoBean on 4/1/2024 in #questions
Issues faced for consistent indexing (both Composite & Mixed) [ElasticSearch]
@Boxuan Li , while i was using multi-tenacy, there were more configs to be given to the elastic search as well, for the graphName. Marking this as resolved.
The index was being created with janusgraph_IndexName, where as it was supposed to be created in tenant_IndexName, It was a misconfiguration, that wasn't carefully considered.
Thanks a lot for the patience, and answering the questions
18 replies
JJanusGraph
•Created by CosmoBean on 4/1/2024 in #questions
Issues faced for consistent indexing (both Composite & Mixed) [ElasticSearch]
Hey @Boxuan Li ,
For the initial details of code,
1. Using embedded version of janusgraph through a java springboot application.
2. Before creating the schema, all the transactions and dangling management objects are closed through the code.
3. Vertex Properties and Edge Labels along with properties are created.
4. Based on the config that I have given inside of a hashMap, specific properties are indexed for a vertex. [ Both Composite and MIxed in my case] --> Unique Constraint Indexes
5. Once we build the index [ According to the documentation ], We check for if the Index is in REGISTERED state.
6. Once REGISTERED state is confirmed through the call, a REINDEX action is called to enable the index.
7. Once all of the indexes are enabled, a call is made to HTTP endpoint of janusgraph to create an entry in the "configurationManagementGraph" in regards to the elastic search endpoints and scylla db endpoints for the specific graph.
8. Once all of the schema creation is done, I open the gremlin console through a terminal, and try to open the graph and try to open the management to print out the schema and see, if all the indexes are ENABLED, This part is successful without any issue.
9. If i try to see if any index has been placed in the ElasticSearch index, I wasn't able to find any trace of the index.
If you need the code in java for any specific part, please let me know, as the over code is big.
18 replies
JJanusGraph
•Created by CosmoBean on 4/1/2024 in #questions
Issues faced for consistent indexing (both Composite & Mixed) [ElasticSearch]
Hey, I am sorry that I am currently not able to give you screenshots.
Here's a more detailed explaination of what happneed
1. Schema was created using the management Object, to insert structured data, along with the indexes, which were enabled using the REINDEX, schema action, before start of insertion.
2. Once Schema & Indexes were created. I made sure to verify that the status of the Index had been set to ENABLED. [ which is a good to go sign that whatever data is going to be inserted is going to be indexed]
ISSUE 1:
after this step of schema creation, which was done using MIXED and COMPOSITE indexes.
I wanted to see, if I was able to find the mixed index which is supposed to be created by Janusgraph. the index wasn't available/created on ElasticSearch.
I assumed, that since, the data has not been inserted, maybe janusgraph is waiting for index to be created.
The data was inserted into janusgraph, waited to see if any indexed data was present on elastic search. Still wasn't able to find any indexed data.
ISSUE 2:
To mitigate the issue of index not being created, I inserted a part of the data, and used the REINDEX operation to make sure the indexes were created, and this operation was working as expected and I was able to find the indexed data.
As the issue of dangling management objects is known, I closed any and all the dangling management objects.
this part data that was reindexed, was performing as expected. Now, I reingested all of the remaining data. Where I found that, the composite index was being updated, but the mixed index data was not being updated. Causing an issue of data inconsistancy when index queries are being used.
Apologies for the delayed reply, I missed the message, please feel free to tag me as required.
Thanks a lot for the help
18 replies
JJanusGraph
•Created by CosmoBean on 4/1/2024 in #questions
Issues faced for consistent indexing (both Composite & Mixed) [ElasticSearch]
Any Idea on the inconsistent indexes, where the indexes aren't getting updated unless I re-index the whole data again and again?, this is something that would be critical with the huge data that we were trying to take onto production
18 replies
JJanusGraph
•Created by CosmoBean on 4/1/2024 in #questions
Issues faced for consistent indexing (both Composite & Mixed) [ElasticSearch]
Thanks a lot for the reply, I think it is about the locking, I thought once a Uniqueness index is placed( Composite Unique index has been placed). the uniqueness would be auto handled by janusgraph. Will look into this. Thanks once again
18 replies