shivam.choudhary
Explore posts from serversJJanusGraph
•Created by shivam.choudhary on 8/14/2023 in #questions
Impact of ID Pool Initialisation on Query Performance
@Boxuan Li any suggestion on this, currently the idpool gets initialised only after the instance start receiving write requests and the queries start to timeout for ~15 minutes until all the idpool are initialised.
Is there a way to initialise the idpools before we send the write throughput to the janusgraph instance?
5 replies
JJanusGraph
•Created by aschwartz on 12/13/2023 in #questions
Reindexing using the Mgmt System
Hi @aschwartz if you use the first approach then your queries which are eligible for that index usage will not run as expected.
After you enable the index and the reindexing is not yet done, then all the queries which are eligible for that index will try to use the index which is non existent resulting in empty result.
While in the second approach the index won't be eligible for usage until the reindexing job is completed after which it will automatically move to enabled state.
https://docs.janusgraph.org/schema/index-management/index-lifecycle/#:~:text=.commit()%3B-,Index%20states%20and%20transitions,-The%20diagram%20below
4 replies
JJanusGraph
•Created by shivam.choudhary on 8/14/2023 in #questions
Impact of ID Pool Initialisation on Query Performance
Hi Everyone, we are still facing the above issue, any help will be greatly appreciated. TIA.
5 replies
JJanusGraph
•Created by 4j4y. on 8/29/2023 in #questions
Does the quintillion edges limitation stil exists after introducing custom vertex id support?
Also sometime we have to restart due to the following reasons:
1. Mapping additional props to vertex/edge label.
2. Changing state of newly created index from INSTALLED to REGISTERED.
In both the cases changes does not reflect by itself unless we restart, we tried waiting for hours for changes to be communicated to all the instances via backend but only after redeployment it happens.
7 replies
JJanusGraph
•Created by 4j4y. on 8/29/2023 in #questions
Does the quintillion edges limitation stil exists after introducing custom vertex id support?
I’m not sure how you calculate ids exhaustion.@porunov Most of our ids got wasted due to server restarts as we have Janusgraph server running on kubernetes pods (~ 20-30 pods). Also, we have set
cluster.max-partitions to 1024 which we didn't know at the start that will reserve 10 bits of id, giving 2^50 ids for edges and 2^49 for vertices.
Initially we ingested a lot of data into the graph , block-size was set to 1million and recently while going through the Janusgraph code we found that Janusgraph id pool for edge namespace has a block size 8 times of base block size which equates to 8million per id pool.
Now one deployment costs us 8m * (1024 id pools) * (~ 25 pods) = 204 billion edges, we've had a lot of deployments in the starting phase. we are planning to move Janusgraph server to VMs and reduce the block size, but unfortunately we cannot change cluster.max-partitions as it is a fixed config.
Please let us know if our understading is wrong and any more step we can take to reduce the wastage of ids.7 replies
JJanusGraph
•Created by shivam.choudhary on 6/27/2023 in #questions
JanusGraph metrics data having value 0 for most metrics
11 replies
JJanusGraph
•Created by shivam.choudhary on 6/27/2023 in #questions
JanusGraph metrics data having value 0 for most metrics
I'm using
jmx_prometheus_javaagent to expose the metrics which are then scraped using ServiceMonitor from all the janusgraph instances and pushed to prometheus and then using grafana to access these metrics.
Some of the metrics which are 0:
Can see that some metrics are having data also, for example:
11 replies
JJanusGraph
•Created by shivam.choudhary on 6/27/2023 in #questions
JanusGraph metrics data having value 0 for most metrics
All these requests have queries which fetches data from the backend storage which in this case in BigTable
11 replies
JJanusGraph
•Created by shivam.choudhary on 6/27/2023 in #questions
JanusGraph metrics data having value 0 for most metrics
11 replies
JJanusGraph
•Created by shivam.choudhary on 5/31/2023 in #questions
Reading the writes on Instance A done by Instance B
Yes, I checked it is set to false.
5 replies