CQL query timeout issues
I am trying to create an edge between two vertices, where one of the vertex contains more than 450K edges.
But everytime the operation fails with Query Timeout error as shown below:
We went through the documentation and found that we can increase the threshold by increasing the default value for "storage.cql.request-timeout".
But what if my edge count exceeds even beyond 800K?, I cannot keep increasing the threshold forever.
My understanding is, as it is trying to create a lock on the edges to create another edge, is there any other better way to do it? that is, something like lazy loading where the graph do not query the entire edges and just create the new one instead.
3 Replies
@Bo @Oleksandr Porunov @Florian Hockmann @Natalia
Your help will be appreciated here 🙂
I don't know your graph configuration, tx configuration, and the queries performed, but I assume it could be related to the Super-Node problem which Boxuan was working here: https://github.com/JanusGraph/janusgraph/pull/4420
GitHub
Super Node Series One: Introduce proxy node type by li-boxuan · Pul...
This is a first attempt towards solving a famous problem in graph databases world - super node. A brief introduction to super node problem, and the status quo of JanusGraph, is documented at #2717....
Hi @HailDevil ! I think having more context might be helpful. Are you using JanusGraphs APIs directly to operate with the edges or you are using Gremlin sever? From my experience, we have some usecases in PROD with 1 million edges between verticies and we don't observe timeouts.
Normally, during edge addition, JanusGraph doesn't read all edges back. Maybe worth checking your Cassandra consistency level, if it cross DC QUORUM, that might be expensive to get a lock when edge is being added.
To experiment and check if this is due to lock, you can turn this configuration on:
storage.cql.use-external-locking=true
if this helps with insertions, might be worth looking into that more.
Also, if the latencies come from reading back the edges, sometimes building a vertex centric index helps: https://docs.janusgraph.org/schema/index-management/index-performance/