AT
Apache TinkerPop•2y ago
ZEE

Creating Indexes In JanusGraph

How do you create indices in JanusGraph and/or Gremlin with Java? JanusGraph still isn't using the index provided. 
2023-05-15 14:35:59,723 [INFO] [o.j.g.d.m.ManagementSystem.Thread-25] ::     Index update job successful for [_id]
2023-05-15 14:35:59,724 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 14:35:59,726 [INFO] [Main.main] ::     REINDEX g.V().hasLabel("entity").count().next():    1
2023-05-15 14:35:59,737 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[_id <> null]]. For better performance, use indexes
2023-05-15 14:35:59,738 [INFO] [Main.main] ::     REINDEX g.V().hasLabel("entity").count().next():    1
2023-05-15 14:35:59,739 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity, _id <> null]]. For better performance, use indexes
2023-05-15 14:35:59,740 [INFO] [Main.main] ::     REINDEX g.V().hasLabel("entity").count().next():    1
2023-05-15 14:35:59,723 [INFO] [o.j.g.d.m.ManagementSystem.Thread-25] ::     Index update job successful for [_id]
2023-05-15 14:35:59,724 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 14:35:59,726 [INFO] [Main.main] ::     REINDEX g.V().hasLabel("entity").count().next():    1
2023-05-15 14:35:59,737 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[_id <> null]]. For better performance, use indexes
2023-05-15 14:35:59,738 [INFO] [Main.main] ::     REINDEX g.V().hasLabel("entity").count().next():    1
2023-05-15 14:35:59,739 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity, _id <> null]]. For better performance, use indexes
2023-05-15 14:35:59,740 [INFO] [Main.main] ::     REINDEX g.V().hasLabel("entity").count().next():    1
        JanusGraph janusGraph = JanusGraphFactory.build().set("storage.backend", "inmemory").open();
        GraphTraversalSource g = janusGraph.traversal();
...   janusGraphManagement.updateIndex(janusGraphManagement.getGraphIndex("_id"), SchemaAction.REINDEX).get();
        janusGraphManagement.commit();
        logger.info("REINDEX g.V().hasLabel(\"entity\").count().next():\t" + g.V().hasLabel("entity").count().next());
        logger.info("REINDEX g.V().hasLabel(\"entity\").count().next():\t" + g.V().has("_id").count().next());
        logger.info("REINDEX g.V().hasLabel(\"entity\").count().next():\t" + g.V().hasLabel("entity").has("_id").count().next());
        JanusGraph janusGraph = JanusGraphFactory.build().set("storage.backend", "inmemory").open();
        GraphTraversalSource g = janusGraph.traversal();
...   janusGraphManagement.updateIndex(janusGraphManagement.getGraphIndex("_id"), SchemaAction.REINDEX).get();
        janusGraphManagement.commit();
        logger.info("REINDEX g.V().hasLabel(\"entity\").count().next():\t" + g.V().hasLabel("entity").count().next());
        logger.info("REINDEX g.V().hasLabel(\"entity\").count().next():\t" + g.V().has("_id").count().next());
        logger.info("REINDEX g.V().hasLabel(\"entity\").count().next():\t" + g.V().hasLabel("entity").has("_id").count().next());
10 Replies
spmallette
spmallette•2y ago
there's lots of information about this message in mailing lists, stackoverflow and other resources, but the one that comes to me readily is the one i think i pointed you at already: https://stackoverflow.com/a/49966807/1831717 you simply aren't using the index when you do hasLabel() because, as good ol' Jason Plurad wrote in that post, your query:
requires a full vertex scan because JanusGraph does not currently allow you to create an index on vertex label.
For an index to be used you can't just filter on T.label. You need to include a property key and value as well for the index to be engaged.
Stack Overflow
Janusgraph - WARNING about iterating over all vertices after schema...
I am using JanusGraph with Cassandra and ElasticSearch backends. I have used the following script to create my schema and indexes. // Create a Janus Graph instance, according to the configuration...
ZEE
ZEEOP•2y ago
Thank you
spmallette
spmallette•2y ago
have you given it a try? like, do: g.V().has('entity', "_id", 123) - does the warning go away?
ZEE
ZEEOP•2y ago
inmemory doesn't get the issue. But switching it back to just JanusGraphFactory.build().set("storage.backend", "cql").set("storage.hostname", "localhost:9042").open(); breaks it all over again
spmallette
spmallette•2y ago
i believe different backends have different properties, so i think that could be expected
ZEE
ZEEOP•2y ago
2023-05-15 15:46:12,852 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 15:46:12,853 [INFO] [Main.main] ::     g.V().hasLabel("entity").count().next():    1
2023-05-15 15:46:12,867 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[_id <> null]]. For better performance, use indexes
2023-05-15 15:46:12,869 [INFO] [Main.main] ::     g.V().has("_id").count().next();:    1
2023-05-15 15:46:12,871 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity, _id <> null]]. For better performance, use indexes
2023-05-15 15:46:12,871 [INFO] [Main.main] ::     g.V().hasLabel("entity").has("_id").count().next():    1
2023-05-15 15:46:12,873 [INFO] [Main.main] ::     g.V().hasLabel("entity").has("_id", "Test1").count().next():    1
2023-05-15 15:46:12,852 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 15:46:12,853 [INFO] [Main.main] ::     g.V().hasLabel("entity").count().next():    1
2023-05-15 15:46:12,867 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[_id <> null]]. For better performance, use indexes
2023-05-15 15:46:12,869 [INFO] [Main.main] ::     g.V().has("_id").count().next();:    1
2023-05-15 15:46:12,871 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity, _id <> null]]. For better performance, use indexes
2023-05-15 15:46:12,871 [INFO] [Main.main] ::     g.V().hasLabel("entity").has("_id").count().next():    1
2023-05-15 15:46:12,873 [INFO] [Main.main] ::     g.V().hasLabel("entity").has("_id", "Test1").count().next():    1
spmallette
spmallette•2y ago
yeah...so, it's basically working as expected
ZEE
ZEEOP•2y ago
JanusGraph janusGraph = JanusGraphFactory.build().set("storage.backend", "inmemory").open();
JanusGraph janusGraph = JanusGraphFactory.build().set("storage.backend", "inmemory").open();
JanusGraph janusGraph = JanusGraphFactory.build().set("storage.backend", "cql").set("storage.hostname", "localhost:9042").open();
JanusGraph janusGraph = JanusGraphFactory.build().set("storage.backend", "cql").set("storage.hostname", "localhost:9042").open();
2023-05-15 15:47:43,206 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-15 15:47:43,231 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 15:47:43,236 [INFO] [Main.main] ::     drop g.V().hasLabel("entity").count().next():    0
2023-05-15 15:47:43,287 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 15:47:43,291 [INFO] [Main.main] ::     REGISTER_INDEX g.V().hasLabel("entity").count().next():    0
2023-05-15 15:47:43,295 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 15:47:43,298 [INFO] [Main.main] ::     updateIndex g.V().hasLabel("entity").count().next():    0
Exception in thread "main" java.lang.IllegalArgumentException: Update action [ENABLE_INDEX] cannot be invoked for index with status [INSTALLED]
    at org.janusgraph.core.schema.SchemaAction.isApplicableStatus(SchemaAction.java:85)
    at org.janusgraph.graphdb.database.management.ManagementSystem.updateIndex(ManagementSystem.java:864)
    at org.janusgraph.graphdb.database.management.ManagementSystem.updateIndex(ManagementSystem.java:845)
    at Test17.main(Test17.java:32)
2023-05-15 15:47:43,206 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-15 15:47:43,231 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 15:47:43,236 [INFO] [Main.main] ::     drop g.V().hasLabel("entity").count().next():    0
2023-05-15 15:47:43,287 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 15:47:43,291 [INFO] [Main.main] ::     REGISTER_INDEX g.V().hasLabel("entity").count().next():    0
2023-05-15 15:47:43,295 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::     Query requires iterating over all vertices [[~label = entity]]. For better performance, use indexes
2023-05-15 15:47:43,298 [INFO] [Main.main] ::     updateIndex g.V().hasLabel("entity").count().next():    0
Exception in thread "main" java.lang.IllegalArgumentException: Update action [ENABLE_INDEX] cannot be invoked for index with status [INSTALLED]
    at org.janusgraph.core.schema.SchemaAction.isApplicableStatus(SchemaAction.java:85)
    at org.janusgraph.graphdb.database.management.ManagementSystem.updateIndex(ManagementSystem.java:864)
    at org.janusgraph.graphdb.database.management.ManagementSystem.updateIndex(ManagementSystem.java:845)
    at Test17.main(Test17.java:32)
JanusGraph blows up when I just switch the JanusGraphFactory
spmallette
spmallette•2y ago
this is a separate issue though from what you were asking about though right? do you understand why you are getting this warning now: "Query requires iterating over all vertices [[~label = entity, _id <> null]]. For better performance, use indexes"?e
ZEE
ZEEOP•2y ago
Yup! Thanks
Want results from more Discord servers?
Add your server