Apache TinkerPop•2y ago

Can I use a query to export data in the form of a query?

In the SQL world you can use SQL Developer to generate INSERT statements from the results of a query. I want to sample some of our Neptune data for use in a unit test that uses Tinkergraph - is it possible to capture the results of a query as something like g.addV() queries that allow me to export my real data into Tinkergraph?

10 Replies

danielcraig23OP•2y ago

After writing my question I remembered that there is the .io() traversal step which I can use to create a .json file with the results of my query, making them easily accessible to TinkerGraph for use in a unit test

spmallette•2y ago

Neptune doesn't support io() for writes so I don' t think it will help you there. In any event, I think it would be easiest to use subgraph() step for what you describe. It's designed to return a TinkerGraph from your Gremlin query. https://tinkerpop.apache.org/docs/current/reference/#subgraph-step

danielcraig23OP•2y ago

Maybe I should try Aerospike instead for this use case

triggan•2y ago

Neptune actually does support the use of the io() step for reading, just not for writing. https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-gremlin-differences.html#feature-gremlin-differences-unsupported-steps But I agree with with Stephen, subgraph() is really the best way to return a subset of a graph into a TinkerGraph object. This only really works in Java, though.

Gremlin standards compliance in Amazon Neptune - Amazon Neptune

Overview of differences between the Neptune and TinkerPop implementations of Gremlin.

spmallette•2y ago

If you aren't using Java and need a subgraph, you can use path() to sort of simulate that functionality: https://stephen.genoprime.com/snippet/2020/08/01/snippet-12.html

stephen mallette

Subgraphing without subgraph()

Subgraphing is a common use case when working with graphs. We often find ourselves wanting to take some small portion of a graph and then operate only upon it. Gremlin provides subgraph() step, which helps to make this operation relatively easy by exposing a way to produce an edge-induced subgraph that is detached from the parent graph.

danielcraig23OP•2y ago

But Tinkergraph supports io() for writes correct? I misread what you said earlier, so I think I can pursue this path if Neptune supports io() for reading and tinkergraph supports io() for writing. This is a 1 time thing that I want to do, just to establish a body of sample vertices and edges that I can unit test my queries against

spmallette•2y ago

TinkerGraph supports both. Neptune only supports reads.

danielcraig23OP•2y ago

Ok thanks! This is what I came up with:

final Graph graph = (Graph) g.V().hasLabel("exampleLabel")
        .union(__.outE("exampleEdgeLabel").otherV(), 
                __.outE("exampleEdgeLabel").otherV().outE("exampleEdgeLabel2").otherV(), 
                __.outE("exampleEdgeLabel").otherV().inE("exampleEdgeLabel3").otherV(),
        )
        .path()
        .unfold()
        .hasLabel("exampleEdgeLabel", "exampleEdgeLabel2", "exampleEdgeLabel3")
        .subgraph("subGraph")
        .cap("subGraph")
        .next();

GraphTraversalSource gts = AnonymousTraversalSource.traversal().withEmbedded(graph);
gts.io("src/test/resources/testGraph.json").write().iterate();

final Graph graph = (Graph) g.V().hasLabel("exampleLabel")
        .union(__.outE("exampleEdgeLabel").otherV(), 
                __.outE("exampleEdgeLabel").otherV().outE("exampleEdgeLabel2").otherV(), 
                __.outE("exampleEdgeLabel").otherV().inE("exampleEdgeLabel3").otherV(),
        )
        .path()
        .unfold()
        .hasLabel("exampleEdgeLabel", "exampleEdgeLabel2", "exampleEdgeLabel3")
        .subgraph("subGraph")
        .cap("subGraph")
        .next();

GraphTraversalSource gts = AnonymousTraversalSource.traversal().withEmbedded(graph);
gts.io("src/test/resources/testGraph.json").write().iterate();

spmallette•2y ago

cool. i don't think it's a big issue for Neptune, but some graphs perform better when you avoid otherV() when you know the direction of the edge. in general, it is considered good form to only use otherV() with bothE(). that said, i wonder if that can't be simplfied a bit? is this the same thing:

g.V().hasLabel("exampleLabel").
  outE("exampleEdgeLabel").subgraph('sg').inV().
  union(outE("exampleEdgeLabel2"),inE("exampleEdgeLabel3")).subgraph('sg').
  cap('sg')

g.V().hasLabel("exampleLabel").
  outE("exampleEdgeLabel").subgraph('sg').inV().
  union(outE("exampleEdgeLabel2"),inE("exampleEdgeLabel3")).subgraph('sg').
  cap('sg')

danielcraig23OP•2y ago

Oh that's great I love this simplification!

Gaming

Programming

Can I use a query to export data in the form of a query?