Apache TinkerPop•11mo ago

G.V() IDE can't visualize path().by(valueMap()) query

Hi @G.V() - Gremlin IDE (Arthur) sorry if this is a duplicate question. I am playing around with G.V() IDE and run into a problem: if I run a path() query, G.V() IDE could properly display the nodes and the edges, but since a path query itself does not return properties, G.V cannot display the properties. If I add .by(valueMap()) to the end of the query, then the Gremlin query result would include both the path and properties, but G.V IDE cannot visualize it. Is this a known problem? Thx!

28 Replies

gdotv•11mo ago

hey, that's odd, the way G.V() works it should just automatically fetch properties on elements returned by queries regardless of the structure they're wrapped in (hashmap, path, list, etc), either by leveraging Tinkerpop's 3.7.0 bundling of props with elements or for tinkerpop < 3.7.0 by fetching the properties "in post" and affixing them to the elements. are you able to share a query perhaps just to check? also im assuming this is happening on a G.V() playground? the .valueMap() by modulator isn't showing a graph because technically valueMap isn't a vertex or an edge, and so G.V() doesn't know to offer a graph view for those so that part is expected behaviour, but properties not being included in your element in a normal path query that contains edges/vertices definitely aint rigth

BoOP•11mo ago

Interesting! I am not using playground - I am using a local TinkerGraph instance: The query is fairly simple:

g.V("1060.1").out().
  limit(1).
  path()

g.V("1060.1").out().
  limit(1).
  path()

I am using TinkerGraph 3.7.0. I do remember it worked for me in the past. Maybe I was using a different TinkerGraph version or G.V version back then. Let me try the latest ones.

gdotv•11mo ago

Yeah that should work. Any chance your tinkergraph is 3.7.0 but not allowing properties to be included with elements? I know that behavior can be configured, can't remember the setting's name off hand

BoOP•11mo ago

If I run g.V("1060.1").out().limit(1), then I can see vertex properties I just tried with TinkerGraph 3.7.1 but no luck My TinkerGraph configurations are fairly simple:

gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
gremlin.tinkergraph.vertexIdManager=ANY

gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
gremlin.tinkergraph.vertexIdManager=ANY

I deleted the existing connection and established a new connection. It works now! So it seems to be there's something off with GdotV

gdotv•11mo ago

hey, weird, i wonder what went wrong there, ill investigate on my end, see if i can spot anything weird

BoOP•11mo ago

I am able to reproduce the problem. Every time I create a new connection,

g.V("1060.1").out().
  limit(1).
  path()

g.V("1060.1").out().
  limit(1).
  path()

shows the vertex properties ONLY for the 1st time. If I run the same query again, I get "no properties available" in the IDE.

gdotv•11mo ago

okay, that's definitely weird, lemme see if i can reproduce this with a standard gremlin server is it only occurring for path queries on any queries returning elements? also thanks for the additional context, this is helpful

BoOP•11mo ago

It's only for path queries If I remove path(), it shows the vertex properties

gdotv•11mo ago

wow, that's super specific, i guess that's how it slipped through the cracks, really interesting in the query editor on G.V(), it should show on the bottom left what the detected gremlin version is, is that showing as 3.7.1?

BoOP•11mo ago

yes, 3.7.1

gdotv•11mo ago

weird, so the version is definitely getting correctly detected

spmallette•11mo ago

was there ever a solution to this one? just a bug @G.V() - Gremlin IDE (Arthur) ?

gdotv•11mo ago

Haven't managed to reproduce the issue so unfortunately not at this time, though i have it tracked internally, could probably just close this and ill follow up directly with Boxuan if/when i get it fixed

BoOP•11mo ago

I created a playground and wasn't able to reproduce either. I can only reproduce it using a standalone tinkergraph server.

gdotv•10mo ago

Hey, bumping this with some recent-ish findings that may provide some further explanation, though I'm still unclear on the exact issue and whether it's TinkerPop related or not. I've seen one separate report on a different tinkerpop database provider using Gremlin >= 3.7.0 where the inVertex/outVertex fields of the Edge object had an empty "properties" field, which effectively resulted in what Boxuan initially reported of properties not showing in G.V() with that provider the issue occurred only under certain queries using the bothE, outE, inE and path steps. I don't know whether it's a framework level issue or provider specific, but I find it suspicious now having heard reports of it for 2 different providers. I wonder if it's at all possible under some circumstances the in/out vertex fields in Edge objects are missing properties when Edge result objects are returned @spmallette i think there's potential a gremlin-server bug here, ive now heard multiple reports from various implementations of the same issue

Lyndon•10mo ago

Arthur mentioned this to me. I tried this very simple test with TinkerGraph behind gremlin-server.

        final Cluster cluster = Cluster.build().addContactPoint("localhost").port(8182).create();
        final DriverRemoteConnection drc = DriverRemoteConnection.using(cluster, "g");
        final GraphTraversalSource g = traversal().withRemote(drc);
        g.V().drop().iterate();
        Vertex foo = g.addV("Foo").property("has", "bar").next();
        Vertex bar = g.addV("Bar").property("has", "baz").next();
        Edge knows = g.V(foo).addE("knows").to(bar).property("hi", "lo").next();
        List<Edge> edges1 = g.V(foo.id()).outE().toList();
        List<Edge> edges2 = g.V(bar.id()).inE().toList();
        Edge edges3 = g.E(knows.id()).next();

        final Cluster cluster = Cluster.build().addContactPoint("localhost").port(8182).create();
        final DriverRemoteConnection drc = DriverRemoteConnection.using(cluster, "g");
        final GraphTraversalSource g = traversal().withRemote(drc);
        g.V().drop().iterate();
        Vertex foo = g.addV("Foo").property("has", "bar").next();
        Vertex bar = g.addV("Bar").property("has", "baz").next();
        Edge knows = g.V(foo).addE("knows").to(bar).property("hi", "lo").next();
        List<Edge> edges1 = g.V(foo.id()).outE().toList();
        List<Edge> edges2 = g.V(bar.id()).inE().toList();
        Edge edges3 = g.E(knows.id()).next();

What I see is that the inVertex/outVertex properties of edges1, edges2, and edges3 are all not populated. With 3.7.0 do we expect the vertices attached to edges to have their properties populated @spmallette? I read the docs but it wasn't clear to me about this specific case.

spmallette•10mo ago

for an edge it is just the id/label of the inV/outV. i think writing properties is meant to only apply to the main element being serialized, not ones incident to the element. like a VertexProperty doesn't have the parent Vertex with all its properties serialized with it (we might have even skipped even the id/label in that case) - @Valentyn Kahamlyk is the VertexProperty issue i just mentioned something to reconsider in terms of consitency in serialization?

Valentyn Kahamlyk•10mo ago

Long story short: I didn't touch serialization of Path, https://github.com/apache/tinkerpop/blob/55a7b7f25677c38ad23a813d19dfb58d69954d35/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONSerializersV3.java#L284

GitHub

tinkerpop/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/s...

Apache TinkerPop - a graph computing framework. Contribute to apache/tinkerpop development by creating an account on GitHub.

spmallette•10mo ago

i think writing properties is meant to only apply to the main element being serialized, not ones incident to the element.

i guess Path falls under that rule, though i'm not sure it holds consistently like for Graph though it may hold for Tree. Graph ~~may need to be~~ is the exception, which seems sane to me

gdotv•10mo ago

If a path were to return the raw elements (vertex/edge), would it make sense then to include properties too?

spmallette•10mo ago

i think the the "element" being serialized there is the Path and that what is contained in the Path is incident to it, so i'd say "no" based on that thinking. to me Graphis the only exception to that rule. i suppose i could be convinced otherwise, which is why i ping'd @Valentyn Kahamlyk since he's thinking about such things right now.

gdotv•10mo ago

i see, I think this feels a bit inconsistent in terms of behaviour, i would expect serialization to behave the same for a given type of element regardless of context given that the data type it's contained within doesn't really exclude it from being serialized, i think in this case we're just saying it's getting serialized differently i was just checking the docs as well and couldn't find any reference to this behaviour (checked https://tinkerpop.apache.org/docs/current/reference/#_properties_of_elements specifically) in terms of use case it's been causing issues with some G.V() users running path queries and not getting properties fetched as part of the downstream graph visualisation it produces - obviously i can alter the way G.V() behave to try and identify elements that don't include properties because they're within a path, but it feels like unnecessary complexity for this behaviour

spmallette•10mo ago

i understand it feels inconsistent and it's bad that we don't document this well. here's what i tend to think about:

gremlin> g.V().outE().inV().path()
==>[v[1],e[9][1-created->3],v[3]]
==>[v[1],e[7][1-knows->2],v[2]]
==>[v[1],e[8][1-knows->4],v[4]]
==>[v[4],e[10][4-created->5],v[5]]
==>[v[4],e[11][4-created->3],v[3]]
==>[v[6],e[12][6-created->3],v[3]]

gremlin> g.V().outE().inV().path()
==>[v[1],e[9][1-created->3],v[3]]
==>[v[1],e[7][1-knows->2],v[2]]
==>[v[1],e[8][1-knows->4],v[4]]
==>[v[4],e[10][4-created->5],v[5]]
==>[v[4],e[11][4-created->3],v[3]]
==>[v[6],e[12][6-created->3],v[3]]

in this really simple example for a Path object, each one would have the Vertex serialized twice. It seems like a costly burden, especially if those vertices have a lot of properties. Also, if we're serializing incident vertices on an Edge, is it inconsistent to not also serialize incident edges on vertices? aside from the serialization costs, what a strange circular place that would be if we included that. Similary, should a VertexProperty serialize a parent Vertex which would then reserialize the same VertexProperty? What happens when we have EdgeProperty (i.e. meta properties on edges) ? My general feeling right now is that a Path and a Tree are containers like a Graph and in that sense should probably contain associated properties for contained elements. On the other hand, I feel like an Element should have an exception and treat references to other elements as references and only include id/label. That seems like a clean distinction and avoids circular, repetitive serialization. Again, just sharing some thoughts here and not a decision. I'd like to hear what others have to say.

gdotv•10mo ago

i get your point now, it's definitely not an easy line to draw - i think what made me feel path would be okay to have these elements serialized is that the Path object is effectively not dissimilar from a simple array. That being said now that I understand the logic it's something I can easily work around - it's very hard to tell how many would actually be impacted by this either way - I suspect not many!

Valentyn Kahamlyk•10mo ago

I agree that it makes sense to add properties on elements to containers like Path, Tree and Graph. We can skip properties by default , send only with explicit with('materializeProperties', 'all'), but this will add more chaos and confusion. Or maybe introduce other options like with('materializeProperties', 'top') or with('materializeProperties', 'nested'). @G.V() - Gremlin IDE (Arthur) what do you think is the best from your point of view as a user of this function?

gdotv•10mo ago

I think at this stage changing the serialization behavior from implicit to explicit would cause a lot of problems. I spotted this materializeProperties and gotta say it's a great option to have as an opt out. I reckon an all or nothing would do, it seems highly unlikely anyone would find much value in an in between behavior. Selfishly of course trees, graphs and paths being treated the same as arrays or hashmaps in terms of serialization would be ideal for G.V(). I suspect it would make serialization more consistent in the sense that these objects are essentially data structures first, not classes like a vertex property might be

spmallette•10mo ago

i'd suggested a more complex approach to configuring property serialization and i'm glad i was convinced that we just went with the all or nothing approach in "materializeProperties". I think we should stick with that for now and just have clear definition for where they should be expected and where they should not. the more i think about my thoughts around "containers" and "elements" the more i like what i proposed there. hopefully that can get implemented and straightened out soon once and for all.

gdotv•10mo ago

Quick update on my end, the upcoming version of gdotv (should be out next week) will include a fix that sorts this out, irregarding of how this is taken forward in apache Tinkerpop

Gaming

Programming

G.V() IDE can't visualize path().by(valueMap()) query