JanusGraph•14mo ago

Unable to use next() in gremlin-python

Hi, has anyone tried to use gremlin-python with janusgraph 1.0.0? It seems that there is a bug that makes next() unusable. Here is an example of how to reproduce the issue:

# Python 3.10.12
# janusgraph 1.0.0
# gremlinpython==3.7.0

from gremlin_python.process.anonymous_traversal import traversal
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection

connection = DriverRemoteConnection("ws://jgserver:8182/gremlin", "g")
g = traversal().withRemote(connection)

g.V().drop().iterate()
g.E().drop().iterate()
v1 = g.addV('vertex').property('name', 'v1').next()

# Python 3.10.12
# janusgraph 1.0.0
# gremlinpython==3.7.0

from gremlin_python.process.anonymous_traversal import traversal
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection

connection = DriverRemoteConnection("ws://jgserver:8182/gremlin", "g")
g = traversal().withRemote(connection)

g.V().drop().iterate()
g.E().drop().iterate()
v1 = g.addV('vertex').property('name', 'v1').next()

This generates a KeyError:

[...]

~/.local/lib/python3.10/site-packages/gremlin_python/structure/io/graphbinaryV1.py in read_object(self, b)
    192             return self.to_object(io.BytesIO(b))
    193         elif isinstance(b, io.BufferedIOBase):
--> 194             return self.to_object(b)
    195 
    196     def to_object(self, buff, data_type=None, nullable=True):

~/.local/lib/python3.10/site-packages/gremlin_python/structure/io/graphbinaryV1.py in to_object(self, buff, data_type, nullable)
    201                     buff.read(1)
    202                 return None
--> 203             return self.deserializers[DataType(bt)].objectify(buff, self, nullable)
    204         else:
    205             return self.deserializers[data_type].objectify(buff, self, nullable)

KeyError: <DataType.custom: 0>

[...]

~/.local/lib/python3.10/site-packages/gremlin_python/structure/io/graphbinaryV1.py in read_object(self, b)
    192             return self.to_object(io.BytesIO(b))
    193         elif isinstance(b, io.BufferedIOBase):
--> 194             return self.to_object(b)
    195 
    196     def to_object(self, buff, data_type=None, nullable=True):

~/.local/lib/python3.10/site-packages/gremlin_python/structure/io/graphbinaryV1.py in to_object(self, buff, data_type, nullable)
    201                     buff.read(1)
    202                 return None
--> 203             return self.deserializers[DataType(bt)].objectify(buff, self, nullable)
    204         else:
    205             return self.deserializers[data_type].objectify(buff, self, nullable)

KeyError: <DataType.custom: 0>

Using g.addV('vertex').property('name', 'v1').iterate() instead of next() works fine. Any thoughts?

Solution:

This is my config https://github.com/Citegraph/citegraph/blob/main/backend/src/main/resources/gremlin-server-cql.yaml. I just tested python driver and java driver and they both worked well.

Jump to solution

10 Replies

rpugaOP•14mo ago

Also, g.V().count().next() works, but g.V().next() yields the same KeyError

Bo•14mo ago

Try using GraphSONMessageSerializerV3 in your server config, e.g.

{ className: org.apache.tinkerpop.gremlin.util.ser.GraphSONMessageSerializerV3, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}

rpugaOP•14mo ago

Thanks for the suggestion. Unfortunately, after setting the serializer to GraphSONMessageSerializerV3 I get the following error 😦

GremlinServerError: 599: Error during serialization: Serializer for type org.janusgraph.graphdb.relations.RelationIdentifier not found

GremlinServerError: 599: Error during serialization: Serializer for type org.janusgraph.graphdb.relations.RelationIdentifier not found

Bo•14mo ago

Did you include config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }? If you are still facing the error, could you please share your server config yaml file?

rpugaOP•14mo ago

Hi, yes, here is the yaml file is attached. And here is the error:

[...]
  File "/var/lib/janusgraph/.local/lib/python3.10/site-packages/gremlin_python/driver/connection.py", line 90, in _receive
    status_code = self._protocol.data_received(data, self._results)
  File "/var/lib/janusgraph/.local/lib/python3.10/site-packages/gremlin_python/driver/protocol.py", line 138, in data_received
    raise GremlinServerError(message['status'])
gremlin_python.driver.protocol.GremlinServerError: 599: Error during serialization: Serializer for type org.janusgraph.graphdb.relations.RelationIdentifier not found

[...]
  File "/var/lib/janusgraph/.local/lib/python3.10/site-packages/gremlin_python/driver/connection.py", line 90, in _receive
    status_code = self._protocol.data_received(data, self._results)
  File "/var/lib/janusgraph/.local/lib/python3.10/site-packages/gremlin_python/driver/protocol.py", line 138, in data_received
    raise GremlinServerError(message['status'])
gremlin_python.driver.protocol.GremlinServerError: 599: Error during serialization: Serializer for type org.janusgraph.graphdb.relations.RelationIdentifier not found

BTW: When I enable the GraphSONMessageSerializerV3 serializer in the yaml config file, I also get the same error from a Java app that connects to JanusGraph (my Java app works fine with the default serializer):

java.util.concurrent.CompletionException: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Error during serialization: Serializer for type org.janusgraph.graphdb.relations.RelationIdentifier not found
        at java.base/java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:412)
        at java.base/java.util.concurrent.CompletableFuture.join(CompletableFuture.java:2044)
        at org.apache.tinkerpop.gremlin.driver.ResultSet.one(ResultSet.java:123)
        at org.apache.tinkerpop.gremlin.driver.ResultSet$1.hasNext(ResultSet.java:175)
        at com.example.GremlinJavaExample.main(GremlinJavaExample.java:49)

java.util.concurrent.CompletionException: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Error during serialization: Serializer for type org.janusgraph.graphdb.relations.RelationIdentifier not found
        at java.base/java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:412)
        at java.base/java.util.concurrent.CompletableFuture.join(CompletableFuture.java:2044)
        at org.apache.tinkerpop.gremlin.driver.ResultSet.one(ResultSet.java:123)
        at org.apache.tinkerpop.gremlin.driver.ResultSet$1.hasNext(ResultSet.java:175)
        at com.example.GremlinJavaExample.main(GremlinJavaExample.java:49)

janusgraph-server.ya...

Solution

Bo•14mo ago

This is my config https://github.com/Citegraph/citegraph/blob/main/backend/src/main/resources/gremlin-server-cql.yaml. I just tested python driver and java driver and they both worked well.

Bo•14mo ago

In your java driver case, when you built

Cluster cluster = Cluster.build("localhost")
  .port(8182)
  .serializer(createGraphBinaryMessageSerializerV1())
  .create();

private MessageSerializer createGraphBinaryMessageSerializerV1() {
  final GraphBinaryMessageSerializerV1 serializer = new GraphBinaryMessageSerializerV1();
  final Map<String, Object> config = new HashMap<>();
  config.put(TOKEN_IO_REGISTRIES, Collections.singletonList(JanusGraphIoRegistry.class.getName()));
  serializer.configure(config, Collections.emptyMap());
  return serializer;
}

Cluster cluster = Cluster.build("localhost")
  .port(8182)
  .serializer(createGraphBinaryMessageSerializerV1())
  .create();

private MessageSerializer createGraphBinaryMessageSerializerV1() {
  final GraphBinaryMessageSerializerV1 serializer = new GraphBinaryMessageSerializerV1();
  final Map<String, Object> config = new HashMap<>();
  config.put(TOKEN_IO_REGISTRIES, Collections.singletonList(JanusGraphIoRegistry.class.getName()));
  serializer.configure(config, Collections.emptyMap());
  return serializer;
}

Make sure it includes the serializer that your server is using. In the above example, it would only work if your server enables GraphBinaryMessageSerializerV1

rpugaOP•14mo ago

Thank you for the examples, they are very useful! I used your .yaml file, and now gremlin-python works. Earlier, I forgot to specify the serializer. I added the following and now it works OK:

message_serializer = serializer.GraphSONSerializersV3d0()
connection = DriverRemoteConnection("ws://jgserver:8182/gremlin", "g", message_serializer=message_serializer)

message_serializer = serializer.GraphSONSerializersV3d0()
connection = DriverRemoteConnection("ws://jgserver:8182/gremlin", "g", message_serializer=message_serializer)

I'll try your solution for the Java code as well. Thanks! Update: the Java solution also worked, thank you very much!

Bo•14mo ago

message_serializer = serializer.GraphSONSerializersV3d0()
connection = DriverRemoteConnection("ws://jgserver:8182/gremlin", "g", message_serializer=message_serializer)

message_serializer = serializer.GraphSONSerializersV3d0()
connection = DriverRemoteConnection("ws://jgserver:8182/gremlin", "g", message_serializer=message_serializer)

Interesting though, coz I didn't have to specify the message serializer when using python. I thought the python driver would somehow figure it out by itself.

cdegroc•14mo ago

My 2🪙: I think it just defaults to GraphSONMessageSerializer here: https://github.com/apache/tinkerpop/blob/master/gremlin-python/src/main/python/gremlin_python/driver/driver_remote_connection.py#L65-L68

Gaming

Programming

Unable to use next() in gremlin-python