Apache TinkerPop•16mo ago

Trying to run a local version for a test, what is the correct serializer?

Windows machine, local host. I can't find the Running this, https://github.com/bricaud/gremlin-server but updated the version to apache-tinkerpop-gremlin-server-3.7.0 I had issue with the serializers: section of the gremlin-conf.yaml so I commented out to see if the defaults would work. The docker image loads fine, but then I can't seem to connect to it. My simple test:

print('Number of nodes {}, number of edges {}.'.format(g.V().count().next(),g.E().count().next()))

print('Number of nodes {}, number of edges {}.'.format(g.V().count().next(),g.E().count().next()))

My error:

aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected

aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected

So I'm trying to find out what serializer goes there, but all of my variants are failing. Anyone know what the correct serializer is?

- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 , config: { serializeResultToString: true }}                                                                     # application/vnd.gremlin-v3.0+gryo-stringd

- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 , config: { serializeResultToString: true }}                                                                     # application/vnd.gremlin-v3.0+gryo-stringd

[WARN] o.a.t.g.s.AbstractChannelizer - Could not find configured serializer class - org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 - it will not be available
[ERROR] o.a.t.g.s.AbstractChannelizer - No serializers were successfully configured - server will not start.

[WARN] o.a.t.g.s.AbstractChannelizer - Could not find configured serializer class - org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 - it will not be available
[ERROR] o.a.t.g.s.AbstractChannelizer - No serializers were successfully configured - server will not start.

GitHub

GitHub - bricaud/gremlin-server: A Docker container with a gremlin ...

A Docker container with a gremlin server. Contribute to bricaud/gremlin-server development by creating an account on GitHub.

24 Replies

ManabuBeach•16mo ago

Hi @billmanh Serializer choice is often the target graph implementation dependent. Please let us know which server you are going to connect to.

billmanhOP•16mo ago

Yeah, I'm creating a server (hosting), then I want to connect to it. I'm just looking for the right ioserializer that's current per my gremlin version.

spmallette•16mo ago

it's not really relevant to your question i suppose but Is there any reason you are using that docker container? we do have an official image that we maintain that you wouldn't have to update: https://tinkerpop.apache.org/docs/current/reference/#gremlin-server-docker-image if you prefer that one, it should just work without having to mess around with the serializers. docker

billmanhOP•16mo ago

I think my reasoning for not doing that was that I wouldn't be able to customize the security or configuration if it was a pre-baked image. I'll want to be able to edit the config process. Using the pre-baked container, can I build it with my own config file? However, I did try that container and got a new error RuntimeError: Event loop is closed when running that image. Any idea? The server gave out a longer error

[WARN] c.c.m.CsvReporter - Error writing to org.apache.tinkerpop.gremlin.server.GremlinServer.user-agent.Python/3.8 aiohttp/3.7.4
java.io.IOException: No such file or directory

@spmallette , is it possible to see the repo that generates that dockerfile? If I could see the buildfile and config then I might be able to answer my own question. I'll bet it's just written right there.

spmallette•16mo ago

https://github.com/apache/tinkerpop/blob/master/gremlin-server/Dockerfile

GitHub

tinkerpop/gremlin-server/Dockerfile at master · apache/tinkerpop

Apache TinkerPop - a graph computing framework. Contribute to apache/tinkerpop development by creating an account on GitHub.

spmallette•16mo ago

in what context did you get that error? "event loop closed" sounds like a python error, so i guess you saw that on the client side trying to connect?

billmanhOP•16mo ago

Yeah, my validation is just running a script with the python line:

graph = Graph()
g = graph.traversal().withRemote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
print('Number of nodes {}, number of edges {}.'.format(g.V().count().next(),g.E().count().next()))

spmallette•16mo ago

i guess the server error isnt too relevant. just a metrics reporting issue. that said we shouldnt have that happening. we should fix that. @

billmanhOP•16mo ago

Looking at the dockerfile, I see that the config looks like a buildtime configuration. Can you tell me which config it runs to get the correct serializes? CONF_FILE=$1

spmallette•16mo ago

not sure why you test would fail offhand. perhaps its worth a test on our end

spmallette•16mo ago

i think it just runs the default config file if you dont specify one: https://github.com/apache/tinkerpop/blob/master/gremlin-server/conf/gremlin-server.yaml

GitHub

tinkerpop/gremlin-server/conf/gremlin-server.yaml at master · apach...

Apache TinkerPop - a graph computing framework. Contribute to apache/tinkerpop development by creating an account on GitHub.

spmallette•16mo ago

@colegreer @kennh couple of issues here regarding docker - (1) the csv metrics reporter probaly shouldn't throw that warning - i dont think it even needs to be configured for the docker image (2) a potential issue with python connecting in 3.7.0 - worth a quick check?

billmanhOP•16mo ago

Interestingly, was able to get the serializers settings from that, however still unable to connect.

  File "C:\Users\william.harding\anaconda3\envs\exoplanets\lib\site-packages\aiohttp\streams.py", line 604, in read
    await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x000001FD10DE6430>

Sounds like a client issue, but is there an issue with how I'm connecting? aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected this is a windows/linux issue regarding handling async is that right?

spmallette•16mo ago

i've tested the official 3.7.0 Gremlin Server Docker image with the 3.7.0 gremlinpython package and i'm not finding any problems connecting:

>>> g = traversal().with_remote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
>>> g.V().count().next()
0
>>> g.addV('person').property('name','bill').iterate()
[['addV', 'person'], ['property', 'name', 'bill'], ['none']]
>>> g.addV('person').property('name','bill').toList()
[v[2]]
>>> g.V().count().next()
2
>>> g.V().toList()
[v[0], v[2]]

>>> g = traversal().with_remote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
>>> g.V().count().next()
0
>>> g.addV('person').property('name','bill').iterate()
[['addV', 'person'], ['property', 'name', 'bill'], ['none']]
>>> g.addV('person').property('name','bill').toList()
[v[2]]
>>> g.V().count().next()
2
>>> g.V().toList()
[v[0], v[2]]

that's with Python 3.8.10

billmanhOP•16mo ago

Is that on Windows? Yeah, I'm still getting the: ~\anaconda3\lib\site-packages\aiohttp\streams.py in read(self) 614 self._waiter = self._loop.create_future() 615 try: --> 616 await self._waiter 617 except (asyncio.CancelledError, asyncio.TimeoutError): 618 self._waiter = None ServerDisconnectedError: Server disconnected But that could be an issue with my cleint, not the server.

ColeGreer•16mo ago

@billmanh I'm not sure this is related to your issue here but I wanted to point out that in 3.7.0, the serializers were migrated from a gremlin.driver.ser package to gremlin.util.ser. Therefore the serializer in the config in your original question should be changed to org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1

billmanhOP•16mo ago

Yup! That is exactly what I was origionally looking for. I was able to get that from looking at the docker buildfile, but thanks for confirming. I'm able to build now. Now if I can just get the client issue (which isn't my OG issue) @colegreer , if the server sends a "GraphBinaryMessageSerializerV1" and I'm using bytecode, then the differences between the platforms shouldn't matter, right? I should be able to conenct on a windows machine. Is there annother step that I'm missing? Perhaps the async loop thing?

ColeGreer•16mo ago

Platform differences shouldn't matter here. I'm wondering if that server metrics error is somehow related to the server disconnecting. I believe I can reproduce that metrics error to confirm this.

billmanhOP•16mo ago

Yes, I was able to do this as well:

billmanhOP•16mo ago

Windows. So I'm unblocked. Thanks for your help. However, @spmallette If I wanted to modify the credentials of the pre-baked container, how would I do that? Don't you have to build it in order to place the secruity file?

spmallette•16mo ago

sorry - i forgot to come back to this after not knowing the answer immediately. @triggan do you happen to know how to do what's asked above?

billmanhOP•16mo ago

I suppose there might be some way to take the pre-baked container, source that in a new build file and copy your config files to it. That should work. I haven't started working on that thought. However, I think that's the pattern I'm going to try.

triggan•16mo ago

You can create your own Dockerfile that uses the TinkerPop-provided Gremlin Server image as a base and then customize it from there. Here's an example that I use to create a Gremlin Server container that behaves more like Neptune. I use RUN sed commands to modify the gremlin-server.yaml file.

FROM tinkerpop/gremlin-server:latest

# Add support for both websockets and http requests
RUN sed -i "s|^channelizer:.*|channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer|" ./conf/gremlin-server.yaml

# Allow for string based IDs
RUN sed -i "s|^gremlin.tinkergraph.vertexIdManager=.*|gremlin.tinkergraph.vertexIdManager=ANY|" ./conf/tinkergraph-empty.properties

# Remove ReferenceElementStrategy for use with graph-explorer - return all properties
RUN sed -i "s|^globals << \[g.*|globals << [g : traversal().withEmbedded(graph).withStrategies()]|" ./scripts/empty-sample.groovy

# Increase thread stack to 2m
ENV JAVA_OPTIONS="-Xss2m -Xms512m -Xmx4096m"

FROM tinkerpop/gremlin-server:latest

# Add support for both websockets and http requests
RUN sed -i "s|^channelizer:.*|channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer|" ./conf/gremlin-server.yaml

# Allow for string based IDs
RUN sed -i "s|^gremlin.tinkergraph.vertexIdManager=.*|gremlin.tinkergraph.vertexIdManager=ANY|" ./conf/tinkergraph-empty.properties

# Remove ReferenceElementStrategy for use with graph-explorer - return all properties
RUN sed -i "s|^globals << \[g.*|globals << [g : traversal().withEmbedded(graph).withStrategies()]|" ./scripts/empty-sample.groovy

# Increase thread stack to 2m
ENV JAVA_OPTIONS="-Xss2m -Xms512m -Xmx4096m"

billmanhOP•16mo ago

Hey thanks for that @triggan . Not my origional question but it's a solid bonus.

Gaming

Programming

Trying to run a local version for a test, what is the correct serializer?