Does graph notebook still work after changes to serializers?
I am trying to use the graph-notebook project, my project previously worked. I updated Aerospike graph to the latest version of tinkerpop and with it the serializers were updated.
Trying to run any
%%gremlin now results in an error of
```...Solution:
https://github.com/aws/graph-notebook/issues/548 workaround provided here.
Implementing Graph Filter for Sub-Graph Loading in Spark Cluster with JanusGraph
Hello,
I'm currently utilizing JanusGraph 0.6.4 with Bigtable as the storage backend and encountering difficulties when attempting to run OLAP queries on my graph via SparkGraphComputer. The graph is quite large, containing billions of vertices, and I'm only able to execute queries on significantly smaller graphs.
My queries are being run through the Gremlin console, and the problem appears to be related to loading the graph into Spark RDD. I'm interested in applying a filter to load only vertices and edges with specific labels before running the query....
Testing against AWS Neptune
Hi, the app that I'm maintaining has a Neptune integration for which I've written some integration tests. For those tests I'm using gremlin-server which so far is fine, but because of the cardinality difference between Neptune and other Gremlin implementation I get different behaviour in prod vs testing. Of course, this is a known difference, but I'm wondering if there's a solution that emulates the Neptune engine locally for such use cases.
I could spin-up a new Neptune instance at testing time, but that seems a bit of an anti-pattern. What do people do to test the clients against multiple engines, if at all?...
Solution:
Hi, the main solution I've seen around for such use case (and I'm using myself) is to start the gremlin-server using a custom configuration.
Basically what you need to do is create a custom 'gremlin-server.yaml' file with the server config. There is a field called
graphs in this file, which you can use to specify the path to a custom property file.
This property file is where you define your graph properties, and in your case the defaultCardinality....Janusgraph vertex property with cardinality as SET type
Hi, need a working example of how a property can be added to a vertex with cardinality as SET type and how we can retrieve it and assign it to SET of strings in JAVA.
Please note: I have tried the janusgraph doc code, google search, chat GPT but every time I put something in that property through java code it get stored in STRING format only.
Ex: "["ABC"]" --> current , ["ABC"] --> this I need. ...
Solution:
for JanusGraph you'd want to set up the schema for your multiproperty:
```
mgmt = graph.openManagement()
mgmt.makePropertyKey('p').dataType(String.class).cardinality(Cardinality.SET).make()
mgmt.commit()...
Error: `Failed to authenticate`, when connection pool size is >1 for GremlinServer with ArcadeDB
Hi Friends,
I am exploring and evaluating ArcadeDB.
The DB is setup with GremlinServerPlugin to expose GremlinServer at port 8182.
Using SpringBoot to create client app.
`@Bean...
How can I use the .io("filename.json").write() pattern to append to an existing graphson file?
I have read about defining a custom GraphWriter using the builder, but wanted to ask first before spending more time
Here is my example code:
```
private static Graph buildGraph() {...
Solution:
The subgraph is stored as a side-effect, so you could use
withSideEffect() to give it your own graph instance and then reuse that across multiple traversals to subgraph:
```gremlin> g = TinkerFactory.createModern().traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> sub = TinkerGraph.open()
==>tinkergraph[vertices:0 edges:0]...using tinkerpop 3.6.x on a jdk 21 project
I'm using tinkerpop 3.6.5 to read/write over an AWS neptune graph database.
Is there any issues using this library on a jdk 21 project ?
While the doc says "You can now run TinkerPop with Java 17." : https://issues.apache.org/jira/browse/TINKERPOP-2703
I have the impression it's only regarding Gremlin console that I do not use....
Solution:
I've not personally tried JDK21 so I can't say for sure how TinkerPop and its dependencies will behave. Note that as of right now we do not use any features of Java beyond JDK8, continually test builds on JDK8, 11 and 17 and that we release on JDK8 bytecode. So presumably, any JDK that can run JDK8 should be able to use TinkerPop without trouble. In practice of course it's hard to say that there aren't subtle differences among JDK versions that you could encounter. As an example, I can think of...
Is the following use of next() allowed?
The update does not seem to work in this case, even when
const changes = { ... } let v0 = g.V(0); if (!(await v0.next()).value) {...
changes is a non-empty object. No error is raised, and when I printed the result, it has { value : null, done : true }.
```const changes = { ... } let v0 = g.V(0); if (!(await v0.next()).value) {...
Solution:
It turns out, no, that's not allowed. It seems like
done: true signifies that the traversal sequence was already committed and I'm doing basically nothing.Gremlin-go cannot bring out props when querying Vertex with Neptune
```golang
g := gremlingo.Traversal_().WithRemote(driverRemoteConnection)
traversal := g.GetGraphTraversal()
result, err := traversal.V("v-1").Next()...
Solution:
The short answer is that you can get properties on your returned elements after 3.7.0 - https://tinkerpop.apache.org/docs/current/upgrade/#_properties_on_elements (note that the graph database that you are using must also support 3.7.x). If you care about the long answer then you should read this: https://lists.apache.org/thread.html/e959e85d4f8b3d46d281f2742a6e574c7d27c54bfc52f802f7c04af3%40%3Cdev.tinkerpop.apache.org%3E
search for vertices where multiple properties
I need to search for vertices where multiple properties are a certain value.
Here is what I am able to come up with.
```
try (TinkerGraph graph = TinkerGraph.open()) {...
repeat and until methods in Javascript Gremlin:
I'm not particularly sure how to use them properly. From my current understanding, repeat() and until() are both instance methods of the GraphTraversal class, but how do I reference any methods within the class body but still refer to the same reference?
...
const org = await g.V().until((r) => r.hasLabel('')).repeat({...}).path().by('name').next()
const org = await g.V().until((r) => r.hasLabel('')).repeat({...}).path().by('name').next()
Solution:
I think this section of the documentation will help with imports: https://tinkerpop.apache.org/docs/current/reference/#gremlin-javascript-imports In particular:
```
const = gremlin.process.statics;
const org = await g.V().until.hasLabel('b')).
repeat(__.in())....
Gremlin driver setup for Amazon Neptune behind a Load balancer
Hi folks,
I've been running into issues connecting to Amazon Neptune behind an HAProxy as detailed on https://aws-samples.github.io/aws-dbs-refarch-graph/src/connecting-using-a-load-balancer/ (2nd option)
The issue is to do with sending subsequent queries via a graph traversal source where it appears to cause NoHostAvailableExceptions frequently. The interesting part here is that this only happens when a Graph traversal source is reused.
I'm using Gremlin driver 3.7.0 in Java....
Gremlin (with Python + Neptune) Out of Memory Error with .toList()[0], .next() Fixes It. But Why?
This is not really a question, but more of a discussions on how internally this would cause an out of memory error.
I have the following construct and last night, it resulted in out of memory error.
*
path_v is a single well defined V. I am expecting it is returning a single V....AWS Neptune and gremlin-javascript versions
Hi,
I'm using the js gremlin client 3.6.2 with AWS Neptune version 1.2.1.0 and I get the following error when trying to upsert nodes with mergeV()
```
ResponseError: Server error: {"code":"InternalFailureException","requestId":"aa89ee40-74d5-46c1-9ad5-0bc3f2fde5c1","detailedMessage":"null: .option(LinkedHashMap, LinkedHashMap)","message":"null: .option(LinkedHashMap, LinkedHashMap)"} (599)...
Solution:
That's a strange error because the error seems to indicate that
option() is being called with two Map arguments, but your example does not seem to indicate that you are doing that. It's interesting that 3.7.0 "fixes" it because I can't think of any reason why that would matter. Perhaps you could try 3.6.5 to see if there were somehow changes after 3.6.2 that solved this problem?
Regarding your more general question of:
My question is: does the version of the javascript client match the version of the server (or should it)? Or should I always strive to use the latest client?Generally speaking you should follow the version table here https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-gremlin-client.html - While it is possible that versions outside of those ranges in that table will work with one another, those are the versions that Neptune tests against and that are supported....
Docker Janusgraph Custom ID Values
I'm trying to setup a janusgraph database with custom verex ID values. I have the following docker-compose configuration:
```yaml
version: '3.8'
services:...
Solution:
You could do both
```
graph.set-vertex-id=true
graph.allow-custom-vid-types=true...
Reusing connections
Hi,
I'm wondering what's the recommended way of using connections to a graph DB. The documentation uses web sockets like so (in javascript)
```...
Solution:
Connection pooling in Javascript is a bit of an oddity, since the language has mostly been intended for single-threaded purposes (running in a browser). With that, the websocket library native to Javascript hasn't traditionally supported connection pooling natively. In testing we've done within AWS, the Javascript websocket library is by far the most efficient in creating a new connection (single digit milliseconds). Whereas something like Python's implementation can be really expensive (10s of milliseconds, or worse). So maintaining long-lived connections in Javascript is likely not as big of a deal as it is in other runtimes.
(My opinion, so take this with a grain of salt). You should really avoid using websockets until you're faced with a use case that would benefit from them. There's a lot of dev overhead in creating and maintaining websocket connections. Websocket connections are also problematic in distributed compute implementations (or for high availability) as they act like "sticky sessions". They can also cause problems with load balancing logic. I would suggest using http requests for as long as you can before relying on the use of websockets. There is obviously the tradeoff of serialization when it comes to using Gremlin with http vs websockets, but that is something you should be able to handle once and be done with it....
Can I surpress gremlin console's warnings?
How can I surpress these WARNING messages? I've tried gremlin -l but can't seem to get the syntax right because it seems to have no effect when I do
gremlin -l ERROR
```
% gremlin
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/usr/local/apache-tinkerpop-gremlin-console-3.6.2/lib/groovy-2.5.15-indy.jar) to method java.lang.Object.finalize()...Solution:
I have found that
export JDK_JAVA_OPTIONS="$JDK_JAVA_OPTIONS --illegal-access=deny" produces the following
```
% export JDK_JAVA_OPTIONS="$JDK_JAVA_OPTIONS --illegal-access=deny"
% gremlin
NOTE: Picked up JDK_JAVA_OPTIONS: --illegal-access=deny...Sequential IDs in Neptune?
@neptune
I'm attempting to implement sequential IDs for the vertices in our AWS Neptune graph. So far, we have added a new property called
vertexNumber, which will store the numeric sequential ID for each vertex. Then, before saving a vertex to the database, I run a simple query to retrieve the current highest vertex number, increment it, and store the new vertexNumber to that vertex. Pseudo-code examples found below.
```...Solution:
@andys1814 How much do you care about the ids being truly sequential or is having some gaps acceptable as long as they are human readable?
I ask as this was a common request when I was working with Cassandra. A common practice was to allot a range of ids to each client on connection versus getting a new one each time. When a client exhausts it's assigned range it then reaches out to get a new range.
This helps to minimize the single point of failure and additional overhead of having to go to a single coordinator to get an id value for each request. It does however means that inserts will not be in sequential order and that you may have gaps in the number. This may or may not be an issue depending on your use case....
This helps to minimize the single point of failure and additional overhead of having to go to a single coordinator to get an id value for each request. It does however means that inserts will not be in sequential order and that you may have gaps in the number. This may or may not be an issue depending on your use case....
Gremlin console vs REST API
I'm trying to get a path and the properties of the vertices and the edges for that path by running a gremlin script via REST API in Neptune. The query is simple with some filters on the edges, but the result is different from what gremlin console gives me.
For example, the following query returns the path with the type of entities (vertex or edge) but only the label and the ID properties.
```
g.V('efc912d3-6cec-49e2-9717-85625bab5243').inE().outV().path()...
Solution:
Generally speaking, the GLVs and the Gremlin Console (if sending queries as bytecode and connecting via websockets) will serialize the results back into types that are common to the runtime that you're using for your application.
Queries sent over HTTP will return the response using GraphSON (GraphSONv3, be default with Neptune) which includes all of the extra type information....
Queries sent over HTTP will return the response using GraphSON (GraphSONv3, be default with Neptune) which includes all of the extra type information....
Cryptic Neptune Gremlin Error Rate Creeping - What Would You Recommend?
This relates more to do with Neptune usage, nevertheless, it is also related to the Gremlin Query error rate that logs in to the Monitor plot page and also triggers cloud watch alert in our case.
Situation:
We've noticed a gradual increase in unexplained Gremlin error counts — with a few popping up every several hours....