Vertex ID collisions
Hello!
I'm following up on my questions from https://discord.com/channels/981533699378135051/1198566881360093274/1198566881360093274 and https://discord.com/channels/981533699378135051/1188579609667711047
We want to migrate our existing JanusGraph from Bigtable to Cassandra.
One of the hypothesis was to try and leverage JG 1.0 custom vertex IDs, to import a GraphSON file while keeping the original IDs.
The experiment went like this:
1. Export graph with JG 0.6, using
graph.io(graphson()).writeGraph("/tmp/my-graph.json")
2. Bootstrap a JG 1.0 insance against Cassandra, setting graph.set-vertex-id to true.
3. Importing the graph with graph.io(IoCore.graphson()).readGraph('/tmp/my-graph.json') (we make sure to call graph.tx().commit() at the end)
4. Rebuilding our Elastic indices.
5. Setting graph.set-vertex-id to false and shutting down the instance.
6. Bring back JG 0.6.4 working with Cassandra, and resume operation.
The import worked and we were able to see the vertices with their original IDs.
The show-stopper came later, when we tried resuming our business logic, we started experiencing vertex lack-of-collisions.
New vertices would get IDs of existing vertices and overwrite them, resulting in a corrupteg graph.
Some thing we experimented with, that did not work
* Explicitly setting "ids.authority.conflict-avoidance-mode") to GLOBAL_AUTO.
* Increasing ids.block-size to a number slightly higher than the number of vertices we import (~26,000)
@Florian Hockmann @Boxuan Li would appreciate any ideas an insights on this.
I know that jumping between JG versions doesn't sound like a great idea, but we still working on migrating out python codebase to Tinkerpop 3.7.0
many thanks in advance!1 Reply
After doing several hours of reading, I wonder if this is related to the
simple placement strategy - since we use the graph.io step which will load the entire graph into a single partition? I assume this can mean that other verices can have the same exact vertex ID if they belong to another partition?