3x1 Comments - Answer Overflow

3x1

•Created by Dragos Ciupureanu on 12/6/2023 in #questions

Testing against AWS Neptune

That's my understanding yes, either local server with a config to be as close to Neptune as possible, either have a small Neptune instance ready for integration tests. Just to add on that, I remember seeing some differences between gremlin server and Neptune. For example I had a query like :

traversal.by(
  __.coalesce(
    __.select('v').values(params.eventTimeAttribute),
    __.constant(0)
  )
)

traversal.by(
  __.coalesce(
    __.select('v').values(params.eventTimeAttribute),
    __.constant(0)
  )
)

which would run fine in gremlin server when eventTimeAttribute was null but not in Neptune. So TLDR : - gremlin server for local unit-test / speed / 0-cost but not 100% matching - Neptune to make sure tests are run exactly like in reality

12 replies

ATApache TinkerPop

•Created by Dragos Ciupureanu on 12/6/2023 in #questions

Testing against AWS Neptune

The graph field I mentioned looks like this in the yaml file :

graphs: { graph: ./gremlin/neptune_graph.properties }

graphs: { graph: ./gremlin/neptune_graph.properties }

12 replies

ATApache TinkerPop

•Created by Dragos Ciupureanu on 12/6/2023 in #questions

Testing against AWS Neptune

Hi, the main solution I've seen around for such use case (and I'm using myself) is to start the gremlin-server using a custom configuration. Basically what you need to do is create a custom 'gremlin-server.yaml' file with the server config. There is a field called graphs in this file, which you can use to specify the path to a custom property file. This property file is where you define your graph properties, and in your case the defaultCardinality. Here is the configuration that I use to mimic Neptune :

gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
gremlin.tinkergraph.vertexIdManager=ANY
gremlin.tinkergraph.edgeIdManager=ANY
gremlin.tinkergraph.defaultVertexPropertyCardinality=set

gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
gremlin.tinkergraph.vertexIdManager=ANY
gremlin.tinkergraph.edgeIdManager=ANY
gremlin.tinkergraph.defaultVertexPropertyCardinality=set

I believe this is explained somewhere in the gremlin server documentation but I don't remember where. A good starting point is this book which I found very useful to start with gremlin server : https://www.kelvinlawrence.net/book/PracticalGremlin.html#serverconfig (Kelvin Lawrence is in the discord by the way)

12 replies

ATApache TinkerPop

•Created by Andys1814 on 11/7/2023 in #questions

Sequential IDs in Neptune?

Hi, I suggest you have a look at this page : https://docs.aws.amazon.com/neptune/latest/userguide/transactions-neptune.html#transactions-neptune-read-only More broadly at this section about transactions : https://docs.aws.amazon.com/neptune/latest/userguide/transactions.html Neptune operates with transactions, so if you don't modify a part of your graph, you will always return the same value and will most likely use its cache to speed up the result. Just as a side note, if you have concurrent threads running the example you show, it's very likely you will face race conditions. To avoid it you either have to make sure only 1 thread updates 1 part of the graph, or build a single query to achieve what you want (read + write in the same query).

16 replies

ATApache TinkerPop

•Created by ManabuBeach on 8/3/2023 in #questions

The Cascading Coalescing - Create a V then Create an E in One Shot

I'm guessing with mergeE/mergeV now it should be possible to do batch processing for this kind of queries, but coalesce was quite hard to use in this case compared to the simple g.V(...).has(..).... I have now

15 replies

ATApache TinkerPop

•Created by ManabuBeach on 8/3/2023 in #questions

The Cascading Coalescing - Create a V then Create an E in One Shot

Just to add my 2 cents on this, I also have a highly concurrent "create if not exist" event processing, and in general the easiest way to deal with this (without mergeV/mergeE at the time) was to deal with the concurrency upfront using some partitioning (Kinesis data streams in my case), then you can run multiple barch queries sequentially (read all vertex at once, write all missing edges at once, etc...)

15 replies

ATApache TinkerPop