Apache TinkerPop•4d ago

Neptune, Gremlin Java & Bindings

Hey, I've got a bit of an issue that I'm trying to find a solution for regarding the use of bindings in Gremlin with Neptune. As stated in the docs, they're not supported. What I'd like to get to is a workaround solution where I can locally apply bindings to a query, convert it to a string and submit it to Neptune with the bindings "applied". So for instance given the following query: g.V().hasLabel(my_label_param).limit(results_limit), and bindings my_label_param='airport', results_limit=100 I'd like to submit the following query on Neptune: g.V().hasLabel('airport').limit(100) Is there a way perhaps through the gremlin-lang implementation to achieve this? The reason I'd prefer it this way is so that I'm not relying on string replacement where gremlin injection could occur.

Solution:

@G.V() - Gremlin IDE (Arthur) with a small modification, it's possible to change the existing translators in 3.x to replace variable placeholders for the values in the bindings. adding this override to GroovyTranslator.DefaultTypeTranslator: ``` @Override protected Script convertToScript(final Object object) { if (object instanceof Bytecode.Binding) {...

Jump to solution

9 Replies

Andrea•4d ago

Hello @G.V() - Gremlin IDE (Arthur) is there a specific tinkerpop version you are looking to use? I am asking because tinkerpop 4.0 had some recent changes to introduce gremlin parameterization using GValues - see https://github.com/apache/tinkerpop/pull/2919

GitHub

[TINKERPOP-2959] Introduce GValue for Improved Parameterization by ...

https://issues.apache.org/jira/browse/TINKERPOP-2959 The parameterization which was added to gremlin-lang in 3.7 has a key limitation as a result of all variables directly resolving to literals as ...

gdotvOP•4d ago

there's potentially going to be some challenges in adopting TP4.0 in G.V() though I haven't looked into it for a while. I am noticing (for the first time) https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/GremlinLang.java#L220 which kinda looks like it would do the trick for me as I expect it would allow me to "string replace" parameter names with properly parsed and sanitized values for the corresponding parameters

GitHub

tinkerpop/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/p...

Apache TinkerPop - a graph computing framework. Contribute to apache/tinkerpop development by creating an account on GitHub.

gdotvOP•4d ago

That function might actually be all I need here - for Neptune specifically, when sending queries, I can preprocess the query by replacing what would be the bindings to their actual value and submit the query with peace of mind that the parameter values have been sanitized properly I'll start off by testing that - im guessing the safest handling for me is to ensure that any string valued parameter is escaped with StringEscapeUtils.escapeJava which I'm seeing tinkerpop does behind the scenes

Andrea•4d ago

Since tinkerpop 4 is not yet released, will it be a problem for you that Neptune will not have yet adopted the TP4 gremlin parameterization?

gdotvOP•4d ago

I'm always going to be stuck having to support "old" and "new" versions of database engines, particularly with Neptune where I have quite a few users. I'm generally having to take lowest common denominator approaches that ensure support across the board. I think deriving the code in GremlinLang to achieve what I need just for Neptune will actually do the trick. Even if it's not standard the important part is it works and is secure - I usually have to do little tweaks here and there for different database providers based on their limitations so it's nothing new for me!

Solution

spmallette•3d ago

        @Override
        protected Script convertToScript(final Object object) {
            if (object instanceof Bytecode.Binding) {
                return super.convertToScript(((Bytecode.Binding) object).value());
            } else {
                return super.convertToScript(object);
            }
        }

        @Override
        protected Script convertToScript(final Object object) {
            if (object instanceof Bytecode.Binding) {
                return super.convertToScript(((Bytecode.Binding) object).value());
            } else {
                return super.convertToScript(object);
            }
        }

seems to make it work how you want:

gremlin> g = TinkerFactory.createModern().traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> b = new org.apache.tinkerpop.gremlin.process.traversal.Bindings()
==>bindings[main]
gremlin> t = g.V().has('name',b.of("n","josh"));[]
gremlin> translator = GroovyTranslator.of("g")
==>translator[g:gremlin-groovy]
gremlin> translator.translate(t).getScript()
==>g.V().has("name","josh")

gremlin> g = TinkerFactory.createModern().traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> b = new org.apache.tinkerpop.gremlin.process.traversal.Bindings()
==>bindings[main]
gremlin> t = g.V().has('name',b.of("n","josh"));[]
gremlin> translator = GroovyTranslator.of("g")
==>translator[g:gremlin-groovy]
gremlin> translator.translate(t).getScript()
==>g.V().has("name","josh")

I guess from your perspective that means you create your own extension to DefaultTypeTranslator then hand it to GroovyTranslator.of("g", <YourTypeTranslator>). i think that would work. The 4.x translators based on the grammar (and not bytecode) are designed to allow this sort of translation more directly, but that's not something you should have to worry about for a while i guess.

gdotvOP•3d ago

I'll have a look, thanks! I was wondering if I could somehow use the Translator classes and either the Java or GroovyTranslator! So just to confirm, that worked. I do have a bit more work to do here as queries are submitted with terminal steps which won't translate to a graphTraversal but it's a good start, for reference, this is what I've got:

try {
                        GraphTraversalSource g = AnonymousTraversalSource.traversal()
                                .withEmbedded(EmptyGraph.instance());
                        ConcurrentBindings b = new ConcurrentBindings();
                        b.putIfAbsent("g", g);
                        parameters.keySet().forEach(k -> {
                            b.putIfAbsent(k, parameters.get(k));
                        });
                        // Cheating a bit here, TODO implement terminal step detection and extraction to allow parsing to graphtraversal
                        String queryNoTerminalStep = query.replaceFirst(".toList\\(\\)", "");
                        GremlinLangScriptEngine glse = new GremlinLangScriptEngine();
                        GraphTraversal traversal = (GraphTraversal) glse.eval(queryNoTerminalStep, b);
                        query = GroovyTranslator.of(databaseConfiguration.getGraphTraversalSourceName() != null ? databaseConfiguration.getGraphTraversalSourceName() : "g",
                                new BindingsToGroovyTypeTranslator(false)).translate(traversal).getScript() + ".toList()";
                    } catch (ScriptException e) {
                        // TODO might need to handle that somehow
                        e.printStackTrace();
                    }

try {
                        GraphTraversalSource g = AnonymousTraversalSource.traversal()
                                .withEmbedded(EmptyGraph.instance());
                        ConcurrentBindings b = new ConcurrentBindings();
                        b.putIfAbsent("g", g);
                        parameters.keySet().forEach(k -> {
                            b.putIfAbsent(k, parameters.get(k));
                        });
                        // Cheating a bit here, TODO implement terminal step detection and extraction to allow parsing to graphtraversal
                        String queryNoTerminalStep = query.replaceFirst(".toList\\(\\)", "");
                        GremlinLangScriptEngine glse = new GremlinLangScriptEngine();
                        GraphTraversal traversal = (GraphTraversal) glse.eval(queryNoTerminalStep, b);
                        query = GroovyTranslator.of(databaseConfiguration.getGraphTraversalSourceName() != null ? databaseConfiguration.getGraphTraversalSourceName() : "g",
                                new BindingsToGroovyTypeTranslator(false)).translate(traversal).getScript() + ".toList()";
                    } catch (ScriptException e) {
                        // TODO might need to handle that somehow
                        e.printStackTrace();
                    }

I'm assuming converting to a graphTraversal is the right approach here, as opposed to Bytecode. I've created a really simple visitor that extends the BaseGremlinVisitor and just extracts the terminalStep's text via

visitTraversalTerminalMethod

visitTraversalTerminalMethod

so that I can easily swap it in and out during the translation process (just putting the information out there for anyone else who might find this useful, however unlikely it is!)

spmallette•3d ago

it will be nice when we have the new translators based on the grammar available. i think it will be easier to extend and will definitely produce better translations.

gdotvOP•3d ago

honestly thought the implementation on my side went very easily - I'll likely be using more visitor patterns in the future for the G.V() backend. Right now a lot of the grammar stuff is happening on the frontend but I can picture situations where backend parsing could be interesting too I'll mark this as resolved, as it stands TP has everything I needed to solve this specific problem

Gaming

Programming

Neptune, Gremlin Java & Bindings

Did you find this page helpful?