logging and alerting inside a gremlin step
I am trying to add a log statement inside the step but I am getting an error Server error: null (599)
ResponseError: Server error: null (599)
This is the error we get // Log the creation of a shell flight
.sideEffect(() => logger.info(
Shell flight created for paxKey ${paxKey} with flightId ${flightLegID}))
)
I only get this error when I log it in the sideEffect
This is the code we have:
await flightLegT
.fold()
.coalesce(
.unfold(),
.addV('flight')
.property(t.id, flightLegId)
.property('property1', .value1)
.property('property2', value2.)
.property('property3', value3)
.property('property4', value4)
.property('isShellFlight', true)
.as('shellFlight')
.sideEffect(
.V(originDateId).addE('vertex1').to(.select('shellFlight'))
)
.sideEffect(
.select('shellFlight').as('sf').addE('vertex2').from_(.select('sf'))
)
// Log the creation of a shell flight
.sideEffect(() => logger.info(Shell flight created for paxKey ${paxKey} with flightId ${flightLegID}))
)
.as('flightLeg')Solution:Jump to solution
Since you are getting a "Server error" I assume you are using Gremlin in a remote context. I'd further assume you are not sending a script to the server, but are using bytecode or are using a graph like Amazon Neptune. Those assumptions all point to the fact that you can't use a lambda that way in remote contexts.
The approach you are using to write your lambda is for embedded use cases only (i.e. where query is executed in the same JVM where it was created). If you want to send a lambda remotely you would need to have a server that supports them (e.g. Neptune does not, but Gremlin Server with Groovy ScriptEngine processing does) and then follow these instructions: https://tinkerpop.apache.org/docs/current/reference/#gremlin-java-lambda
The other thing to consider is that the lambda will be executed remotely so it might not know what "logger" is and the log message will appear on the server note the client. ...
10 Replies
Solution
Since you are getting a "Server error" I assume you are using Gremlin in a remote context. I'd further assume you are not sending a script to the server, but are using bytecode or are using a graph like Amazon Neptune. Those assumptions all point to the fact that you can't use a lambda that way in remote contexts.
The approach you are using to write your lambda is for embedded use cases only (i.e. where query is executed in the same JVM where it was created). If you want to send a lambda remotely you would need to have a server that supports them (e.g. Neptune does not, but Gremlin Server with Groovy ScriptEngine processing does) and then follow these instructions: https://tinkerpop.apache.org/docs/current/reference/#gremlin-java-lambda
The other thing to consider is that the lambda will be executed remotely so it might not know what "logger" is and the log message will appear on the server note the client.
Gremlin doesn't have a better way to really do what you are trying to do. I've often thought a
debug() or print() step might be helpful in some way, but we've never gotten around to building that.yeah this is really a typescript lambda and I wanted to log and alert , that a database search for a flight didnt give me a result, so I am creating a shell flight in DB and want to alert and log it in a single step asynchronously, without adding an extra blocking DB call(this is all a part of a bigger gremlin transaction gtx). Database we use is AWS Neptune. I tried several ways to log and alert it using different steps but couldn't do it. However, if I use __.fail('No data found') it prints the logs and works fine, but I dont want to do failure, I want to add the dummy record(shell flight) and log it.
Neptune definitely does not offer a way to do that sort of thing since it doesn't support closures (i.e. lambdas).
fail() is the closest thing there is. you can sort of use constant() to some degree, but that implies that you are able to get a result back to work with the constant(). i dont think there are options to do a side-effect like that.
this might be a good use case for Neptune to consider for call() perhaps, where Neptune could write to logs you could view in aws console. @Dave what do you think?I hadn't actually thought of custom logging as a use of
call() but I think that makes sense. Since the available logging features/locations are going to be different per provider this use would fit well into that paradigm. In this case though it would still have some amount of blocking even if it was a fire-and-forget type request.hey @Dave do you suggest using next() and then putting check whether we get data in the middle of a non blocking transaction, could you please give me details of it
i'm not sure that even works exactly as you want because
next() isn't telling the server you want one more object. the call to next() is saying give me the next item that is in my client-side result set (i.e. the results are streaming from the server as soon as you call next() in the first place). are you just trying to tell if the shell flight was created or updated so that you can log that?yes pretty much and later I would build an alert on that log, I am just trying to log that shell flight was created with ${vertexID}
i think the best you can really do is to return something in your results to tell you what happened. something like:
sure Thanks I will try that @spmallette
Hi @spmallette this query worked but when I put my code in production I am getting ton of {"detailedMessage":"Failed to complete operation due to conflicting concurrent operations. Please retry. 0 transactions are currently rolling back.","code":"ConcurrentModificationException","requestId":"15c91b13-772f-4a98-a7fa-35630de30679","message":"Failed to complete operation due to conflicting concurrent operations. Please retry. 0 transactions are currently rolling back."} (500)" which is causing issues in my lambda. I have heard that changing coalesce/addV to mergeV might prevent this, is it true, I almost changed to mergeV and mergeE everywhere except this particular code where I am unable to do it
This is how attempted to change to mergeV but it doesnt work due to type issues
await flightLegT
.fold()
.coalesce(
.unfold(),
.mergeV(
new Map([
[t.id, shellFlightLegId], // ID should be string
[t.label, 'flight'] // Label should be string
])
)
.option(
merge.onCreate,
new Map([
['purgeDate', poflVertexAttrs.purgeDate], // Ensure this is of the correct type
['flightNumber', flightLegKey.flightNumber.toString()], // Ensure this is a string
['instanceNumber', flightLegInstanceNumber], // Ensure this is of the correct type
['flightSuffix', flightLegSuffix], // Ensure this is a string
['creationTimestamp', Date.now()], // Date.now() returns a number
['carrierCode', flightLegKey.carrierCode], // Ensure this is a string
['isShellFlight', true] // Boolean value
])
)
)
.project('v', 'status')
.by()
.by(.choose(.hasLabel('flight'), .constant('merged'), .constant('created')))
.next();
you'll likely not be able to completely avoid
ConcurrentModificationException situations. You can only hope to limit them. Usually implementing a simply automatic retry system with exponential backoff will solve the problem.