Apache TinkerPop•2y ago

How can I filter by property type

Some time ago, we discovered that a certain timestamp property was being stored as a String instead of a Long in our database. This issue is now fixed, but some of the old data is still hanging around and I want to filter it out of a query that I'm writing. How can I exclude all the vertices that contain the property with type String? I tried using a lambda but I think they are disallowed and always return errors in Neptune.

6 Replies

danielcraig23OP•2y ago

The error that I'm seeing looks like this: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: {"detailedMessage":"Cannot compare '1686369420000' (String) and '1689741600001' (Long) as both need to be an instance of Number or Comparable (and of the same type)","requestId":"9fd61d32-e3cb-40f1-9916-93cf6fffcfda","code":"InvalidParameterException"} java.util.concurrent.CompletionException: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: {"detailedMessage":"Cannot compare '1686369420000' (String) and '1689741600001' (Long) as both need to be an instance of Number or Comparable (and of the same type)","requestId":"9fd61d32-e3cb-40f1-9916-93cf6fffcfda","code":"InvalidParameterException"}

spmallette•2y ago

At this time, Gremlin doesn't have mechanisms for testing types. It remains to be one of the troublesome areas for Gremlin. As you have found Lambdas are not allowed by Neptune. I think your only recourse is to pull back all the vertex ids and values, check them locally and then write the update back to fix it. anyone else at @neptune have any ideas for this?

ManabuBeach•2y ago

Just curious, can we apply a pattern matching type strategy in Gremlin? I have not used that so I do not know. And come to think of it, on NoSQL type situation, schema inconsistencies cannot almost be avoided to me, so perhaps, pattern match construct may be helpful. My situation is that Java/Scala people (can) write numbers in longs while JavaScript people write numbers out either in strings or in double float and it's difficult to control developers to be consistent.... just some thoughts.

triggan•2y ago

A current workaround for this in Neptune is to use openCypher and the type casting functions that it has: Example graph with two vertices each with a property of two different data types:

g.addV('testTypes').property(id,'testTypes01').property('prop1',100).
    addV('testTypes').property(id,'testTypes02').property('prop1','100')

g.addV('testTypes').property(id,'testTypes01').property('prop1',100).
    addV('testTypes').property(id,'testTypes02').property('prop1','100')

Check for nodes with a property value of type Int:

MATCH (n:testTypes)
WHERE toInt(n.prop1) = n.prop1
RETURN n

MATCH (n:testTypes)
WHERE toInt(n.prop1) = n.prop1
RETURN n

Returns:

{'results': [{'n': {'~id': 'testTypes01',
    '~entityType': 'node',
    '~labels': ['testTypes'],
    '~properties': {'prop1': 100}}}]}

{'results': [{'n': {'~id': 'testTypes01',
    '~entityType': 'node',
    '~labels': ['testTypes'],
    '~properties': {'prop1': 100}}}]}

Check for nodes with a property value of type String:

MATCH (n:testTypes)
WHERE toString(n.prop1) = n.prop1
RETURN n

MATCH (n:testTypes)
WHERE toString(n.prop1) = n.prop1
RETURN n

Returns:

{'results': [{'n': {'~id': 'testTypes02',
    '~entityType': 'node',
    '~labels': ['testTypes'],
    '~properties': {'prop1': '100'}}}]}

{'results': [{'n': {'~id': 'testTypes02',
    '~entityType': 'node',
    '~labels': ['testTypes'],
    '~properties': {'prop1': '100'}}}]}

spmallette•2y ago

forgot that...thanks

danielcraig23OP•2y ago

that's interesting, thank you!

Gaming

Programming

How can I filter by property type