Gremlin Driver and frequently changing servers
In a containerised environment, hosts are frequently replaced and their IP address can change several times a day.
As far as I can tell, Gremlin Driver was designed for long-lived hosts given that:
(1) Contact Points are resolved on startup and a connection pool is assigned to them at that time - this makes varying contact points over time not possible I think?
(2) Unavailable hosts are retried but the list of hosts is not refreshed - wouldn't it make more sense to give up on hosts after a few retries and refresh the list of contact points?
I was wondering if the community had any guidance for containerised environments, if handling such cases within Gremlin Driver made sense, and if it'd be worth filing a JIRA ticket.
Thanks!
Solution:Jump to solution
yes, this is a bit of an issue in some cases like the one you describe. a similar issue occurs for Neptune where it would be helpful if the drivers knew the cluster topology. you're basically left to periodically recreate the
Cluster object and updating the contact points. Neptune has a special client that wraps the TinkerPop driver to do just that: https://aws.amazon.com/blogs/database/load-balance-graph-queries-using-the-amazon-neptune-gremlin-client/ i've always thought it would be nice if...Amazon Web Services
Load balance graph queries using the Amazon Neptune Gremlin Client ...
[Updated August 2021]Ā The Gremlin Client for AmazonĀ NeptuneĀ is now available fromĀ Maven Central. Some APIs have changed since this article was published. Please review theĀ demo code in the GitHub repositoryĀ for the latest examples of how to use the APIs. Amazon Neptune is a fast, reliable, fully managed graph database service that makes it easy ...
4 Replies
Solution
yes, this is a bit of an issue in some cases like the one you describe. a similar issue occurs for Neptune where it would be helpful if the drivers knew the cluster topology. you're basically left to periodically recreate the
Cluster object and updating the contact points. Neptune has a special client that wraps the TinkerPop driver to do just that: https://aws.amazon.com/blogs/database/load-balance-graph-queries-using-the-amazon-neptune-gremlin-client/ i've always thought it would be nice if the TinkerPop drivers (all of them) were smarter about this sort of things. i think we'd be open to ideas an improvements in this area. cc/ @kennhAmazon Web Services
Load balance graph queries using the Amazon Neptune Gremlin Client ...
[Updated August 2021]Ā The Gremlin Client for AmazonĀ NeptuneĀ is now available fromĀ Maven Central. Some APIs have changed since this article was published. Please review theĀ demo code in the GitHub repositoryĀ for the latest examples of how to use the APIs. Amazon Neptune is a fast, reliable, fully managed graph database service that makes it easy ...
Thanks Stephen šš» I was thinking of implementing something very similar actually (i.e. adding a refresh method to Client and Cluster, and refresh either periodically or on exception). The API is well locked so that requires copying and editing a couple classes in a tinkerpop package, but at least the linked code convinced me I am on the right track. Thank you.
if you plan to make changes, i'd encourage you to consider making a more formal proposal for how you think it should work and submitting a pull request. then you don't have to worry about patching and other such modifications in the future.
we're happy to help you through the steps of doing that if you'd like to contribute
Sure! I'll spend some time on this. If this looks worth contributing I'll ping here or will open a JIRA with more details.