Vitor Martins
ATApache TinkerPop
•Created by Vitor Martins on 8/8/2024 in #questions
Optimizing connection between Python API (FastAPI) and Neptune
Hey @triggan , thanks a lot for your response, that's also a lot for us to study, I'm really happy to dive more into this! 😄
We've seen that t3 and t4g are not recommended for production, that's why we tried with r6g.large at some point, but it'll be good for us to make these tests with this instance to compare with the t4g, and also with serverless. We'll do it and then we can return with the results.
Good to know about the connection pool size, it's good to have a solid base to this.
For now I think websockets fits better for our purpose, but it's also a good thing to know and maybe try http at some point.
The count query was just an example that we thought as an average of the queries we do, not so simple but not too complex either. The idea was to fast experiment the query without having to code a query of our workloads on another application without our DSL. Maybe it can be a thing that we can explore better in future tests.
But again, a lot of things for us to study and try, thanks for your help and I'd like to keep this thread open so we can go further with this discussion.
11 replies
ATApache TinkerPop
•Created by Vitor Martins on 8/8/2024 in #questions
Optimizing connection between Python API (FastAPI) and Neptune
The results we achieved can be checked on this spreadsheet:
https://docs.google.com/spreadsheets/d/1lTd0ke4zhhWZ-LkpgkD2-1lqY01G8HbSo9DbJhKfHf4/edit?gid=0#gid=0
There are a few comments that I want to mention about these results:
- Using a connection pool greater than 1 sometimes make the queries fail with MemoryLimitExceededException.
- Running with asyncio.wrap_future in some tests caused all requests to fail, without logs on the application. In these cases the tests were executed again. There were cases where only a few requests failed, but also without logs.
And finally, my third and last question (for now 😅) is:
3. “What can we do to improve our performance when dealing with concurrent requests?”
11 replies
ATApache TinkerPop
•Created by Vitor Martins on 8/8/2024 in #questions
Optimizing connection between Python API (FastAPI) and Neptune
Here's the shape of the response, that represents a bit of our graph:
11 replies
ATApache TinkerPop
•Created by Vitor Martins on 8/8/2024 in #questions
Optimizing connection between Python API (FastAPI) and Neptune
The query we were doing in the tests is a group count by label:
g.V().groupCount().by(__.label()).next()
Running it locally (connected to Neptune through VPN) takes about 1030ms to run, and running it using the profile step from gremlin gives:
11 replies
ATApache TinkerPop
•Created by Vitor Martins on 8/8/2024 in #questions
Optimizing connection between Python API (FastAPI) and Neptune
Moving on, in some tests we saw that the slowness increase when dealing with concurrent requests/queries. So we decided to do some “load tests” using
ab with different scenarios. The application was running in a k8s cluster with a single pod in the same region of the Neptune.
We experiment across three different characteristics: synchronicity, connection pool and concurrent requests.
For synchronicity we tried with:
- Synchronous queries: https://github.com/aca-so/neptune-poc/tree/sync-routes
- Asynchronous queries using asyncio.to_thread: https://github.com/aca-so/neptune-poc
- Asynchronous queries using asyncio.wrap_future: https://github.com/aca-so/neptune-poc/tree/async-future
For connection pool we tried:
- Without connection pool (Pool size = 1)
- Tests with pool size of 5
- Tests with pool size of 10
For concurrent requests we run:
- Without concurrent requests
- 5 concurrent requests
- 10 concurrent requests
- 20 concurrent requests
- 50 concurrent requests
- 100 concurrent requests
My second question is:
2. "What's the best way to implement asynchronous queries?"11 replies