Integration Testing for APIs that interact with a Third-Party System
I have built an API (using FastAPI) where it's only job is to collect data from a Third-Party System via Webhooks, process that data then store it into my MySQL DB using prisma.
from these Events that get sent to my API from this external system, depending on the event, I have a series of API requests back to this Third-Party System to actually gather the data that is a part of that specific event from that system (don't ask why that data isn't just present in the webhook event data, I have no idea.)
That being said, I'm trying to move forward with building out a testing framework (using pytest) to make sure that everything works and continues to work.
I'm having issues understanding how to go about doing this so I'm hoping someone could break down where my thinking is going wrong and help better explain things for my specific scenario
(in comments... too many words)
Totally understand there's a lot to read here. If it's just one bullet point you respond to, even that would be super helpful. Thank you for your time!!
1 Reply
- Mocking data. From my understanding it is generally regarded as bad practice to implement "mock data" from an API service. So, in my case where all my data is being delivered from a Third-Party System and that's all my API is listening for. doing things this way requires me to not only mock the Webhook request data letting me there was an update but also mock the data from the API request I make right after.
- if I try to go about not mocking data, my Integration tests have to use the API in order to set up conditions so that a test can run. (but these set up conditions are never used in my API itself only in the tests)
- for example, lets say my test is to get a project and all data associated with that project from the external system. the set up condition would be that a project needs to be created inside that external system.
- in my scenario here, I am never needing to "create a project" in the external system from with in my API.
- is this okay to do?
- to add, there might be 15 different "fields" that go into every project that need to also be tested. which would mean that for those instances I would need to create those for the events that update the project. It seems like a "heavy" set up process with an external API service where I'm only using these creation endpoints for testing? It seems like I'm creating more of a headache than I am preventing them. so maybe I'm going about this the wrong way?
- Most things in this external system require everything to be associated to a project (as which I have mimicked in my database design)
- so a "note" object cannot exist without a project. how am I supposed to adhere to this in tests where everything is supposed to exist in isolation? or is that not a concern for integration tests?
- my other thought is that I don't even know how it would be possible to unit test everything considering how dependent my API is on this external system (but again, I'm probably doing something wrong) and the whole existing in isolation thing if that's only a unit test thing.