Problem with long running api endpoint
Hi, can someone advise me on a problem I'm having with a long running api endpoint that works like this:
1. api endpoint is scheduled to run every 90mins and is used to publish data from one database to another.
2. when it runs the endpoint is called and it picks up all database records marked with 'publish=1' field, does some processing, and then calls another api which updates the other database.
One thing I've noticed is that when there are a large number of records (e.g. 13k records) the process just gets slower and slower over until it stops running,
in this case it only processed about 1000 records and now doesn't seem to be picking up any new records, every time i query the database there are still 12k records marked publish=1
This is just a regular endpoint with a get request on a .net 6 api. Does anyone know why this would be happening? Should I be using hangfire or some kind of background process?
5 Replies
does the API return any data, or is it only used to trigger the migrations?
no it doesn't return anything it's just to trigger the migrations
In that case I'd probably use hangfire to trigger your migrations, but your problems with records not being picked up could be unrelated
Do you have any logging built into your app? Any exceptions being thrown? Code to show?
Why can’t you go for background jobs like azure functions or WebJobs and especially you can configure timer so this way in background it process automatically there you can handle your logic
azure functions seems like a good alternative to running the endpoint, as for why it would stop I think you would need to write logs or attach a debugger or something to really figure out why it gets stuck