What is the best practice for large data insertion and updates
Hello, I have kind of large data that I'm getting from external API. I need to insert this data in my database and update it nightly with a cron job. i'd like to know what's the best practice to do it
4 Replies
Research bulk instead options for whatever database tech you’re using. For some there are multiple options.
How large are you talking?
Maybe SqlBulkCopy if you're using sql? It's pretty straightforward to use
I'm doing an identical thing at my job and it's been a good option so far
Do you get all the data from single call to external API ? Or you just keep loading the data during the day through external API ?
if you get all the data from single call, bulk insert is okay to use, If you are partially gathering the data, should consider different approaches
Thank you guys, yes i've used bulkinsert (just saw the messages) didnt got notifications.