On inserting many, many rows
In the documentation, the following insert with multiple rows is described:
In a project of mine, an insert query grew to 10k rows to be inserted at once. So, I was wondering, if it is good practice to throw this whole thing at the insert query or if I am responsible to batch the inserts to smaller more reasonable chunks? I am using Postgres. Thank you for any hint on this topic.
4 Replies
👋 you should batch, it will fail for sure with such amount of rows.
So there is no batching logic behing the insert call? It probably has to do with it not necessarily being a transaction 🤔
No, Drizzle just passes your query to the driver with no interference.
You could use a transaction and batch inserts using it though.
I use https://lodash.com/docs/4.17.15 because I am lazy 😆
I am already in a transaction, so yeah, batching shouldn't be too complicated. A mention of insert limits would be nice for the drizzle docs
Thank you!
Ah! Just hit the hard ceiling! "Error: MAX_PARAMETERS_EXCEEDED: Max number of parameters (65534) exceeded"