❔ PostgresSQL how to do couple hundred inserts per second
Hi, I am currently working on a mqtt->DB project for the first time. I am having problems with the DB part as the way I am currently doing it I quickly run out of connection slots (to the DB, using Npgsql), I thought of doing a buffer but the messages have to be in the database more or less in the same second.
5 Replies
If your constant load is higher than your write capacity, this just won't work, whatever you do. The whole purpose of using a message queue is to be able to process from the queue as fast as you can, letting the queue act as the buffer.
But if you never ever "catch up", that wont work
if its just the connections running out, try wrapping the writes in a semaphore - you'll still have the problem of being write capacity capped, but you wont run out of connections
Any other approaches that would solve my problem but still put the data into the DB in realtime?
Not really. High capacity low latency is hard.
as in, I don't know of any. Im sure there are databases that are specifically written to optimize for fast writes, so maybe one of those?
@Pobiega Thank you I'll look into other DB
I am open to other ideas if anybody has them
Was this issue resolved? If so, run
/close - otherwise I will mark this as stale and this post will be archived until there is new activity.