Optimizing MQTT Communication for High-Frequency Data Transmission with ESP32
Hey guys, while setting up communication between
ESP32 and industrial/DIY sensors using
MQTT, I noticed performance bottlenecks, message loss, and delayed logging, especially during high-frequency data transmission. How do different MQTT buffering techniques and architectural patterns compare when dealing with high data throughput in systems with network instability? What strategies can ensure message reliability and minimize delays when transmitting large volumes of sensor data from ESP32 to the MQTT broker?
2 Replies
Hmm, I think one thing that might help is adjusting the QoS level in MQTT for more reliable message delivery, though it can add some overhead. Have you also tried implementing message buffering on the ESP32 side??? That could smooth out transmission during network instability. And depending on your sensor frequency, using an async publishing model might help reduce delays too
Adjusting QoS levels, message buffering, and async publishing make sense. Also I was thinking of using a dynamic buffer (10-50 KB) and FreeRTOS tasks/Arduino AsyncMQTT library. Is this a good idea, or do you have an idea of a better alternative?
Thanks for the suggestions tho🙏