youcef_ali
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
@Joseph Ogbonna You can use buffering, each time you send message check if it was tranmitted or not if not put in a buffer (Queue,file in sd card for example) once connetion restablished ( wifi / internet connextion mqtt connection) send all the untrasmitted mssages in your queue
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
We use something like history packets. Packers are stored in a lifo stack and it will be appended with the live packets and sent as history packets . Server separately handles history packets and stores accordingly
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
Preventing data loss on ESP32 during MQTT disconnects
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
Yes, both mqtt server and the client can handle these chunks as long as the payload size doesn't exceed. I use Mosquitto Broker which supports a maximum payload size of 268435455 bytes
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
Then this will be handled on iot server side
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
I will break down the file into chunks in binary format and then publish them separately. And then merge all the data again when it finishes sending the last chunk
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
thank you
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
You can do that in the firmware, maybe you can a file for each dataset and then in sequence
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
How would you handle data if it is saved in file will you chuck all data to mqtt server at once? After reading the file back
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
This situation could be handled by adding SDCard or using the built-in Flash memory of ESP32 for local storage to ensure data will not be lost during network issues. And, then implement the Retry Mechanism to reconnect ESP32 back to the network
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
@Umesh Lokhande
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
The type of data here is also to be considered as well
14 replies
DIIDevHeads IoT Integration Server
•Created by youcef_ali on 9/24/2024 in #edge-networking
Preventing data loss on ESP32 during MQTT disconnects
I haven't done something like this, but maybe you can put a backup storage in the device that can store the data when the WiFi is disconnected. Then after it gains back connection it sends the stored data to the cloud.That will be of a great help.
14 replies