wafa_ath
wafa_ath
DIIDevHeads IoT Integration Server
Created by wafa_ath on 9/27/2024 in #firmware-and-baremetal
Improving Gesture Recognition Consistency on ESP32 with TinyML
Hello, I'm working on a gesture recognition project using TinyML on an ESP32 with an accelerometer (MPU6050). My goal is to detect specific gestures (e.g., wave, swipe) using machine learning. I trained a model using Edge Impulse and successfully deployed it onto the ESP32. However, when I run the inference code, I get inconsistent results, and sometimes the output is incorrect even when performing the same gesture. Occasionally, the ESP32 throws the following error:
E (1155) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
- IDLE (CPU 0)
- ml_task
Tasks currently running:
CPU 0: ml_task
CPU 1: IDLE
E (1155) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
- IDLE (CPU 0)
- ml_task
Tasks currently running:
CPU 0: ml_task
CPU 1: IDLE
Here is the inference code I'm running on the ESP32:
from machine import Pin, I2C
from mpu6050 import MPU6050
import time
import numpy as np
from tinyml_model import predict_gesture # Edge Impulse generated model

# Initialize MPU6050
i2c = I2C(0, scl=Pin(22), sda=Pin(21))
mpu = MPU6050(i2c)

# Function to collect data from MPU6050
def get_sensor_data():
accel = mpu.accel
return np.array([accel.x, accel.y, accel.z])

# Main loop for gesture recognition
while True:
try:
data = get_sensor_data()
gesture = predict_gesture(data) # Run inference on collected data

if gesture == "wave":
print("Wave gesture detected!")
elif gesture == "swipe":
print("Swipe gesture detected!")
else:
print("No gesture detected")

time.sleep(0.5) # Delay to avoid rapid re-inference

except Exception as e:
print("Error:", e)
time.sleep(1)
from machine import Pin, I2C
from mpu6050 import MPU6050
import time
import numpy as np
from tinyml_model import predict_gesture # Edge Impulse generated model

# Initialize MPU6050
i2c = I2C(0, scl=Pin(22), sda=Pin(21))
mpu = MPU6050(i2c)

# Function to collect data from MPU6050
def get_sensor_data():
accel = mpu.accel
return np.array([accel.x, accel.y, accel.z])

# Main loop for gesture recognition
while True:
try:
data = get_sensor_data()
gesture = predict_gesture(data) # Run inference on collected data

if gesture == "wave":
print("Wave gesture detected!")
elif gesture == "swipe":
print("Swipe gesture detected!")
else:
print("No gesture detected")

time.sleep(0.5) # Delay to avoid rapid re-inference

except Exception as e:
print("Error:", e)
time.sleep(1)
I suspect the issue might be related to timing or resource limitations on the ESP32. How can I fix the watchdog error and improve the consistency of gesture detection? Should I adjust the sampling rate, modify the inference loop, or implement additional optimizations for TinyML on the ESP32?
4 replies