Why too long delay time even if I have active worker ?

I have set the active worker to 1. I am manually testing the response delay. I submit the next task only after the previous task is completed, so there is no waiting time. However, many times, the delay time is still very long, sometimes even reaching more than 4 seconds. Why is this? In my code, the model has been loaded before runpod.serverless.start({"handler": run})