Deployments Failing on Health Check Timeout

All deployments are failing after 5 minutes. The # of healthcheck attempts never rises above 2. It's not clear whether this means the service never comes up (which would be surprising as the code hasn't really changed) or whether the healthcheck daemon process is dying. In either case we're blocked from all deployments and need help.
11 Replies
Percy
Percy2y ago
Project ID: 2db6fbc5-2ff6-48d5-b635-66e792767de5
Percy
Percy2y ago
It appears that the healthcheck daemon process is timing out after 5 minutes, causing the deployments to fail. To increase the timeout, change the number of seconds on the service settings page. Build Logs include detailed healthcheck failure logs and Deploy logs for failed deployments, allowing healthcheck debugging.
⚠️ experimental feature
nootrality
nootrality2y ago
2db6fbc5-2ff6-48d5-b635-66e792767de5
angelo
angelo2y ago
Hey @nootrality - going to raise this to the team
Percy
Percy2y ago
Flagging this thread. A team member will be with you shortly.
angelo
angelo2y ago
You didn't change the path it tests against?
nootrality
nootrality2y ago
No
angelo
angelo2y ago
Gotcha, raised to the team. But I am looking at DD logs to see if I can catch anything.
nootrality
nootrality2y ago
thank you
angelo
angelo2y ago
My guess is that we aren't living the application properly so a non 200 is just going to return Added a var, one sec Okay, not that. One sec Okay, my back of napkin theory that I need to confirm (the eng who worked on that feature is out so going to get someone else to look at it) But before I get there: did you recently add new domains?
nootrality
nootrality2y ago
no the only thing that has changed in the past hour (since it went down) is the code and the code is basic react junk -- nothing significant oh, latest deployment just succeeded (!) hopefully that's not transient