Http reverse proxy disconnects
Hi, I have noticed that when we send an API request to our Pod, we are getting a 524 response back if we send through the reverse proxy and the inference job takes over ~20 seconds. This does not happen when we use direct tcp. But with that method, we have to handle dynamic ports. This is happening on all of our pods running this job. Mostly 4090s
1 Reply
this is in EUR-IS-1