Rageus
RRunPod
•Created by Rageus on 7/30/2024 in #⚡｜serverless
Inconsistent 400 Bad Response from sending /run and /runSync.
Good evening everyone.
I am getting the following error message when requesting Runpod's serverless:
response error StatusCode: 400, <html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>cloudflare</center>
</body>
</html>
- My Code runs in AWS Lambda
- I am using /run and /runsync
- It happens very inconsistently, maybe 1/30 requests.
- I retry 5 times with up to 20s delay, but that doesn't fix the problem.
- However, rerunning the failed request locally works 99%+ of the time.
- The lambda might run multiple runpod's serverless /Run requests in parallel if one fails, it does not affect the other runpod requests in the same lambda.
- I checked the request size and the response size, and both are ~1.5 MB, much smaller than the 10 MB /Run limit
- I retried with /runsync after a /run failed with that error that does not seem to fix it.
- replicating the issue locally is much harder than getting it in lambda
- but when it happens, retrying locally doesn't work
- A similar error can be triggered when removing the Host header from the request, but I ensured that it was always set, and the error still happens.
It seems this is a bug.
What surprises me the most is that retrying locally works, but on lambda, it does not.
Maybe there is an issue with the lambda instance the request is running on, such as some IP rate limit.
If anyone has a fix or walkaround, I would appreciate it.
11 replies