Mihály
RRunPod
•Created by spooky on 10/30/2024 in #⚡｜serverless
jobs queued for minuets despite lots of available idle worker
@yhlong00000
21 replies
RRunPod
•Created by Mihály on 10/17/2024 in #⚡｜serverless
Worker keeps running after finishing job, burning money?
5 replies
RRunPod
•Created by vitalik on 10/10/2024 in #⚡｜serverless
Job retry after successful run
For me, upgrading the SDK from 1.7.1 to ,1.7.2 got rid of the retries
27 replies
RRunPod
•Created by rougsig on 9/27/2024 in #⚡｜serverless
Stuck IN_PROGRESS but job completed and worker exited
Ah, makes sense!
17 replies
RRunPod
•Created by rougsig on 9/27/2024 in #⚡｜serverless
Stuck IN_PROGRESS but job completed and worker exited
I'm not sure i follow 😄
17 replies
RRunPod
•Created by rougsig on 9/27/2024 in #⚡｜serverless
Stuck IN_PROGRESS but job completed and worker exited
Yeah, its
noxhy2en39n3y3 my dev endpoint.17 replies
RRunPod
•Created by rougsig on 9/27/2024 in #⚡｜serverless
Stuck IN_PROGRESS but job completed and worker exited
Hey @deanQ I've upgraded to 1.7.2 (5-6 hours ago) but still getting stuck jobs the same way. 😦
17 replies
RRunPod
•Created by Untrack4d on 9/30/2024 in #⚡｜serverless
Callback Function
Yep, i think this is what you are looking for
https://docs.runpod.io/serverless/endpoints/send-requests#webhooks
3 replies
RRunPod
•Created by rougsig on 9/27/2024 in #⚡｜serverless
Stuck IN_PROGRESS but job completed and worker exited
On my side, it was added to the code after the above issue started happening. Didn't affect the outcomes
17 replies
RRunPod
•Created by rougsig on 9/27/2024 in #⚡｜serverless
Stuck IN_PROGRESS but job completed and worker exited
I dont know if that helps, but the 1.7.1 is more verbose with the errors as well :
"payload_size_bytes": 0 <-- seems sus?
17 replies
RRunPod
•Created by rougsig on 9/27/2024 in #⚡｜serverless
Stuck IN_PROGRESS but job completed and worker exited
17 replies
RRunPod
•Created by rougsig on 9/27/2024 in #⚡｜serverless
Stuck IN_PROGRESS but job completed and worker exited
I'm having the same issue. SDK 1.6.2, 1.7.0 and 1.7.1 all produces this, however very rarely.
Sometimes 1 our of 6, sometimes 1 out of 30.
Re-submitting the same payload can run without issues a second time.
17 replies
RRunPod
•Created by Mihály on 9/10/2024 in #⚡｜serverless
Jobs in queue for a long time, even when there is a worker available
12 replies
RRunPod
•Created by Mihály on 9/10/2024 in #⚡｜serverless
Jobs in queue for a long time, even when there is a worker available
12 replies
RRunPod
•Created by Mihály on 9/10/2024 in #⚡｜serverless
Jobs in queue for a long time, even when there is a worker available
Thank you, will do!
12 replies
RRunPod
•Created by Mihály on 9/10/2024 in #⚡｜serverless
Jobs in queue for a long time, even when there is a worker available
Sure! It was either noxh y2en 39n3 y3 or k5hi ftra iqq8 dw
12 replies
RRunPod
•Created by Mihály on 9/10/2024 in #⚡｜serverless
Jobs in queue for a long time, even when there is a worker available
12 replies
RRunPod
•Created by Mihály on 3/26/2024 in #⚡｜serverless
Failed to return job results.
Update
For later readers with similar problem:
In this specific case, i have found the root of the issue.
It was indeed the response size, however it was not exceeded when returning the final response, but when using the
runpod.serverless.progress_update
Also, in an email conversation, the following was pointer out:
- "limit for /runsync is 20mb"
- "/run has an even smaller limit for 10mb at the moment"
- "its both the input and the output payload limit"8 replies
RRunPod
•Created by Mihály on 3/26/2024 in #⚡｜serverless
Failed to return job results.
ps: Also, i'm uploading the results in code, and cleaning the response of them to avoid this, but still.
8 replies
