yasyf
RRunPod
•Created by yasyf on 9/26/2024 in #⚡｜serverless
524 Timeouts when waiting for new serverless messages
oh I'm not using VLLM, I meant the
concurrency_modifier11 replies
RRunPod
•Created by yasyf on 9/26/2024 in #⚡｜serverless
524 Timeouts when waiting for new serverless messages
its 4 right now. whats recommended value?
11 replies
RRunPod
•Created by yasyf on 9/26/2024 in #⚡｜serverless
524 Timeouts when waiting for new serverless messages
yes, using LLMs. ok cool, will keep an eye out for that. anything else to do in the interim?
11 replies
RRunPod
•Created by yasyf on 9/26/2024 in #⚡｜serverless
524 Timeouts when waiting for new serverless messages
and the result is the idle timeout is not expected and the worker stays alive longer than it should
11 replies
RRunPod
•Created by yasyf on 9/26/2024 in #⚡｜serverless
524 Timeouts when waiting for new serverless messages
yea I understand all of that, but the 524 happens very reproducibly and very frequently, so I dont think its a temp network problem
11 replies