AdamOH
RRunPod
•Created by AdamOH on 5/21/2024 in #⚡｜serverless
OSError: [Errno 9] Bad file descriptor on all requests
i already have, but they are so slow to respond.
6 replies
RRunPod
•Created by AdamOH on 5/21/2024 in #⚡｜serverless
OSError: [Errno 9] Bad file descriptor on all requests
yes
6 replies
RRunPod
•Created by AdamOH on 3/7/2024 in #⚡｜serverless
Possible memory leak on Serverless
It's a very popular model and I can't find any other people complaining about this problem so I can only assume its related to the docker container or hardware. I'm using the runpod/worker-vllm:0.3.0-cuda11.8.0 image which is also popular and I haven't found anyone complaining about this issue of empty messages after X number of messages. It seems like it must be hardware related as the messages being sent are all very similar to each other, yet after a while it just starts returning "\r\n\r\n..." in a long string
9 replies
RRunPod
•Created by AdamOH on 2/29/2024 in #⚡｜serverless
Serverless GPU low capacity
if i was to bake the models into an image, would that mean i wouldn't need to use a network volume, and therefore the workers could load up in any region where there is available capacity? i can't find any definitive info on the benefits of volumes vs baked images
7 replies
RRunPod
•Created by AdamOH on 2/29/2024 in #⚡｜serverless
Serverless GPU low capacity
can you copy a volume from one region to another?
7 replies