Sarcagian
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
I think I just did it haha. I'll post details later. Need sleep.
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
Hahahaha
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
I realized though, I've learned a ton of good info I didn't know two days ago throughout trying to solve this problem lol. So not all bad.
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
Oh man, I had just sworn off continuing to pursue this and just waiting for official support. And then you throw this at me lol. Now I'm gonna have to go back at it at least a little today.
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
I'm beyond angry right now, I've been at this since this morning
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
I realize that it takes time to develop this stuff but it's extremely frustrating that nvidia would release a new architecture, charge thousands of dollars for the GPU, and not support this part of the community especially to help develop support for sm120 and cuda 12.8
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
well I made some serious progress on getting vLLM working and built for blackwell, but after all that, it seems there's no way to compile xformers to work with torch 2.8.x dev builds so I wont be able to use models like gemma3
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
gotcha
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
can vllm server multiple models simultaneously or dynamically unload/load different models as needed?
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
FYI dont build it in develop mode, it failed on the last step, starting over again lol
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
oh wow haha
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
not having anything close to that building locally
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
oh wow lol, I wonder why the RAM usage is so high?
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
I've only got like 25GB of RAM used up at tthe moment, that's odd it failed with that much system RAM
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
oof
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
hahaha
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
no, just irresponsible with what I buy haha
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
sec
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
no idea, havent looked at the logs yet
320 replies
RRunPod
•Created by Sarcagian on 4/6/2025 in #⚡｜serverless
Serverless Requests Queuing Forever
I've got plenty haha, more than that
320 replies