nerdylive

PostsComments

RRunPod

•Created by NGTK on 11/18/2024 in #⚡｜serverless

Not using cached worker

Maybe the flashboot doesn't work everytime and keep it "warm" for a long time if your request are sporadic, or it's your endpoint scaling type that starts a new worker

4 replies

RRunPod

•Created by NGTK on 11/18/2024 in #⚡｜serverless

Not using cached worker

Is it like 1 second difference or it loaded like fron a long time ago?

4 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Your welcome bro

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

I heard it's good too

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Try sglang if you like to

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Interesting, never checked that

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Oh on vllm? Or vllm-worker

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Hmm maybe, or the gpu hardware, ssd is slower on runpod, depending on how the app works

82 replies

RRunPod

•Created by 살려주세요 on 11/18/2024 in #⛅｜pods

I am not using GPU, but someone else is occupying my GPU. What is the solution?

Ah you haven't run anything andthe memory is already used?

11 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Is it slower

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Oh what about in runpod ?

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

I used A40 and like 1.5b only, pretty fast

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Ah it's still slow?

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Maybe I never tested performance with or without that

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Or other cards

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Maybe the performance will be different

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

Hmm have you tested in other pods?

82 replies

RRunPod

•Created by Thibaud on 8/13/2024 in #⛅｜pods

vllm seems not use GPU

What llm model?

82 replies

RRunPod

•Created by Amir on 11/17/2024 in #⛅｜pods

Stuck on Pod Initialization

See if there's any request failure

4 replies

RRunPod

•Created by Amir on 11/17/2024 in #⛅｜pods

Stuck on Pod Initialization

Maybe check in network tab in your browser, open by pressing f12 then switch to network tab

4 replies