Space Burger Comments - Answer Overflow

Space Burger

•Created by Space Burger on 12/30/2024 in #⛅｜pods

Is there a limit in the number of threads?

Yeah definitely. But like it should still multithread in time sharing. E.g. Even with 1 vcpu I should be able to get 10 threads. But here it seems that I can get max 2 threads per vcpu

18 replies

RRunPod

•Created by Space Burger on 12/30/2024 in #⛅｜pods

Is there a limit in the number of threads?

I would be happy to run 64 say in parallel. At some point I am hitting ram and vram limits. But that is OK. I don't understand why I am hitting multithread limits when there is still ram and vram available.

18 replies

RRunPod

•Created by Space Burger on 12/30/2024 in #⛅｜pods

Is there a limit in the number of threads?

Yes for some pod I can run like 19 in parallel and for some like 28. It is related to the number of vcpu. But I don't understand how it is related.

18 replies

RRunPod

•Created by Space Burger on 12/30/2024 in #⛅｜pods

Is there a limit in the number of threads?

With vllm serve

18 replies

RRunPod

•Created by Space Burger on 12/30/2024 in #⛅｜pods

Is there a limit in the number of threads?

Yeah. I am running 20 vllms in different screen

18 replies

RRunPod

•Created by Space Burger on 12/30/2024 in #⛅｜pods

Is there a limit in the number of threads?

Thanks. Yeah the problem is that I get it with just 20 vllm in parallel. What do you mean by thread swarming? Should I just spin off a number of threads to see what the limit is?

18 replies

RRunPod

•Created by Space Burger on 12/30/2024 in #⛅｜pods

Is there a limit in the number of threads?

OpenBLAS blas_thread_init: pthread_create failed for thread 57 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 58 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 59 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 60 of 64: Resource temporarily unavailable

OpenBLAS blas_thread_init: pthread_create failed for thread 57 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 58 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 59 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 60 of 64: Resource temporarily unavailable

This is the error I get

18 replies

RRunPod

•Created by Space Burger on 2/18/2024 in #⛅｜pods

screen spot

Also, how long are Spot instance usually be down for?

4 replies

Gaming

Programming