Space Burger
Space Burger
PostsComments
RRunPod
Created by Space Burger on 12/30/2024 in #⛅｜pods-clusters
Is there a limit in the number of threads?
Yeah definitely. But like it should still multithread in time sharing. E.g. Even with 1 vcpu I should be able to get 10 threads. But here it seems that I can get max 2 threads per vcpu
18 replies
RRunPod
Created by Space Burger on 12/30/2024 in #⛅｜pods-clusters
Is there a limit in the number of threads?
I would be happy to run 64 say in parallel. At some point I am hitting ram and vram limits. But that is OK. I don't understand why I am hitting multithread limits when there is still ram and vram available.
18 replies
RRunPod
Created by Space Burger on 12/30/2024 in #⛅｜pods-clusters
Is there a limit in the number of threads?
Yes for some pod I can run like 19 in parallel and for some like 28. It is related to the number of vcpu. But I don't understand how it is related.
18 replies
RRunPod
Created by Space Burger on 12/30/2024 in #⛅｜pods-clusters
Is there a limit in the number of threads?
With vllm serve
18 replies
RRunPod
Created by Space Burger on 12/30/2024 in #⛅｜pods-clusters
Is there a limit in the number of threads?
Yeah. I am running 20 vllms in different screen
18 replies
RRunPod
Created by Space Burger on 12/30/2024 in #⛅｜pods-clusters
Is there a limit in the number of threads?
Thanks. Yeah the problem is that I get it with just 20 vllm in parallel. What do you mean by thread swarming? Should I just spin off a number of threads to see what the limit is?
18 replies
RRunPod
Created by Space Burger on 12/30/2024 in #⛅｜pods-clusters
Is there a limit in the number of threads?
OpenBLAS blas_thread_init: pthread_create failed for thread 57 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 58 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 59 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 60 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: pthread_create failed for thread 57 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 58 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 59 of 64: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 60 of 64: Resource temporarily unavailable
This is the error I get
18 replies
RRunPod
Created by Space Burger on 2/18/2024 in #⛅｜pods-clusters
screen spot
Also, how long are Spot instance usually be down for?
4 replies