RunPod•13mo ago

accelerate launch best --num_cpu_threads_per_process value ?

Hi guys, I try to do some lora training on a serverless endpoint and I wonder how many cpu cores are available with the different GPU types? Is there a specification on that somewhere? And / or what do you use? My first tests ran on a single thread but would love to maximize performance. 🙂

Solution:

You can use this environment variable:

RUNPOD_CPU_COUNT=6

RUNPOD_CPU_COUNT=6

...

Jump to solution

3 Replies

Solution

ashleyk•13mo ago

You can use this environment variable:

RUNPOD_CPU_COUNT=6

RUNPOD_CPU_COUNT=6

derRaabOP•13mo ago

Thank you so much! Somehow I never saw https://docs.runpod.io/docs/pod-env-variables . 🤦🏻

RunPod

Pod Environment Variables

Environment variables are accessible within in a pod. You can access this page by clicking on the menu icon and Edit Pod.

ashleyk•13mo ago

serverless environment variables are a bit different

RUNPOD_WEBHOOK_POST_STREAM=https://api.runpod.ai/v2/12345657890/job-stream/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_ENDPOINT_ID=mpoacd7wrmv2fc
RUNPOD_CPU_COUNT=6
RUNPOD_POD_ID=p8btjjjjq865pi
RUNPOD_GPU_SIZE=AMPERE_24
RUNPOD_MEM_GB=62
RUNPOD_GPU_COUNT=1
RUNPOD_VOLUME_ID=hbsp3mav9e
RUNPOD_POD_HOSTNAME=p8btjjjjq865pi-64410f26
RUNPOD_DEBUG_LEVEL=INFO
RUNPOD_ENDPOINT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_DC_ID=EU-RO-1
RUNPOD_AI_API_ID=mpoacd7wrmv2fc
RUNPOD_AI_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_WEBHOOK_GET_JOB=https://api.runpod.ai/v2/12345657890/job-take/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_PING=https://api.runpod.ai/v2/12345657890/ping/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_POST_OUTPUT=https://api.runpod.ai/v2/12345657890/job-done/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_PING_INTERVAL=4000
CUDA_VERSION=11.8.0
NV_CUDNN_VERSION=8.9.6.50

RUNPOD_WEBHOOK_POST_STREAM=https://api.runpod.ai/v2/12345657890/job-stream/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_ENDPOINT_ID=mpoacd7wrmv2fc
RUNPOD_CPU_COUNT=6
RUNPOD_POD_ID=p8btjjjjq865pi
RUNPOD_GPU_SIZE=AMPERE_24
RUNPOD_MEM_GB=62
RUNPOD_GPU_COUNT=1
RUNPOD_VOLUME_ID=hbsp3mav9e
RUNPOD_POD_HOSTNAME=p8btjjjjq865pi-64410f26
RUNPOD_DEBUG_LEVEL=INFO
RUNPOD_ENDPOINT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_DC_ID=EU-RO-1
RUNPOD_AI_API_ID=mpoacd7wrmv2fc
RUNPOD_AI_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_WEBHOOK_GET_JOB=https://api.runpod.ai/v2/12345657890/job-take/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_PING=https://api.runpod.ai/v2/12345657890/ping/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_POST_OUTPUT=https://api.runpod.ai/v2/12345657890/job-done/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_PING_INTERVAL=4000
CUDA_VERSION=11.8.0
NV_CUDNN_VERSION=8.9.6.50

Gaming

Programming

accelerate launch best --num_cpu_threads_per_process value ?