RunPod•12mo ago

serverless: any way to figure out what gpu type a job ran on?

trying to get data on speeds across gpu types for our jobs, and i'm wondering if the api exposes this anywhere, and if not, what the best way to sort it out would be.

16 Replies

Unknown User•12mo ago

Message Not Public

flash-singh•12mo ago

dont need nvidia-smi, we expose an env variable to the worker with the gpu name in it, look at our docs for env variable names

nerdylive•12mo ago

is this what you're referring to?

nerdylive•12mo ago

i dont think thats the one, after browsing for some mins i still cant be able to find it please send the docs url for that

Unknown User•12mo ago

Message Not Public

nerdylive•12mo ago

what is the env variable key/name for it? yeah lets just wait for support im too lazy changing and rebuilding my image for that

Unknown User•12mo ago

Message Not Public

nerdylive•12mo ago

ye lol

ashleyk•12mo ago

Those are GPU cloud environment variables, these are serverless ones:

RUNPOD_WEBHOOK_POST_STREAM=https://api.runpod.ai/v2/12345657890/job-stream/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_ENDPOINT_ID=mpoacd7wrmv2fc
RUNPOD_CPU_COUNT=6
RUNPOD_POD_ID=p8btjjjjq865pi
RUNPOD_GPU_SIZE=AMPERE_24
RUNPOD_MEM_GB=62
RUNPOD_GPU_COUNT=1
RUNPOD_VOLUME_ID=hbsp3mav9e
RUNPOD_POD_HOSTNAME=p8btjjjjq865pi-64410f26
RUNPOD_DEBUG_LEVEL=INFO
RUNPOD_ENDPOINT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_DC_ID=EU-RO-1
RUNPOD_AI_API_ID=mpoacd7wrmv2fc
RUNPOD_AI_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_WEBHOOK_GET_JOB=https://api.runpod.ai/v2/12345657890/job-take/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_PING=https://api.runpod.ai/v2/12345657890/ping/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_POST_OUTPUT=https://api.runpod.ai/v2/12345657890/job-done/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_PING_INTERVAL=4000
CUDA_VERSION=11.8.0
NV_CUDNN_VERSION=8.9.6.50

RUNPOD_WEBHOOK_POST_STREAM=https://api.runpod.ai/v2/12345657890/job-stream/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_ENDPOINT_ID=mpoacd7wrmv2fc
RUNPOD_CPU_COUNT=6
RUNPOD_POD_ID=p8btjjjjq865pi
RUNPOD_GPU_SIZE=AMPERE_24
RUNPOD_MEM_GB=62
RUNPOD_GPU_COUNT=1
RUNPOD_VOLUME_ID=hbsp3mav9e
RUNPOD_POD_HOSTNAME=p8btjjjjq865pi-64410f26
RUNPOD_DEBUG_LEVEL=INFO
RUNPOD_ENDPOINT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_DC_ID=EU-RO-1
RUNPOD_AI_API_ID=mpoacd7wrmv2fc
RUNPOD_AI_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_WEBHOOK_GET_JOB=https://api.runpod.ai/v2/12345657890/job-take/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_PING=https://api.runpod.ai/v2/12345657890/ping/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_POST_OUTPUT=https://api.runpod.ai/v2/12345657890/job-done/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_PING_INTERVAL=4000
CUDA_VERSION=11.8.0
NV_CUDNN_VERSION=8.9.6.50

Doesn't seem to have the GPU type set directly but you can get it from the end of RUNPOD_WEBHOOK_POST_STREAM, RUNPOD_WEBHOOK_GET_JOB, RUNPOD_WEBHOOK_PING, RUNPOD_WEBHOOK_POST_OUTPUT as shown above.

nerdylive•12mo ago

nice thanks eh isnt this the one RUNPOD_GPU_SIZE=AMPERE_24

ashleyk•12mo ago

Not sure how that translates to L4 though

nerdylive•12mo ago

Ah ye no full gpu type

flash-singh•12mo ago

looks like it isn't, i will plan to add it, this is good to expose in pod env

nerdylive•12mo ago

Wow you're a Rp developer

ashleyk•12mo ago

He isn't just a developer, he is the CTO 😉

nerdylive•12mo ago

Wow that's amazing he's still handling the supports lol

Gaming

Programming

serverless: any way to figure out what gpu type a job ran on?