RunPod•7mo ago

how do you create a compatible docker file?

I want to run a custom docker file, but I'm not sure how to make one that's compatible. for example when I use this to create an image that's saved to my registry, the pod seems to start but I can't connect to it over ssh. I noticed that if I picked an official pytorch pod I had checkmarks for ssh and jupter lab, but not if I use my custom one. What's the minimal dockerfile I need to run?

FROM pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime

RUN apt-get update --yes && \
    apt install --yes --no-install-recommends git wget curl bash libgl1 software-properties-common openssh-server nginx && \
    apt-get autoremove -y && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* && \
    echo "en_US.UTF-8 UTF-8" > /etc/locale.gen

EXPOSE 8888
EXPOSE 22

FROM pytorch/pytorch:2.2.2-cuda12.1-cudnn8-runtime

RUN apt-get update --yes && \
    apt install --yes --no-install-recommends git wget curl bash libgl1 software-properties-common openssh-server nginx && \
    apt-get autoremove -y && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* && \
    echo "en_US.UTF-8 UTF-8" > /etc/locale.gen

EXPOSE 8888
EXPOSE 22

12 Replies

justin•7mo ago

https://github.com/justinwlin/Runpod-GPU-And-Serverless-Base

GitHub

GitHub - justinwlin/Runpod-GPU-And-Serverless-Base

Contribute to justinwlin/Runpod-GPU-And-Serverless-Base development by creating an account on GitHub.

justin•7mo ago

I just made one u can take a look here that is meant for testing on gpu pod before deploying on serverless. but u could just ignore the “serverless” pod if u want expose port 8888 for http for jupyter notebook, and expose port 22 for tcp The way I recommend to make it compatible is use a runpod pytorch template as a base And comes with a lot of additional bonus stuff I usually start with pytorch runpod template on gpu pod, run through some steps manually, and then basically start with a FROM from runpod template, and add what i ran manually This github repo, u can for the dockerfile, just: 1) Add stuff to the requirements.txt 2) Append whatever other libraries and commands u want to run / install 3) I override the default start.sh script by runpod to have some better launching configuration options

justin•7mo ago

Here is a more advance repo that uses the above github repo to demonstrate the difference after customizing it more https://github.com/justinwlin/Runpod-OpenLLM-Pod-and-Serverless

GitHub

GitHub - justinwlin/Runpod-OpenLLM-Pod-and-Serverless: A repo for O...

A repo for OpenLLM to run pod. Contribute to justinwlin/Runpod-OpenLLM-Pod-and-Serverless development by creating an account on GitHub.

nerdylive•7mo ago

SSH, jupyter lab those services must be installed manually because they arent actually included in the base image

justin•7mo ago

^yea, u need to install openssh. why if u use the runpod template as basis ull get to skip a lot of annoying stuff

waspinatorOP•7mo ago

when I use

FROM runpod/pytorch:2.2.1-py3.10-cuda12.1.1-devel-ubuntu22.04

I get this error when trying to use the gpu locally.

UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)

UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)

I don't get that error when using the official pytorch image. Have you seen that before?

Madiator2011•7mo ago

It's most likely that your system is running on cuda version older than 12.1

nerdylive•7mo ago

yeah use the cuda filter and select 12.1 and later

Madiator2011 (Work)•7mo ago

@nerdylive though the user tries to run local 😄

nerdylive•7mo ago

Lol How did you figure that out

digigoblin•7mo ago

"I get this error when trying to use the gpu locally."

nerdylive•7mo ago

Oh yeah Maybe update graphics driver

Gaming

Programming

how do you create a compatible docker file?