RunPod•11mo ago

Problem with venv

Hi, This is my handler.py file:

#!/usr/bin/env python3
""" Example handler file. """

import runpod
import subprocess

# If your handler runs inference on a model, load the model here.
# You will want models to be loaded into memory before starting serverless.


def handler(job):
    
    return run_shell_script('./entrypoint.sh') 

def run_shell_script(script_path):
    try:
        result = subprocess.run([script_path], check=True, text=True, capture_output=True)
        return result.stdout
    except subprocess.CalledProcessError as e:
        return e.stderr


runpod.serverless.start({"handler": handler})

#!/usr/bin/env python3
""" Example handler file. """

import runpod
import subprocess

# If your handler runs inference on a model, load the model here.
# You will want models to be loaded into memory before starting serverless.


def handler(job):
    
    return run_shell_script('./entrypoint.sh') 

def run_shell_script(script_path):
    try:
        result = subprocess.run([script_path], check=True, text=True, capture_output=True)
        return result.stdout
    except subprocess.CalledProcessError as e:
        return e.stderr


runpod.serverless.start({"handler": handler})

when entrypoint.sh has this code:

export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin

ls

export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin

ls

it works fine and gives

{
  "delayTime": 3917,
  "executionTime": 1082,
  "id": "3d4277ce-31d8-442c-830b-8817be24ecae-e1",
  "output": "Activate.ps1\naccelerate\naccelerate-config\naccelerate-estimate-memory\naccelerate-launch\nactivate\nactivate.csh\nactivate.fish\nconvert-caffe2-to-onnx\nconvert-onnx-to-caffe2\ndistro\nf2py\nhttpx\nhuggingface-cli\nisympy\nnormalizer\nopenai\npip\npip3\npip3.11\npython\npython3\npython3.11\nspacy\ntorchrun\ntqdm\ntransformers-cli\nweasel\n",
  "status": "COMPLETED"
}

{
  "delayTime": 3917,
  "executionTime": 1082,
  "id": "3d4277ce-31d8-442c-830b-8817be24ecae-e1",
  "output": "Activate.ps1\naccelerate\naccelerate-config\naccelerate-estimate-memory\naccelerate-launch\nactivate\nactivate.csh\nactivate.fish\nconvert-caffe2-to-onnx\nconvert-onnx-to-caffe2\ndistro\nf2py\nhttpx\nhuggingface-cli\nisympy\nnormalizer\nopenai\npip\npip3\npip3.11\npython\npython3\npython3.11\nspacy\ntorchrun\ntqdm\ntransformers-cli\nweasel\n",
  "status": "COMPLETED"
}

Solution:

Finally works, thanks for all the help! The issue was the base image that was different.

Jump to solution

18 Replies

DaanOP•11mo ago

If it has this code:

#!/bin/bash

export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin
pip list

#!/bin/bash

export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin
pip list

it gives

{
  "delayTime": 6319,
  "executionTime": 1100,
  "id": "6f292b30-3562-463d-8f66-26a5355d9cf4-e1",
  "output": "./entrypoint.sh: /workspace/.venv/bin/pip: /workspace/.venv/bin/python: bad interpreter: No such file or directory\n",
  "status": "COMPLETED"
}

{
  "delayTime": 6319,
  "executionTime": 1100,
  "id": "6f292b30-3562-463d-8f66-26a5355d9cf4-e1",
  "output": "./entrypoint.sh: /workspace/.venv/bin/pip: /workspace/.venv/bin/python: bad interpreter: No such file or directory\n",
  "status": "COMPLETED"
}

Any idea where it goes wrong? I understood that runpod already chose the mount path as "runpod-volume", that is why I did "rm -rf /workspace && ln -s /runpod-volume /workspace". Also, when executing pip list in a pod, ade with the same network volume, no errors.

ashleyk•11mo ago

Does your docker image for your serverless worker have the same Python version as the one you used to create the venv?

DaanOP•11mo ago

I think it is, this is the docker image used to create the venv (i made a pod with this templatte and manually created the venv):

# Gebruik een basisimage met Python 3.11
FROM python:3.11

# Stel de werkdirectory in op de root om te beginnen
WORKDIR /

# Kopieer de entrypoint  in de container
COPY ./entrypoint.sh /entrypoint.sh

# Maak de scripts uitvoerbaar
RUN chmod +x /entrypoint.sh

# Installeer vim
#RUN apt-get update && apt-get install -y vim

# Stel het entrypoint script in als het entrypoint van de container
ENTRYPOINT ["/entrypoint.sh"]

# Gebruik een basisimage met Python 3.11
FROM python:3.11

# Stel de werkdirectory in op de root om te beginnen
WORKDIR /

# Kopieer de entrypoint  in de container
COPY ./entrypoint.sh /entrypoint.sh

# Maak de scripts uitvoerbaar
RUN chmod +x /entrypoint.sh

# Installeer vim
#RUN apt-get update && apt-get install -y vim

# Stel het entrypoint script in als het entrypoint van de container
ENTRYPOINT ["/entrypoint.sh"]

and this is the one to make serverless endpoint:

# Base image -> https://github.com/runpod/containers/blob/main/official-templates/base/Dockerfile
# DockerHub -> https://hub.docker.com/r/runpod/base/tags
FROM runpod/base:0.4.0-cuda11.8.0

# The base image comes with many system dependencies pre-installed to help you get started quickly.
# Please refer to the base image's Dockerfile for more information before adding additional dependencies.
# IMPORTANT: The base image overrides the default huggingface cache location.


# --- Optional: System dependencies ---
# COPY builder/setup.sh /setup.sh
# RUN /bin/bash /setup.sh && \
#     rm /setup.sh


# Python dependencies
COPY builder/requirements.txt /requirements.txt
RUN python3.11 -m pip install --upgrade pip && \
    python3.11 -m pip install --upgrade -r /requirements.txt --no-cache-dir && \
    rm /requirements.txt

# NOTE: The base image comes with multiple Python versions pre-installed.
#       It is reccommended to specify the version of Python when running your code.


# Add src files (Worker Template)
ADD src .

CMD python3.11 -u /handler.py

# Base image -> https://github.com/runpod/containers/blob/main/official-templates/base/Dockerfile
# DockerHub -> https://hub.docker.com/r/runpod/base/tags
FROM runpod/base:0.4.0-cuda11.8.0

# The base image comes with many system dependencies pre-installed to help you get started quickly.
# Please refer to the base image's Dockerfile for more information before adding additional dependencies.
# IMPORTANT: The base image overrides the default huggingface cache location.


# --- Optional: System dependencies ---
# COPY builder/setup.sh /setup.sh
# RUN /bin/bash /setup.sh && \
#     rm /setup.sh


# Python dependencies
COPY builder/requirements.txt /requirements.txt
RUN python3.11 -m pip install --upgrade pip && \
    python3.11 -m pip install --upgrade -r /requirements.txt --no-cache-dir && \
    rm /requirements.txt

# NOTE: The base image comes with multiple Python versions pre-installed.
#       It is reccommended to specify the version of Python when running your code.


# Add src files (Worker Template)
ADD src .

CMD python3.11 -u /handler.py

ashleyk•11mo ago

Where are you from @Daan ? Netherlands?

DaanOP•11mo ago

I live in Belgium 🙂

ashleyk•11mo ago

Ah, thats why I can read it, I can't really understand Dutch 😆 Is the Docker image public? Or is it private?

DaanOP•11mo ago

it is private, but I can make it public, one second daana2003/orca-gpu-runner:no_vim_10 daana2003/orca-gpu-runner:serverless_60

ashleyk•11mo ago

Is the first one for GPU cloud and the second one for serverless?

DaanOP•11mo ago

Yes, indeed Made the first one to manually make a .venv in worspace directory

ashleyk•11mo ago

I am getting this error with the first image:

2024-01-02T15:16:53.747731452Z Wachten op app.sh om te bestaan...
2024-01-02T15:16:53.747750310Z /entrypoint.sh: line 14: .venv/bin/activate: No such file or directory

2024-01-02T15:16:53.747731452Z Wachten op app.sh om te bestaan...
2024-01-02T15:16:53.747750310Z /entrypoint.sh: line 14: .venv/bin/activate: No such file or directory

DaanOP•11mo ago

strange, I didn't had that error. I will retry it

ashleyk•11mo ago

Maybe your .venv folder already existed in /workspace

DaanOP•11mo ago

oh, no it is because you do not use my network volume. I use a network volume that is mounted in /workspace

ashleyk•11mo ago

Yeah seems like it expects certain things to already exist on the network volume

DaanOP•11mo ago

Yes indeed, so I made manually a venv and app.sh file. My script works fine then if I execute it in the GPU. But in the serverless endpoint, with the other image it goes wrong. I really can't understand why

export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin

ls

export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin

ls

works, but then if i change ls by "pip list" it gives the error that pip is not found...

ashleyk•11mo ago

Yeah there seems to be an issue with the venv, which sometimes happens when the python versions are not the same

DaanOP•11mo ago

I can try to add "FROM python:3.11" to the dockerfile for teh sereverless endpoint,and delete "RUN python3.11 -m pip install --upgrade pip", then it certainly would be the same version, right? Although I would exspect both images use python3.11 with the provided code

Solution

Daan•11mo ago

Finally works, thanks for all the help! The issue was the base image that was different.

Gaming

Programming

Problem with venv