gnarley_farley. Posts - Answer Overflow

gnarley_farley.

Explore posts from servers

•Created by gnarley_farley. on 9/8/2024 in #⚡｜serverless

Upgrading VLLM to v0.6.0

I was wondering what the process would be to upgrade the serverless worker code to be compatible with the latest version of VLLM.

2 replies

RRunPod

•Created by gnarley_farley. on 8/8/2024 in #⚡｜serverless

Charged while not using service

Spun up a serverless api. Did not use it at all. Got billed 60$ since last night. Could you check what caused this behavior. EnerpriseDna Team

3 replies

RRunPod

•Created by gnarley_farley. on 8/8/2024 in #⛅｜pods

Account Drained overnight with nothing running

2 replies

RRunPod

•Created by gnarley_farley. on 8/7/2024 in #⚡｜serverless

How can I cause models to download on initialization?

FROM runpod/pytorch:2.2.1-py3.10-cuda12.1.1-devel-ubuntu22.04
WORKDIR /content
ENV PATH="/home/camenduru/.local/bin:${PATH}"
RUN adduser --disabled-password --gecos '' camenduru && \
    adduser camenduru sudo && \
    echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers && \
    chown -R camenduru:camenduru /content && \
    chmod -R 777 /content && \
    chown -R camenduru:camenduru /home && \
    chmod -R 777 /home

RUN apt update -y && add-apt-repository -y ppa:git-core/ppa && apt update -y && apt install -y aria2 git git-lfs unzip ffmpeg

USER camenduru

RUN pip install -q opencv-python imageio imageio-ffmpeg ffmpeg-python av runpod \
    xformers==0.0.25 torchsde==0.2.6 einops==0.8.0 diffusers==0.28.0 transformers==4.41.2 accelerate==0.30.1

RUN git clone -b flux https://github.com/camenduru/ComfyUI /content/ComfyUI

COPY --chown=camenduru:camenduru ./worker_runpod.py /content/ComfyUI/worker_runpod.py
COPY --chown=camenduru:camenduru ./download_models.sh /content/ComfyUI/download_models.sh

WORKDIR /content/ComfyUI

USER root
RUN chmod +x download_models.sh
USER camenduru

# Create a startup script
RUN echo '#!/bin/bash\n./download_models.sh\npython worker_runpod.py' > /content/ComfyUI/startup.sh
RUN chmod +x /content/ComfyUI/startup.sh

CMD ["/bin/bash", "/content/ComfyUI/startup.sh"]

FROM runpod/pytorch:2.2.1-py3.10-cuda12.1.1-devel-ubuntu22.04
WORKDIR /content
ENV PATH="/home/camenduru/.local/bin:${PATH}"
RUN adduser --disabled-password --gecos '' camenduru && \
    adduser camenduru sudo && \
    echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers && \
    chown -R camenduru:camenduru /content && \
    chmod -R 777 /content && \
    chown -R camenduru:camenduru /home && \
    chmod -R 777 /home

RUN apt update -y && add-apt-repository -y ppa:git-core/ppa && apt update -y && apt install -y aria2 git git-lfs unzip ffmpeg

USER camenduru

RUN pip install -q opencv-python imageio imageio-ffmpeg ffmpeg-python av runpod \
    xformers==0.0.25 torchsde==0.2.6 einops==0.8.0 diffusers==0.28.0 transformers==4.41.2 accelerate==0.30.1

RUN git clone -b flux https://github.com/camenduru/ComfyUI /content/ComfyUI

COPY --chown=camenduru:camenduru ./worker_runpod.py /content/ComfyUI/worker_runpod.py
COPY --chown=camenduru:camenduru ./download_models.sh /content/ComfyUI/download_models.sh

WORKDIR /content/ComfyUI

USER root
RUN chmod +x download_models.sh
USER camenduru

# Create a startup script
RUN echo '#!/bin/bash\n./download_models.sh\npython worker_runpod.py' > /content/ComfyUI/startup.sh
RUN chmod +x /content/ComfyUI/startup.sh

CMD ["/bin/bash", "/content/ComfyUI/startup.sh"]

My internet connection doesn't support uploading massive dockerbuilds to the hub. How can i get the models to download on runpod using a strategy similar to this?

26 replies

Gaming

Programming