How can I cause models to download on initialization?

FROM runpod/pytorch:2.2.1-py3.10-cuda12.1.1-devel-ubuntu22.04
WORKDIR /content
ENV PATH="/home/camenduru/.local/bin:${PATH}"
RUN adduser --disabled-password --gecos '' camenduru && \
adduser camenduru sudo && \
echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers && \
chown -R camenduru:camenduru /content && \
chmod -R 777 /content && \
chown -R camenduru:camenduru /home && \
chmod -R 777 /home

RUN apt update -y && add-apt-repository -y ppa:git-core/ppa && apt update -y && apt install -y aria2 git git-lfs unzip ffmpeg

USER camenduru

RUN pip install -q opencv-python imageio imageio-ffmpeg ffmpeg-python av runpod \
xformers==0.0.25 torchsde==0.2.6 einops==0.8.0 diffusers==0.28.0 transformers==4.41.2 accelerate==0.30.1

RUN git clone -b flux https://github.com/camenduru/ComfyUI /content/ComfyUI

COPY --chown=camenduru:camenduru ./worker_runpod.py /content/ComfyUI/worker_runpod.py
COPY --chown=camenduru:camenduru ./download_models.sh /content/ComfyUI/download_models.sh

WORKDIR /content/ComfyUI

USER root
RUN chmod +x download_models.sh
USER camenduru

# Create a startup script
RUN echo '#!/bin/bash\n./download_models.sh\npython worker_runpod.py' > /content/ComfyUI/startup.sh
RUN chmod +x /content/ComfyUI/startup.sh

CMD ["/bin/bash", "/content/ComfyUI/startup.sh"]
FROM runpod/pytorch:2.2.1-py3.10-cuda12.1.1-devel-ubuntu22.04
WORKDIR /content
ENV PATH="/home/camenduru/.local/bin:${PATH}"
RUN adduser --disabled-password --gecos '' camenduru && \
adduser camenduru sudo && \
echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers && \
chown -R camenduru:camenduru /content && \
chmod -R 777 /content && \
chown -R camenduru:camenduru /home && \
chmod -R 777 /home

RUN apt update -y && add-apt-repository -y ppa:git-core/ppa && apt update -y && apt install -y aria2 git git-lfs unzip ffmpeg

USER camenduru

RUN pip install -q opencv-python imageio imageio-ffmpeg ffmpeg-python av runpod \
xformers==0.0.25 torchsde==0.2.6 einops==0.8.0 diffusers==0.28.0 transformers==4.41.2 accelerate==0.30.1

RUN git clone -b flux https://github.com/camenduru/ComfyUI /content/ComfyUI

COPY --chown=camenduru:camenduru ./worker_runpod.py /content/ComfyUI/worker_runpod.py
COPY --chown=camenduru:camenduru ./download_models.sh /content/ComfyUI/download_models.sh

WORKDIR /content/ComfyUI

USER root
RUN chmod +x download_models.sh
USER camenduru

# Create a startup script
RUN echo '#!/bin/bash\n./download_models.sh\npython worker_runpod.py' > /content/ComfyUI/startup.sh
RUN chmod +x /content/ComfyUI/startup.sh

CMD ["/bin/bash", "/content/ComfyUI/startup.sh"]
My internet connection doesn't support uploading massive dockerbuilds to the hub. How can i get the models to download on runpod using a strategy similar to this?
16 Replies
nerdylive
nerdylive4mo ago
So you call the download inside the script /content/ComfyUI/startup.sh it will be called every runtime right make sure you add logic to check if some file exists it wont download or something alike
gnarley_farley.
gnarley_farley.OP4mo ago
Trying to launch this on serverless worker https://github.com/camenduru/flux.1-dev-tost
GitHub
GitHub - camenduru/flux.1-dev-tost
Contribute to camenduru/flux.1-dev-tost development by creating an account on GitHub.
nerdylive
nerdylive4mo ago
. Did you do that And what's not working btw
gnarley_farley.
gnarley_farley.OP4mo ago
Yeah thanks I saw your message. I am a bit stupid. First time using runpod Nothing downloads
nerdylive
nerdylive4mo ago
I'm going to sleep soon sorry hope someone can check on this
gnarley_farley.
gnarley_farley.OP4mo ago
I think I have something wrong in my setup Yeah me too lol. Been trying all night
nerdylive
nerdylive4mo ago
But you can use wget or any download command in your start script
gnarley_farley.
gnarley_farley.OP4mo ago
If you get a chance later, would be much appreciated
nerdylive
nerdylive4mo ago
Hahaha of course yeah I'll try to
gnarley_farley.
gnarley_farley.OP4mo ago
❤️
NERDDISCO
NERDDISCO4mo ago
What is your actual use case for using ComfyUI with Flux? Do you want an API to use ComfyUI or do you want to play around with Flux? Because if you are doing this in serverless, then every time your endpoint is shut down as there is no activity, you will then have to download the models again when a request comes in. So I would first love to know what your goal is with Flux! Then we can take a look on how we can achieve this.
gnarley_farley.
gnarley_farley.OP4mo ago
I just want to use it as an API, sent a request, get some images. Same as one would do with the comfyui runpod worker/serverless I was trying to use a download script within my code rather than pushing all the files to docker at once. hence the setup. Not sure if it's required though. If anyone has a working image would happily use that(flux).
nerdylive
nerdylive4mo ago
So I think it's best to use network storage then
NERDDISCO
NERDDISCO4mo ago
I was working on the image for runpod-worker-comfy yesterday and it seems it is working. Will have to update the README + provide an example workflow. Then I will ping you!
NERDDISCO
NERDDISCO4mo ago
@gnarley_farley. both FLUX.1 dev and schnell are released (including example workflows): https://github.com/blib-la/runpod-worker-comfy?tab=readme-ov-file#quickstart
GitHub
GitHub - blib-la/runpod-worker-comfy: ComfyUI as a serverless API o...
ComfyUI as a serverless API on RunPod. Contribute to blib-la/runpod-worker-comfy development by creating an account on GitHub.
NERDDISCO
NERDDISCO4mo ago
For the other idea related to download models when the serverless endpoint does init: https://github.com/blib-la/runpod-worker-comfy/issues/60
GitHub
Download models on the fly · Issue #60 · blib-la/runpod-worker-comfy
Is your feature request related to a problem? Please describe. The Docker images are very huge and users would like to just use the base image (which doesn't contain any models). This would mak...
Want results from more Discord servers?
Add your server