can 3 different serverless workers running from same network volume?
Hi @digigoblin
I have checked your answer about symbol linking network volume dir to serverless dir and run worker from the network volume as it did as if it was a separate pod instance.
https://github.com/ashleykleynhans/runpod-worker-comfyui/blob/main/start.sh#L5-L7
I fully understand it and thank you for the instruction. Nevertheless, if I am running 3 serverless workers, all of them will launch comfyui main script from the same network volume folder and read and write the ./input and ./output directory or the big model files simultaneously from the same volume. is this a problem in terms of the performance, or do you have a best practice for such a typical scenario where multiple workers sharing same source directory because we do not want to copy big models and working directory for each of these workers.
thank you so much and looking forward to your kindly response.
GitHub
runpod-worker-comfyui/start.sh at main · ashleykleynhans/runpod-wor...
RunPod Serverless Worker for the ComfyUI Stable Diffusion API - ashleykleynhans/runpod-worker-comfyui
48 Replies
i dont think thats a problem with input and output to the same dir, dont copy the models. just use the right paths, or symlink works too
thank you @nerdylive for your prompt response
no problem!
for the starter.sh I cited, I saw the comfyUI server is running within the venv (the code is listed below), while the rp_handler.py is running out of the venv. is this the mandate from runpod serverless? or can I run the python3 -u /rp_handler.py also within the venv.
echo "Starting ComfyUI API"
source /workspace/venv/bin/activate
TCMALLOC="$(ldconfig -p | grep -Po "libtcmalloc.so.\d" | head -n 1)"
export LD_PRELOAD="${TCMALLOC}"
export PYTHONUNBUFFERED=true
export HF_HOME="/workspace"
cd /workspace/ComfyUI
python main.py --port 3000 > /workspace/logs/comfyui.log 2>&1 &
deactivate
echo "Starting RunPod Handler"
python3 -u /rp_handler.py
the reason why I am asking, I have installed all the dependencies in the venv, trying to avoid more installing operation in the dockerfile only for rp_handler.py
@nerdylive sorry for my ignorance to runpod serverless dev
Its up to you, you can run it inside or outside of venv as long as its run
so it is not a mandatory to run rp_handler outside of venv?
No
even I run it within the venv in the docker, the serverless will hook it up
Serverless doesn't care how you run it, as long as as serverless.start() is called.
got it, thank you @digigoblin you are the hero
additional silly question @digigoblin I have deployed the customized serverless workers running on network volume, but got one error here
2024-06-12T12:06:12.254441486Z Worker Initiated
2024-06-12T12:06:12.254454526Z Symlinking files from Network Volume
2024-06-12T12:06:12.256156489Z Activating Virtual Environment
2024-06-12T12:06:12.260177702Z runpod-worker-comfy: Starting ComfyUI
2024-06-12T12:06:12.260310884Z runpod-worker-comfy: Starting RunPod Handler
2024-06-12T12:06:12.261887625Z /start.sh: line 23: /workspace/ComfyUI/logs/comfyui.log: No such file or directory
2024-06-12T12:06:12.270993788Z Traceback (most recent call last):
2024-06-12T12:06:12.271003338Z File "/rp_handler.py", line 1, in <module>
2024-06-12T12:06:12.271021379Z import runpod
2024-06-12T12:06:12.271028059Z ModuleNotFoundError: No module named 'runpod'
I have installed runpod module using pip3 install runpod in the venv already, why it still asking for this installation.
my start.sh script is very simple like this
have you activated venv before calling the rp_handler.py?
yes
did you activate the venv before installing runpod?
yes
I did
its weird then if you did. it should be there
is it attached to the network volume?
yes, all the files on the network volume is there accessible to the docker. they works fine
let me do it again.
let me activate and install runpod module again
any reason you're storing venv in workspace? just root install it in docker container
will just make it slower since network volume is slower than container's disk
network volume should be used for stuff like model weights
by the way, the only way to debug serverless docker is I build the docker image locally, push it to the docker hub, and create template activate worker from runpod? it is very slow. is there better way to change the docker @nerdylive
the custom nodes in comfyui needs it, when you install one it installs its deps
well its up to him too, ease of use when installing
haha, I have installed the whole comfy, dependancies, custom node on a network volume, use it as a separate pod. I do not want to install all of these again it would be a nightmare, I do not know what did I installed and where 😄
edit the template Or
use a pod
oh lol I use a1111 , didn't know comfyui has such a strange system
yeah, custom nodes are from community mostly, so they have much more customizeability and well other dependencies
lucky you dear
this was such a pain that I ended up buying my own PC with 4070ti just so I could test stuff with cuda lol
got it. thank you so much @nerdylive
yeas
You probably didn't install runpod module into the venv
maybe, I am doing it as instructed
The repo I gave you above does not use venv for runpod module
but he says "I have installed runpod module using pip3 install runpod in the venv already, why it still asking for this installation."
I did it mannually I thought. but not sure, I am doing it again
GitHub
runpod-worker-comfyui/start.sh at main · ashleykleynhans/runpod-wor...
RunPod Serverless Worker for the ComfyUI Stable Diffusion API - ashleykleynhans/runpod-worker-comfyui
It deactivates venv then has runpod module installed into the system not venv, so does not look like you are using this
Just use this, it works, then you don't have to fiddle around for hours
here i see no deactivate
yeah its different
Don't think he is using the repo still trying to get it to work
haha you've guided 2 ppl to do that today nice
just use the repo because its wasting too much time to keep helping with this and getting nowhere
yeah thats working from the tutorial on the github
I did not use the original because i do not want to install dependancy in the docker for the rp_handler.py. so I will make all dependancies in the venv and run both comfyui and rp_handler.py all within the venv in the worker
Oh is this someone else, I thought it was same person, my bad.
yeah same case, using comfyui worker
Don't know why you need to do this
dw, its small not really huge
ok, I will mannual check this again. if still has the problem, I will rollback to use the orinial version of start.py and use the original dockerfile as will
If you want to use the venv for the handler, install runpod module into the venv
The repo doesn't do that since runpod module is installed into the system and run from there
understood sir