streamize
community cloud spot POD
A spot instance suddenly automatically switched to an on-demand instance. Is this normal?
Also, when downloading Docker images, it often fails or becomes slow. (The speed variance when downloading Docker images each time a pod is created is too large (depends on luck)). Is this normal? Is there a way to minimize this?
I host an average of 20 RTX 4090 instances for about 12 hours a day, automatically removing or adding pods to match demand. I'm curious about situations where Docker image downloads suddenly fail and about the behavior of spot instances
5 replies
RRunPod
•Created by streamize on 7/14/2024 in #⚡｜serverless
retrieving queue position for a specific task in RunPod serverless API
Hello, I'm using the RunPod serverless API and I have a question about the following:
Assuming there are 10 jobs in the QUEUE for the serverless setup, is there a way to retrieve, using the RunPod API, the position in the queue for a specific taskId that is waiting? In my app service, I need to fetch the order in which a user's diffusion model will be trained so that I can provide the correct waiting time to the user.
7 replies
RRunPod
•Created by streamize on 5/19/2024 in #⚡｜serverless
runpod serverless start.sh issue
Hello, I'm having an issue with my Dockerfile and start.sh script. At the end of my Dockerfile, I have ENTRYPOINT ["./start.sh"] which runs the start.sh script. This script starts the comfyui server and then runs rp_handler at the end.
When the API is called for the first time, the docker image initializes and the start.sh script successfully starts the comfyui server. However, on the second API call, it seems to skip the contents of start.sh and directly runs rp_handler.py, causing issues.
Here's the content of my start.sh file:
#!/usr/bin/env bash
echo "Worker Initiated"
echo "Symlinking files from Network Volume"
if [ -L "/workspace/ComfyUI" ]; then
echo "/workspace/ComfyUI is already a symlink"
else
ln -s /runpod-volume/ComfyUI /workspace
fi
if [ -L "/workspace/comfyui-venv" ]; then
echo "/workspace/comfyui-venv is already a symlink"
else
ln -s /runpod-volume/comfyui-venv /workspace
fi
echo "Starting ComfyUI API"
if [ -d "comfyui-venv" ]; then
echo "Virtual environment directory exists"
else
echo "Virtual environment directory does not exist"
exit 1
fi
if [ -f "comfyui-venv/bin/activate" ]; then
echo "Activate script exists"
else
echo "Activate script does not exist"
exit 1
fi
source comfyui-venv/bin/activate
if [[ $VIRTUAL_ENV == "/workspace/comfyui-venv" ]]; then
echo "Virtual environment activated successfully"
else
echo "Failed to activate virtual environment"
echo "Content of the activate script:"
cat comfyui-venv/bin/activate
exit 1
fi
export HF_HOME="/workspace"
cd /workspace/ComfyUI && python main.py --port 3000 &
deactivate
echo "Starting RunPod Handler"
python -u /workspace/rp_handler.py
13 replies
RRunPod
•Created by streamize on 5/12/2024 in #⚡｜serverless
dockerless issue
I am using runpod serverless. To reduce deployments, I store the server executable files in a network volume, as described in the link https://blog.runpod.io/runpod-serverless-no-docker-stress/.
I have been using this method for several months, but recently, even when I modify the rp_handler.py file in the network volume, the changes do not seem to be reflected immediately and appear to be cached. As a result, I am currently unable to use it properly.
Are there any recent changes regarding this issue?
13 replies
RRunPod
•Created by streamize on 5/6/2024 in #⚡｜serverless
reduce serverless execution time
My runpod api generation text to image. image generation is just use 1 sec but upload the generated image api call spend 4sec. How to reduce it? We have to provide image file to the user
9 replies
RRunPod
•Created by streamize on 3/26/2024 in #⚡｜serverless
how to prevent restart serverless instance
runpod serverless instance is restarting when error in the code . but I dont want to restarting . how to prevent it?
4 replies
RRunPod
•Created by streamize on 3/14/2024 in #⚡｜serverless
how to know about serverless api region information
I want to get region where the serverless api deployed in the rp_handler.py for upload file to choose the bucket region
6 replies