Creating endpoint through runpodclt
Hello there. I am trying to test and deploy a basic serverless endpoint. Issues that I have and don't understand:
- If I create a serverless endpoint through the web, then I don't need to select a network volume. In this case I think I am forced to use Docker.
- Because I don't want to explicitly use docker, iam using runpodclt as described:
https://blog.runpod.io/runpod-dockerless-cli-innovation/
I created a new project with:
gpu_types = [
"NVIDIA GeForce RTX 4080", # 16GB
"NVIDIA RTX A4000",
"NVIDIA RTX A4500", # 16GB
"NVIDIA RTX A4500 Ada", # 16GB
"NVIDIA RTX A5000", # 24GB
]
I created a new network volume in the CA-MTL-1 region, which, according to the web page has a high A5000 availability.
But when I call runpodctl project dev I get:
name = "test-project"
[endpoint]
active_workers = 0
flashboot = true
max_workers = 3
[project]
base_image = "runpod/base:0.6.1-cuda11.8.0"
container_disk_size_gb = 30
gpu_count = 1
gpu_types = ["NVIDIA GeForce RTX 4080", "NVIDIA RTX A4000", "NVIDIA RTX A4500", "NVIDIA RTX A4500 Ada", "NVIDIA RTX A5000"]
ports = "4040/http, 7270/http, 22/tcp"
uuid = "8777bf75"
volume_mount_path = "/runpod-volume"
[project.env_vars]
POD_INACTIVITY_TIMEOUT = "120"
RUNPOD_DEBUG_LEVEL = "debug"
UVICORN_LOG_LEVEL = "warning"
[runtime]
handler_path = "src/handler.py"
python_version = "3.10"
requirements_path = "builder/requirements.txt"
Deploying project Pod on RunPod...
Trying to get a Pod with NVIDIA GeForce RTX 4080... Unavailable.
Trying to get a Pod with NVIDIA RTX A4000... Unavailable.
Trying to get a Pod with NVIDIA RTX A4500... Unavailable.
Trying to get a Pod with NVIDIA RTX A4500 Ada... Unavailable.
Trying to get a Pod with NVIDIA RTX A5000... Unavailable.
none of the selected GPU types were available
Can you help me?
It is possible to avoid having to select a region when creating a network volume? Thans
RunPod Blog
RunPod's Latest Innovation: Dockerless CLI for Streamlined AI Devel...
Discover the future of AI development with RunPod's Dockerless CLI tool. Experience seamless deployment, enhanced performance, and intuitive design, revolutionizing how you bring AI projects from concept to reality.
3 Replies
RunPod uses docker images for all of it's pods/workers. Even if you create deploy and endpoint through runpodcclt or the web interface you will still require a docker image. A network volume should be optional in either case.
Yes, for that I said not using docker explicitly.
The network volume seems to be mandatory, otherwhise: "No network volumes found. Please create one and try again. (https://runpod.io/console/user/storage)"
when calling runpodclt project dev
I suggest submitting a ticket for that. Network volume is optional. You can do so from RunPod main site under Help/Contact.