szeth4401
Skypilot + Runpod: No resource satisfying the request
Hi team. I'm trying to use Skypilot + vllm+ Runpod to serve a custom trained LLM. I cannot make the skypilot to launch a resource. I get the following error:
I 02-22 00:16:32 optimizer.py:1206] No resource satisfying <Cloud>({'NVIDIA RTX A6000': 1}, ports=['8888']) on RunPod.
sky.exceptions.ResourcesUnavailableError: Catalog does not contain any instances satisfying the request:
I tried numerous GPU ids and none worked. Please see below my skypilot yaml file.
service:
readiness_probe: /v1/models
replicas: 1
resources:
ports: 8888
accelerators: {NVIDIA RTX A6000:1} <-- have tried A10G, A100, etc, nothing works.
setup: |
conda create -n vllm python=3.9 -y
conda activate vllm
pip install vllm
run: |
conda activate vllm
python -m vllm.entrypoints.openai.api_server \
--tensor-parallel-size $SKYPILOT_NUM_GPUS_PER_NODE \
--host 0.0.0.0 --port 8888 \
--model mistralai/....
What am I doing wrong?
Thanks
7 replies