jackson
RRunPod
•Created by Thibaud on 8/20/2024 in #⚡｜serverless
SGLang
i ran with command:
/bin/bash -c "python3 -m sglang.launch_server --model-path NousResearch/Hermes-3-Llama-3.1-8B --context-length 8192 --quantization fp8 --host 0.0.0.0 --port 8000 --tp 8 --nccl-init sgl-dev-0:50000 --nnodes 2 --node-rank 0"
and got error mesage: [W909 10:25:24.935506246 socket.cpp:697] [c10d] The IPv6 network addresses of (sgl-dev-0, 50000) cannot be retrieved (gai error: -2 - Name or service not known)120 replies
RRunPod
•Created by Thibaud on 8/20/2024 in #⚡｜serverless
SGLang
hi guys, can i run multiple nodes on runpod? example: two nodes with 4 GPUs on each node
120 replies