RunPod•3w ago

Linking runpod-volume subfolder doesn't work

Hey, I've been trying to create a serverless runpod that has some network volume attached to it. I want to link specific folders from the network volume to the runpod. To do so, im running the following bash file.

echo "Symlinking files from Network Volume"

rm -rf /comfyui/custom_nodes
rm -rf /comfyui/models
rm -rf /venv
rm -rf /logs

ln -s /runpod-volume/ComfyUI/custom_nodes /comfyui/custom_nodes
ln -s /runpod-volume/ComfyUI/models /comfyui/models

cd /comfyui
python main.py --port 3000 > /runpod-volume/logs/comfyui.log 2>&1

echo "Symlinking files from Network Volume"

rm -rf /comfyui/custom_nodes
rm -rf /comfyui/models
rm -rf /venv
rm -rf /logs

ln -s /runpod-volume/ComfyUI/custom_nodes /comfyui/custom_nodes
ln -s /runpod-volume/ComfyUI/models /comfyui/models

cd /comfyui
python main.py --port 3000 > /runpod-volume/logs/comfyui.log 2>&1

So far so good, the runpod starts and seems to load the custom nodes. 🎊 But when trying to load a checkpoint, it fails with the following error :

WARNING path /comfyui/models/checkpoints/sd_xl_base_1-0.safetensors exists but doesn't link anywhere, skipping.
!!! Exception during processing !!! Model in folder 'checkpoints' with filename 'sd_xl_base_1-0.safetensors' not found.

WARNING path /comfyui/models/checkpoints/sd_xl_base_1-0.safetensors exists but doesn't link anywhere, skipping.
!!! Exception during processing !!! Model in folder 'checkpoints' with filename 'sd_xl_base_1-0.safetensors' not found.

Traceback (most recent call last):
  File "/comfyui/execution.py", line 327, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "/comfyui/execution.py", line 202, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "/comfyui/execution.py", line 174, in _map_node_over_list
[...]
FileNotFoundError: Model in folder 'checkpoints' with filename 'sd_xl_base_1-0.safetensors' not found.

Traceback (most recent call last):
  File "/comfyui/execution.py", line 327, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "/comfyui/execution.py", line 202, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "/comfyui/execution.py", line 174, in _map_node_over_list
[...]
FileNotFoundError: Model in folder 'checkpoints' with filename 'sd_xl_base_1-0.safetensors' not found.

When I open the network storage via some other pod I can correctly see the sd_xl_base_1-0.safetensors in it.. If anyone has already faced this issue, I would be very thankful for some help 😊

4 Replies

xeith_OP•2w ago

Update: it works locally with a docker compose

services:
  comfy-ui-serverless:
    image: comfy-ui-serverless
    build:
      context: .
      dockerfile: Dockerfile
    container_name: comfy-ui-serverless
    volumes:
      # Map a local directory to /runpod-volume inside the container
      - ./storage/runpod-volume:/runpod-volume
      # Runpod test with the test-input.json file
      - ./test_input.json:/test_input.json
    ports:
      # Expose any necessary ports (adjust if your application uses specific ports)
      - "3000:3000" # Example: Replace 8000 with the actual port if needed
    environment:
      # Pass any environment variables you need
      NVIDIA_VISIBLE_DEVICES: all
      NVIDIA_DRIVER_CAPABILITIES: compute,utility
    runtime: nvidia

services:
  comfy-ui-serverless:
    image: comfy-ui-serverless
    build:
      context: .
      dockerfile: Dockerfile
    container_name: comfy-ui-serverless
    volumes:
      # Map a local directory to /runpod-volume inside the container
      - ./storage/runpod-volume:/runpod-volume
      # Runpod test with the test-input.json file
      - ./test_input.json:/test_input.json
    ports:
      # Expose any necessary ports (adjust if your application uses specific ports)
      - "3000:3000" # Example: Replace 8000 with the actual port if needed
    environment:
      # Pass any environment variables you need
      NVIDIA_VISIBLE_DEVICES: all
      NVIDIA_DRIVER_CAPABILITIES: compute,utility
    runtime: nvidia

So I guess it has something to do with how runpod works with network volumes ? Any help is appreciated :))

flash-singh•2w ago

we can plan to support this through config, symlinks have limitation maybe this is one of them with network storage

xeith_OP•2w ago

@flash-singh Hey, I was about to update this thread. Actually /comfyui/models/checkpoints/sd_xl_base_1-0.safetensors exists was a symlink ITSELF that what pointing nowhere... I deleted it, re-imported the model in my network storage and everything works fine. The only limitation is that it's REALLY slow to load onto the VRAM, so I wont rely on the network storage for big files like such anyway 🐋

flash-singh•2w ago

yes that will be slower, local disk is much faster

Gaming

Programming

Linking runpod-volume subfolder doesn't work

Did you find this page helpful?