RunPod•11mo ago

Attaching a Network Volume fails when using GraphQL

I have created a Network Volume and would like to start a container with the volume attached. It works without problem in the web UI after clicking the "Deploy" button. However, when using runpod-python's method create_pod, GraphQL endpoint returns the following error: There are no longer any instances available with the requested specifications. Please refresh and try again. (I have tried multiple times) Here is the minimal code:

pod = runpod.create_pod(
        "worker",
        "myorg/worker:latest",
        "NVIDIA GeForce RTX 3070",
        network_volume_id=args.network_volume_id,
    )

pod = runpod.create_pod(
        "worker",
        "myorg/worker:latest",
        "NVIDIA GeForce RTX 3070",
        network_volume_id=args.network_volume_id,
    )

args.network_volume_id is probably correct because when I alter it in any way the error message changes to Something went wrong. Please try again later or contact support. The generated GraphQL query looks like this:

mutation {
      podFindAndDeployOnDemand(
        input: {
          name: "worker", imageName: "myorg/worker:latest", gpuTypeId: "NVIDIA GeForce RTX 3070", cloudType: ALL, startSsh: true, supportPublicIp: true, dataCenterId: "EU-RO-1", gpuCount: 1, volumeInGb: 0, containerDiskInGb: 10, minVcpuCount: 1, minMemoryInGb: 1, dockerArgs: "", volumeMountPath: "/runpod-volume", networkVolumeId: "l6y182wxyv"
        }
      ) {
        id
        desiredStatus
        imageName
        env
        machineId
        machine {
          podHostId
        }
      }
    }

mutation {
      podFindAndDeployOnDemand(
        input: {
          name: "worker", imageName: "myorg/worker:latest", gpuTypeId: "NVIDIA GeForce RTX 3070", cloudType: ALL, startSsh: true, supportPublicIp: true, dataCenterId: "EU-RO-1", gpuCount: 1, volumeInGb: 0, containerDiskInGb: 10, minVcpuCount: 1, minMemoryInGb: 1, dockerArgs: "", volumeMountPath: "/runpod-volume", networkVolumeId: "l6y182wxyv"
        }
      ) {
        id
        desiredStatus
        imageName
        env
        machineId
        machine {
          podHostId
        }
      }
    }

Could you please help me to attach the network volume?

3 Replies

ashleyk•11mo ago

You probably need to change cloudType to SECURE. Here is an example from someone else: https://discord.com/channels/912829806415085598/1184480647994167407 You probably also need to remove this, its most likely going to cause issues:

volumeInGb: 0

volumeInGb: 0

gQsoQaOP•11mo ago

Thank you for your response! I changed cloudType to SECURE and removed the volumeInGb field, but the error message still stays the same. The query now looks like this:

mutation {
      podFindAndDeployOnDemand(
        input: {
          name: "worker", imageName: "myorg/worker:latest", gpuTypeId: "NVIDIA GeForce RTX 3070", cloudType: SECURE, startSsh: true, supportPublicIp: true, dataCenterId: "EU-CZ-1", gpuCount: 1, containerDiskInGb: 10, minVcpuCount: 1, minMemoryInGb: 1, dockerArgs: "", volumeMountPath: "/runpod-volume", networkVolumeId: "l6y182wxyv"
        }
      ) {
        id
        desiredStatus
        imageName
        env
        machineId
        machine {
          podHostId
        }
      }
    }

mutation {
      podFindAndDeployOnDemand(
        input: {
          name: "worker", imageName: "myorg/worker:latest", gpuTypeId: "NVIDIA GeForce RTX 3070", cloudType: SECURE, startSsh: true, supportPublicIp: true, dataCenterId: "EU-CZ-1", gpuCount: 1, containerDiskInGb: 10, minVcpuCount: 1, minMemoryInGb: 1, dockerArgs: "", volumeMountPath: "/runpod-volume", networkVolumeId: "l6y182wxyv"
        }
      ) {
        id
        desiredStatus
        imageName
        env
        machineId
        machine {
          podHostId
        }
      }
    }

Deploying without the network volume (omitting the network_volume_id parameter) works fine. I think I've identified the issue, thanks again for your help! First of all, cloudType: SECURE was needed as you suggested. After that, the error was not caused by the network volume but rather by the GPU (unfortunately the error message is quite vague). This happened because runpod-python automatically sets dataCenterId to the datacenter of the network volume, which is different than my default. This means that attaching the volume changed the datacenter to a different one where the concrete GPU was not available.

ashleyk•11mo ago

Yeah the error messages from GraphQL are pretty horrible, its like trying to solver a murder mystery to get to the root of the actual problem.

Gaming

Programming

Attaching a Network Volume fails when using GraphQL