Mount a Host folder as a volume

I am trying to mount a host folder /data/atif inside my workspace using the following settings.

resource "docker_container" "workspace" {
  count = data.coder_workspace.me.start_count
  image = docker_image.coder_image.latest
  cpu_shares = var.cpu
  memory = "${var.ram*1024}"
  runtime = "nvidia"
  gpus = "all"
  # Uses lower() to avoid Docker restriction on container names.
  name = "coder-${data.coder_workspace.me.owner}-${lower(data.coder_workspace.me.name)}"
  # Hostname makes the shell more user friendly: coder@my-workspace:~$
  hostname = lower(data.coder_workspace.me.name)
  dns      = ["1.1.1.1"]
  # Use the docker gateway if the access URL is 127.0.0.1
  command = ["sh", "-c", replace(coder_agent.dev.init_script, "127.0.0.1", "host.docker.internal")]
  env     = ["CODER_AGENT_TOKEN=${coder_agent.dev.token}"]

  host {
    host = "host.docker.internal"
    ip   = "host-gateway"
  }
  volumes {
    container_path = "/home/${data.coder_workspace.me.owner}/"
    volume_name    = docker_volume.home_volume.name
    host_path      = "/data/${data.coder_workspace.me.owner}/"
    read_only      = false
  }
}

resource "docker_container" "workspace" {
  count = data.coder_workspace.me.start_count
  image = docker_image.coder_image.latest
  cpu_shares = var.cpu
  memory = "${var.ram*1024}"
  runtime = "nvidia"
  gpus = "all"
  # Uses lower() to avoid Docker restriction on container names.
  name = "coder-${data.coder_workspace.me.owner}-${lower(data.coder_workspace.me.name)}"
  # Hostname makes the shell more user friendly: coder@my-workspace:~$
  hostname = lower(data.coder_workspace.me.name)
  dns      = ["1.1.1.1"]
  # Use the docker gateway if the access URL is 127.0.0.1
  command = ["sh", "-c", replace(coder_agent.dev.init_script, "127.0.0.1", "host.docker.internal")]
  env     = ["CODER_AGENT_TOKEN=${coder_agent.dev.token}"]

  host {
    host = "host.docker.internal"
    ip   = "host-gateway"
  }
  volumes {
    container_path = "/home/${data.coder_workspace.me.owner}/"
    volume_name    = docker_volume.home_volume.name
    host_path      = "/data/${data.coder_workspace.me.owner}/"
    read_only      = false
  }
}

But I am not seeing anything inside my workspace /home/atif/ What I am doing wrong. The folder already exists on the docker host and have some sub directories and files.

33 Replies

Cian•3y ago

Can you docker inspect the container that gets created?

maf•3y ago

Seems like others have the same issue with the Docker provider: https://github.com/kreuzwerker/terraform-provider-docker/issues/87

GitHub

docker_container volumes host_path ignored · Issue #87 · kreuzwerke...

This issue was originally opened by @schlitzered as hashicorp/terraform-provider-docker#139. It was migrated here as a result of the community provider takeover from @kreuzwerker. The original body...

maf•3y ago

I would start by removing volume_name = docker_volume.home_volume.name It seems weird for me to reference a docker_volume when we're trying to mount the local filesystem into the container.

AtifOP•3y ago

so any suggestions?

maf•3y ago

Did you try removing volume_name?

Phorcys•3y ago

^ I would try this

AtifOP•3y ago

it worked. I can also now safely delete the docker_volume from my template. right?

maf•3y ago

Yep, that should be fine Posted an update on that issue: https://github.com/kreuzwerker/terraform-provider-docker/issues/87#issuecomment-1225988145

AtifOP•3y ago

Doing this makes the /home/coder volatile and all my installed packages vanish after a workspace restart. How can I keep packages installed persistent?

Cian•3y ago

Doing this makes the /home/coder volatile

It sounds like it's doing an overlay mount and not a bind, which is probably what you want

How can I keep packages installed persistent?

If you're using a Docker image, add your packages to the base image

AtifOP•3y ago

EVerything installed by the user should survive workspace restarts I added most of them to the docker image is about 13 GB now. But my users want to build or install additional packages Those should survive restarts

Cian•3y ago

If you run your workspace and then docker inspect the container that it creates, you should see a big JSON object -- can you copy that, redact any information you don't want others to see, and then paste it here?

AtifOP•3y ago

message.txt

Phorcys•3y ago

you should make a volume at the apt folder then otherwise your packages will always disappear

AtifOP•3y ago

some of the packages are installed by pip or conda and some are built from source

Phorcys•3y ago

then change pip packages' dir or mount a volume at their path there's not much else you can do sadly

AtifOP•3y ago

This is my current main.tf

resource "docker_container" "workspace" {
  count = data.coder_workspace.me.start_count
  image = docker_image.coder_image.latest
  cpu_shares = var.cpu
  memory = "${var.ram*1024}"
  runtime = "nvidia"
  gpus = "all"
  # Uses lower() to avoid Docker restriction on container names.
  name = "coder-${data.coder_workspace.me.owner}-${lower(data.coder_workspace.me.name)}"
  # Hostname makes the shell more user friendly: coder@my-workspace:~$
  hostname = lower(data.coder_workspace.me.name)
  dns      = ["1.1.1.1"]
  # Use the docker gateway if the access URL is 127.0.0.1
  command = ["sh", "-c", replace(coder_agent.dev.init_script, "127.0.0.1", "host.docker.internal")]
  env     = ["CODER_AGENT_TOKEN=${coder_agent.dev.token}"]

  host {
    host = "host.docker.internal"
    ip   = "host-gateway"
  }
  volumes {
    container_path = "/home/${data.coder_workspace.me.owner}/data/"
    #volume_name    = docker_volume.home_volume.name
    host_path      = "/data/${data.coder_workspace.me.owner}/"
    read_only      = false
  }
}

resource "docker_container" "workspace" {
  count = data.coder_workspace.me.start_count
  image = docker_image.coder_image.latest
  cpu_shares = var.cpu
  memory = "${var.ram*1024}"
  runtime = "nvidia"
  gpus = "all"
  # Uses lower() to avoid Docker restriction on container names.
  name = "coder-${data.coder_workspace.me.owner}-${lower(data.coder_workspace.me.name)}"
  # Hostname makes the shell more user friendly: coder@my-workspace:~$
  hostname = lower(data.coder_workspace.me.name)
  dns      = ["1.1.1.1"]
  # Use the docker gateway if the access URL is 127.0.0.1
  command = ["sh", "-c", replace(coder_agent.dev.init_script, "127.0.0.1", "host.docker.internal")]
  env     = ["CODER_AGENT_TOKEN=${coder_agent.dev.token}"]

  host {
    host = "host.docker.internal"
    ip   = "host-gateway"
  }
  volumes {
    container_path = "/home/${data.coder_workspace.me.owner}/data/"
    #volume_name    = docker_volume.home_volume.name
    host_path      = "/data/${data.coder_workspace.me.owner}/"
    read_only      = false
  }
}

I am already mounting a host folder inside the workspace as /home/user/data

Phorcys•3y ago

yeah that will only make /home/user/data persist nothing else

AtifOP•3y ago

yes So how to keep everything persist? I mean to survive workspace restarts

Phorcys•3y ago

so you should look at where APT pkgs are installed and where pip/conda you don't you could but it would take way too much space I'm not even sure you could but yeah, just find the locations you want to persist and assign volumes to them

AtifOP•3y ago

ahan How can I add another volume? without affecting the existing one?

Phorcys•3y ago

hmm I don't know maybe add another volumes {} entry that might overwrite the other one but idn idk try it out

AtifOP•3y ago

trying Lets see

  volumes {
    container_path = "/home/${data.coder_workspace.me.owner}/data/"
    #volume_name    = docker_volume.home_volume.name
    host_path      = "/data/${data.coder_workspace.me.owner}/"
    read_only      = false
  }
  volumes {
      container_path = "/home/${data.coder_workspace.me.owner}"
    volume_name    = docker_volume.home_volume.name
    read_only      = false
  }

  volumes {
    container_path = "/home/${data.coder_workspace.me.owner}/data/"
    #volume_name    = docker_volume.home_volume.name
    host_path      = "/data/${data.coder_workspace.me.owner}/"
    read_only      = false
  }
  volumes {
      container_path = "/home/${data.coder_workspace.me.owner}"
    volume_name    = docker_volume.home_volume.name
    read_only      = false
  }

Phorcys•3y ago

I wouldn't do that

  volumes {
    container_path = "/home/${data.coder_workspace.me.owner}/data/"
    #volume_name    = docker_volume.home_volume.name
    host_path      = "/data/${data.coder_workspace.me.owner}/"
    read_only      = false
  }
  volumes {
      container_path = "/home/${data.coder_workspace.me.owner}/otherfolder"
    volume_name    = docker_volume.other_volume.name
    read_only      = false
  }

  volumes {
    container_path = "/home/${data.coder_workspace.me.owner}/data/"
    #volume_name    = docker_volume.home_volume.name
    host_path      = "/data/${data.coder_workspace.me.owner}/"
    read_only      = false
  }
  volumes {
      container_path = "/home/${data.coder_workspace.me.owner}/otherfolder"
    volume_name    = docker_volume.other_volume.name
    read_only      = false
  }

AtifOP•3y ago

This worked ❤️

Phorcys•3y ago

great to know for the future ! so now you'd just have to locate where the folders you need are I would use actual volumes rather than host overlay mounts btw if you need to access the volumes' content it's available at /var/lib/docker/volumes

AtifOP•3y ago

what is the difference? and actual volumes are your preference?

Phorcys•3y ago

yes they are, because docker creates them automatically for you

AtifOP•3y ago

how would I do that? What should I change to achieve that?

Phorcys•3y ago

resource "docker_volume" "home_volume" {
  name = "coder-${data.coder_workspace.me.owner}-${data.coder_workspace.me.name}-root"
}

resource "docker_container" "workspace" {
  volumes {
    container_path = "/home/coder/"
    volume_name    = docker_volume.home_volume.name
    read_only      = false
  }

  # [...]
}

resource "docker_volume" "home_volume" {
  name = "coder-${data.coder_workspace.me.owner}-${data.coder_workspace.me.name}-root"
}

resource "docker_container" "workspace" {
  volumes {
    container_path = "/home/coder/"
    volume_name    = docker_volume.home_volume.name
    read_only      = false
  }

  # [...]
}

so here, the volume for /home/coder will be associated to the coder-<username>-<workspacename>-root docker volume

Phorcys•3y ago

AtifOP•3y ago

Thanks fr the 2nd mount I am using a volume I guess

Phorcys•3y ago

I would use volumes for everything but you do you marking as resolved

Gaming

Programming

Mount a Host folder as a volume