C
Coder.com•2y ago
Sharp

host volumes from dind on kubernetes

https://files.mcaq.me/8919b.jpg Here is the terraform: https://hastebin.com/ipayakinex.nginx Basically, I want the docker-compose file to, when run, correctly mount from the container, not host. This is not happening. Any advice would be greatly appreciated :))
Hastebin: Send and Save Text or Code Snippets for Free | Toptal®
Hastebin is a free web-based pastebin service for storing and sharing text and code snippets with anyone. Get started now.
14 Replies
thy
thy•2y ago
this is likely because your docker daemon is running in a sidecar so it's mounting the test folder in the sidecar, not your workspace
Sharp
Sharp•2y ago
Yeah, that is what I thought. I think if I had access to the sysbox method, it would be fine, but I don't Have you got any other suggetions? Maybe at this point I move away from k3s and try and find another lightweight k8s equivalent that can run sysbox
thy
thy•2y ago
if you don't need to provision on >1 node you could just use docker to spawn workspaces
Sharp
Sharp•2y ago
Can I get DIND to run in the same container by making a funky image?
thy
thy•2y ago
unless this is a PoC for a larger k8s deploy
Sharp
Sharp•2y ago
I don't right now - this is more me using this as an oppertunity to learn k8s
thy
thy•2y ago
you can do dind if you make the workspace container privileged, instead of a sidecar if that's acceptable
Sharp
Sharp•2y ago
Yeah, that is acceptable for me If it comes to running it in a bigger org, I won't have the k3s problem and I can use sysbox (Which should work fine?) Is there anything complicated to adding dind to the codercom enterprize image that I should know before I start googling?
thy
thy•2y ago
yeah, sysbox is basically a wrapper around a privileged container to make it safer
thy
thy•2y ago
i don't think dind is doing anything really special so it should work just fine on our image, if it doesn't work you can try taking something from here: https://github.com/docker-library/docker/blob/023a45d6e2a9009fdf01da87499c6f9ca6e27bef/23.0-rc/dind/Dockerfile
GitHub
docker/Dockerfile at 023a45d6e2a9009fdf01da87499c6f9ca6e27bef · doc...
Docker Official Image packaging for Docker. Contribute to docker-library/docker development by creating an account on GitHub.
Sharp
Sharp•2y ago
Okay. Cheers :)) Hopefully sysbox + k3s support isn't too far away Yeah your default image worked with a simple sudo dockerd & in the terraform init script. Thanks for the helping hand. One step closer to getting this done :^)
thy
thy•2y ago
glad to hear! reach back out if you have any more questions 😃
Sharp
Sharp•2y ago
ERROR: failed to create cluster: command "docker run --name armada-test-control-plane --hostname armada-test-control-plane --label io.x-k8s.kind.role=control-plane --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run --volume /var --volume /lib/modules:/lib/modules:ro -e KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER --detach --tty --label io.x-k8s.kind.cluster=armada-test --net kind --restart=on-failure:1 --init=false --publish=0.0.0.0:5001:80/TCP --publish=0.0.0.0:11000:11000/TCP --publish=0.0.0.0:6443:6443/TCP --publish=127.0.0.1:39719:6443/TCP -e KUBECONFIG=/etc/kubernetes/admin.conf kindest/node:v1.24.7" failed with error: exit status 125
Command Output: docker: Error response from daemon: error creating aufs mount to /var/lib/docker/aufs/mnt/4174d5bb9f2a1448da29aa20f3be3d5bca2815afe8cebf9cc5ba4ed8cff81510-init: mount target=/var/lib/docker/aufs/mnt/4174d5bb9f2a1448da29aa20f3be3d5bca2815afe8cebf9cc5ba4ed8cff81510-init data=br:/var/lib/docker/aufs/diff/4174d5bb9f2a1448da29aa20f3be3d5bca2815afe8cebf9cc5ba4ed8cff81510-init=rw:/var/lib/docker/aufs/diff/9ca52ece048bb0cd42ae58e359d91b16694d5081c1697c2a328547a028b17cc0=ro+wh:/var/lib/docker/aufs/diff/df9eb543f133e61a6942529dc2ec7135ec6385f59fe862bbe031e3f7876b909d=ro+wh,dio,xino=/dev/shm/aufs.xino: invalid argument.
ERROR: failed to create cluster: command "docker run --name armada-test-control-plane --hostname armada-test-control-plane --label io.x-k8s.kind.role=control-plane --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run --volume /var --volume /lib/modules:/lib/modules:ro -e KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER --detach --tty --label io.x-k8s.kind.cluster=armada-test --net kind --restart=on-failure:1 --init=false --publish=0.0.0.0:5001:80/TCP --publish=0.0.0.0:11000:11000/TCP --publish=0.0.0.0:6443:6443/TCP --publish=127.0.0.1:39719:6443/TCP -e KUBECONFIG=/etc/kubernetes/admin.conf kindest/node:v1.24.7" failed with error: exit status 125
Command Output: docker: Error response from daemon: error creating aufs mount to /var/lib/docker/aufs/mnt/4174d5bb9f2a1448da29aa20f3be3d5bca2815afe8cebf9cc5ba4ed8cff81510-init: mount target=/var/lib/docker/aufs/mnt/4174d5bb9f2a1448da29aa20f3be3d5bca2815afe8cebf9cc5ba4ed8cff81510-init data=br:/var/lib/docker/aufs/diff/4174d5bb9f2a1448da29aa20f3be3d5bca2815afe8cebf9cc5ba4ed8cff81510-init=rw:/var/lib/docker/aufs/diff/9ca52ece048bb0cd42ae58e359d91b16694d5081c1697c2a328547a028b17cc0=ro+wh:/var/lib/docker/aufs/diff/df9eb543f133e61a6942529dc2ec7135ec6385f59fe862bbe031e3f7876b909d=ro+wh,dio,xino=/dev/shm/aufs.xino: invalid argument.
It seems it is causing me problems when I try and run kind I am not even sure what is causing this mounting problem...
Sharp
Sharp•2y ago
GitHub
document how to run kind in a kubernetes pod · Issue #303 · kuberne...
NOTE: We do NOT recommend doing this if it is at all avoidable. We don't have another option so we do it ourselves, but it has many footguns. xref: #284 additionally these mounts are known ...
Want results from more Discord servers?
Add your server