Kubernetes with DinD bind volumes issues
Hello everyone,
I'm running Coder in my Kubernetes cluster on my Ubuntu 22.04 server. I've created a template based on the default Kubernetes template plus the edits to run a privileged sidecar to be able to build and run Docker images inside the workspaces.
Unfortunately, I can't use sysbox and envbox as they fail (sysbox install on nodes fails during the sysbox installer helper step, and envbox throws an error about the driver not supported: overlay2). So, I had to take the privileged DinD sidecar option. But I think I have to make the sidecar container able to use the storage of the main workspace container to be able to mount the folders as binded volumes.
- Any tips on how to solve this issue?
- Also, I have to make NVIDIA GPU acceleration available to all containers running, not only for the workspace.
6 Replies
<#1271032726782476328>
Category
Help needed
Product
Coder OSS (v2)
Platform
Linux
Logs
Please post any relevant logs/error messages.
You can see my current main.tf at this link: https://pastebin.com/g0jz3hSx
Pastebin
main.tf k8s workspaces wit DinD and GPU accel - Pastebin.com
Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
Any updates or tips about this?
hello @MRColorRain, sorry for the delayed answer, any luck?
envbox and sysbox serve different purposes though, envbox is for building devcontainer images and sysbox is a container runtime
the docs for the privileged sidecar should help but I suppose you've already seen those, though try to troubleshoot sysbox before resorting to this option because it's insecure
also, could you resend your template over? the link has expired
Hi, thank you for your response.
Sysbox install fails after the nodes labeling step and manifest apply. So as the k8s nodes run on an on-premise Kubernetes cluster inside our intranet I've resorted to running a privileged sidecar container.
I had to build a custom docker image to have a DinD with CUDA capabilities built inside.
I'll send the updated link tomorrow as if possible I'd like to use either sysbox or podman with a fuse device
alright, feel free to ping me!