Coder.com•16mo ago

envbox + envbuilder template

In my attempts to get a secure k3s-compatiable coder setup working, I am trying to get envbuilder to work from inside envbox.

{"ts":"2023-08-21T03:24:35.37819366Z","level":"ERROR","msg":"send startup logs","caller":"/home/runner/work/envbox/envbox/buildlog/coder.go:116","func":"github.com/coder/envbox/buildlog.(*CoderLogger).processLogs.func1","fields":{"error":"Patch \"https://coder.mcaq.me/api/v2/workspaceagents/me/startup-logs\": context canceled"}}
run: get image metadata: start container: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: container_linux.go:425: starting container process caused: exec: "sleep": executable file not found in $PATH: unknown

{"ts":"2023-08-21T03:24:35.37819366Z","level":"ERROR","msg":"send startup logs","caller":"/home/runner/work/envbox/envbox/buildlog/coder.go:116","func":"github.com/coder/envbox/buildlog.(*CoderLogger).processLogs.func1","fields":{"error":"Patch \"https://coder.mcaq.me/api/v2/workspaceagents/me/startup-logs\": context canceled"}}
run: get image metadata: start container: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: container_linux.go:425: starting container process caused: exec: "sleep": executable file not found in $PATH: unknown

Currently this is my issue. Find the full log attatched. Find my current template here https://github.com/Sharpz7/Sharpz7/blob/main/templates/devcontainer/main.tf

GitHub

Sharpz7/templates/devcontainer/main.tf at main · Sharpz7/Sharpz7

Contribute to Sharpz7/Sharpz7 development by creating an account on GitHub.

dev.log

37 Replies

Codercord•16mo ago

<#1143024634099413084>

Category

Help needed

Product

Coder OSS (v2)

Platform

Linux

Logs

Please post any relevant logs/error messages.

kyle•16mo ago

envbox starts the container with sleep as the init to get a dummy entry to read /etc/passwd to find the user to run as, so it won't work under the current structure.

SharpOP•16mo ago

Ah. So I need to create a custom envbuilder image?

kyle•16mo ago

Yup, tragically

SharpOP•16mo ago

Right. I'll look into that. Maybe I just directly contribute changes to the repo? I imagine k3s+coder+env builder+envbox would be a hobbyists quickest way to a full coder solution? I have halted my efforts to contribute my stuff upstream because I still have docker issues. im hoping envbox fixes them. If it does I'll definitely get contributions+a blog post written

SharpOP•16mo ago

@Phorcys I am attempting to put both into a template: https://github.com/Sharpz7/Sharpz7/blob/main/templates/devcontainer/main.tf https://github.com/Sharpz7/Sharpz7/blob/main/dockerfiles/envbuilder.Dockerfile (Needed to deal with the fact envbox assumes access to sleep in container https://drive.google.com/uc?id=1GtrxNXWPWJDkYPdW-hvqkmqbZLgP8Xwg (the pod has no errors)

GitHub

Sharpz7/templates/devcontainer/main.tf at main · Sharpz7/Sharpz7

Contribute to Sharpz7/Sharpz7 development by creating an account on GitHub.

GitHub

Sharpz7/dockerfiles/envbuilder.Dockerfile at main · Sharpz7/Sharpz7

Contribute to Sharpz7/Sharpz7 development by creating an account on GitHub.

SharpOP•16mo ago

My guess is I now need a change to actually trigger the code in the container? Or maybe my code-server script needs moved? I am not sure

Phorcys•16mo ago

honestly i'm looking at envbuilder and i don't really understand it sooo i don't know if i'll be able to help

SharpOP•16mo ago

Hahha no problem! I see if I can get someone else

Phorcys•16mo ago

i'll look into it if i get some free time today

SharpOP•16mo ago

Thanks, I have a few ideas yet so I will try em Although, I really should go to bed

Phorcys•16mo ago

health 1st but fun 0st

SharpOP•16mo ago

Trying to convince my uni to flood me with gcp credits

SharpOP•16mo ago

{"output":"Failed to run envbox: boostrap container: timed out boostrapping container: boostrap container (): \u0002\u0000\u0000\u0000\u0000\u0000\u0000\u003eerror: CODER_AGENT_URL must be set if CODER_AGENT_TOKEN is set: exit code 1","time":"2023-08-23T07:18:01.018889288Z","type":"error"}
{"output":"Failed to run envbox: run: boostrap container: timed out boostrapping container: boostrap container (): \u0002\u0000\u0000\u0000\u0000\u0000\u0000\u003eerror: CODER_AGENT_URL must be set if CODER_AGENT_TOKEN is set: exit code 1","time":"2023-08-23T07:18:01.019184447Z","type":"error"}
{"output":"","time":"2023-08-23T07:18:01.019200808Z","type":"done"}
run: boostrap container: timed out boostrapping container: boostrap container (): >error: CODER_AGENT_URL must be set if CODER_AGENT_TOKEN is set: exit code 1

{"output":"Failed to run envbox: boostrap container: timed out boostrapping container: boostrap container (): \u0002\u0000\u0000\u0000\u0000\u0000\u0000\u003eerror: CODER_AGENT_URL must be set if CODER_AGENT_TOKEN is set: exit code 1","time":"2023-08-23T07:18:01.018889288Z","type":"error"}
{"output":"Failed to run envbox: run: boostrap container: timed out boostrapping container: boostrap container (): \u0002\u0000\u0000\u0000\u0000\u0000\u0000\u003eerror: CODER_AGENT_URL must be set if CODER_AGENT_TOKEN is set: exit code 1","time":"2023-08-23T07:18:01.019184447Z","type":"error"}
{"output":"","time":"2023-08-23T07:18:01.019200808Z","type":"done"}
run: boostrap container: timed out boostrapping container: boostrap container (): >error: CODER_AGENT_URL must be set if CODER_AGENT_TOKEN is set: exit code 1

This is as far as I got, see the links above for more information. I think the problem is the init_scripts, like what to run, and where: https://github.com/Sharpz7/Sharpz7/blob/main/templates/devcontainer/main.tf#L184

GitHub

Sharpz7/templates/devcontainer/main.tf at main · Sharpz7/Sharpz7

Contribute to Sharpz7/Sharpz7 development by creating an account on GitHub.

SharpOP•16mo ago

@kyle Sorry for the ping, but I think at this point its an intracasy I could do with a staff member look at.

kyle•16mo ago

Can you manually check if the CODER_AGENT_TOKEN env var is being set properly?

SharpOP•16mo ago

Where? Like in a pod shell? Or add it to one of the commands in the terraform?

kyle•16mo ago

In a Pod shell. I'm confused why it's not ther

SharpOP•16mo ago

The Pod fails before I can. I have added a echo to the /envbuild command, we'll see if that works https://drive.google.com/uc?id=1L5iPFP9iIeSxU1DwZRrySNFOuKTEknJP That is using my k8s monitoring tool. so it seems its getting to k8s at least TESTC-https://coder.mcaq.me That worked.

    container {
      name              = "dev"
      image             = "ghcr.io/coder/envbox:latest"
      image_pull_policy = "Always"
      command           = ["sh", "-c", "echo TESTC-$CODER_AGENT_URL && '/envbox docker'"]

    container {
      name              = "dev"
      image             = "ghcr.io/coder/envbox:latest"
      image_pull_policy = "Always"
      command           = ["sh", "-c", "echo TESTC-$CODER_AGENT_URL && '/envbox docker'"]

Not sure what to do now I am imagining that the error being suggested is not actually what is going wrong?

SharpOP•16mo ago

dev.log

SharpOP•16mo ago

@kyle Okay so an update, I can get it working with the base envbox terraform file, but when I add my image (with no other changes) it hangs

message.txt

SharpOP•16mo ago

Further update, I no longer have that error, but now it just hangs. Is there a good way to debug this? I think the problem is viewing the logs of the inner container.

message.tf

SharpOP•16mo ago

Wait a second. Maybe I am being silly. Does envbuilder work in a k8s pod without any kind of sysbox of DIND solution?

SharpOP•16mo ago

@Phorcys you might know the answer. Looking at https://github.com/coder/coder/blob/main/examples/templates/devcontainer-kubernetes/main.tf it seems the answer is no

GitHub

coder/examples/templates/devcontainer-kubernetes/main.tf at main · ...

A tool that provisions remote development environments via Terraform - coder/coder

Phorcys•16mo ago

honestly, no clue sorry

SharpOP•16mo ago

Time to test. I'm going to feel like an idiot if it just works 🤣 Although I feel like it can't... How would I access docker in the Dev container? I guess we shall see @Phorcys In case you are ever asked this again, it seems that devcontainer does just work, but docker is not present from inside the workspace So that is half of the problem solved, but I still need envbox it seems (And then I can't imagine the pain of trying to then bundle all this in a google vm smh)

Phorcys•16mo ago

thank you!

SharpOP•16mo ago

Actually @kyle: ignoring my k3s envbox/envbuilder issue for a second, what would be the right way of putting envbuilder inside a gcp vm? Because you probably are okay accepting DooD at that point and just mounting the sock directly since its already in a gcp vm?

kyle•16mo ago

I believe GCP VMs allow a container to be specified

SharpOP•16mo ago

Yeah, but I am still going to need to expose docker to the container (the envbuilder container) in some way right? Or does GCP just handle that? I also struggled to get GPU's to work with that

kyle•16mo ago

envbuilder doesn't need Docker though

SharpOP•16mo ago

If I want to run docker commands inside of envbuilder? I do need something. That is why I am trying to add envbox in my k3s setup But on GCP, envbox seems like a lot of effort to go to when in theory I can just mount the docker sock

kyle•16mo ago

Ahh, I see. Can you use envbuilder with sysbox?

SharpOP•16mo ago

On k3s, no. They don't support it yet That is why I am trying to get envbox and envbuilder to play nice (But I think that is impossible right now from my testing?) Would sysbox be the solution to use on gcp? I don't think its unreasonable for people to still want docker build access inside a devcontainer

SharpOP•16mo ago

https://github.com/coder/envbuilder/issues/25 https://github.com/coder/envbuilder/issues/50 It looks like I need both these issues fixed before I can continue. Good to know. I can put this to bed for now

GitHub

Doesn't work with docker-in-docker devcontainer feature · Issue #...

While experimenting with a .devcontainer that uses docker-in-docker feature, envbuilder gives docker run -it --rm \ -v /tmp/envbuilder:/workspaces \ -e GIT_URL=https://github.com/matifali/coder-dev...

GitHub

Envbuilder does not run in a sysbox container · Issue #50 · coder/e...

Envbuilder fails if it runs using the sysbox container runtime. This is unfortunate because sysbox is neccessary to run k3s or docker within a workspace. The issue is described already in the kanik...

ATXadam•5mo ago

Did you ever figure this out? I've been trying on k3s and running into quite a few issues, not sure if it is worth continued efforts

Cian•5mo ago

Hey @ATXadam , we recently (with the help of some other community members) figured out a fix for the above issue #50 -- does that help? I'm currently developing Coder through a template that uses Docker+sysbox -- I haven't played around with k3s + sysbox + envbuilder yet though.

Gaming

Programming

envbox + envbuilder template