Coder.com•2mo ago

unable to remove docker image

Error: Unable to remove Docker image: Error response from daemon: conflict: unable to remove repository reference "ghcr.io/coder/envbuilder:latest" (must force) - container e1d72f3774ba is using its referenced image 0043aade3996 I'm trying to delete a devcontainers workspace, and no matter which workspace I try to delete, it always fails with a variant of the above message where the only difference is the hex vomit. How do I get the platform to do the forced removal that docker is demanding here?

9 Replies

Codercord•2mo ago

<#1322435181755240549>

Category

Bug report

Product

Coder (v2)

Platform

Linux

Logs

Please post any relevant logs/error messages.

arkevorkhatOP•2mo ago

message.txt

arkevorkhatOP•2mo ago

All of this wouldn't be an issue if it wasn't so obscenely difficult to set up access to private github repos. I've now tried the oauth app method 3 times, and I'm trying to clean up containers so I don't bog down my server while I try the personal access token method. ... SIGH Turns out the fix is that you have to stop every single workspace created from the same template that was used to create the one workspace you want to delete. Then you can delete the workspace(s) you want to get rid of, then if there are any left, you can restart them. CLOSE

Phorcys•2mo ago

this shouldn't be the case though, and it seems to be a new issue, i will investigate you can't have it do the forced removal because other containers still need the image, the real issue is that it shouldn't be trying to remove the image did you fix this part? EDIT: didn't see https://discord.com/channels/747933592273027093/1322449893058547833

arkevorkhatOP•2mo ago

Yes I did, the ultimate fix is posted here: https://discord.com/channels/747933592273027093/1322449893058547833 I'll go ahead and see if I can replicate the issue now that I've fixed the other problems in my install, since that'll narrow down whether it's only an issue when envbuilder uses a fallback image. Alright, just re-ran the test, steps below: * Created two workspaces using the vercel/next.js template named test-2 and test-3 * waited for both to finish setting up * attempted to delete test-3 * got error message

Error: Unable to remove Docker image: Error response from daemon: conflict: unable to remove repository reference "ghcr.io/coder/envbuilder:latest" (must force) - container 8a6a83b04dd9 is using its referenced image 0043aade3996

- error banner says "Workspace build failed - terraform apply: exit status 1" Confirmed reproducible on my setup. Happy to send you my system specs and docker compose file if you'd like. alright, here's a bit more info: * created a second user * created another workspace under their name using the same github repo as one of my own workspaces * attempted to delete it * got same error message with identical hex data.

Phorcys•2mo ago

hey @arkevorkhat @Katorly @tsa @Some Dinosaur, since you all have the same issue, i really think this is a bug we've introduced in a template change, would you be able to answer the following questions to help us pinpoint where it went bad? - Which version of Coder are you running? - When did you first notice this problem? - When was your template created? (i.e when did you choose the devcontainer template from the examples, you can see that by going in the "versions" tab of your template) nevermind, i think this issue has always been present and we didn't notice it yet i am able to reproduce on my end, the issue is caused by the fact that deleting a workspace will delete all of its resources (which makes sense), but we didn't plan that using a shared image would cause this okay, this should be fixed by adding the lifecycle.prevent_destroy attribute to the builder image: (diff, do not add the plus signs)

resource "docker_image" "devcontainer_builder_image" {
  name = local.devcontainer_builder_image

+  lifecycle {
+    prevent_destroy = true
+  }
}

resource "docker_image" "devcontainer_builder_image" {
  name = local.devcontainer_builder_image

+  lifecycle {
+    prevent_destroy = true
+  }
}

nevermind, i'm not sure if there's an immediate fix to this

Error: Instance cannot be destroyed
on main.tf line 156:
  156: resource "docker_image" "devcontainer_builder_image" {
Resource docker_image.devcontainer_builder_image has lifecycle.prevent_destroy set, but the plan calls for this resource to be destroyed. To avoid this error and continue with the plan, either disable lifecycle.prevent_destroy or reduce the scope of the plan using the -target option.

Error: Instance cannot be destroyed
on main.tf line 156:
  156: resource "docker_image" "devcontainer_builder_image" {
Resource docker_image.devcontainer_builder_image has lifecycle.prevent_destroy set, but the plan calls for this resource to be destroyed. To avoid this error and continue with the plan, either disable lifecycle.prevent_destroy or reduce the scope of the plan using the -target option.

Phorcys•2mo ago

I've opened https://github.com/coder/coder/issues/15972 for our engineers to take a look at this issue

GitHub

Impossible to delete docker-devcontainer workspaces when 1+ other...

When multiple docker-devcontainer workspaces are running, deleting one of them will trigger the following issue: Terraform 1.9.8 coder_agent.main: Plan to delete docker_volume.workspaces: Plan to d...

arkevorkhatOP•2mo ago

Thank you Phorcys, both for your effort and your patience

Phorcys•2mo ago

hey @arkevorkhat @Katorly @tsa @Some Dinosaur, this issue has been fixed! 🎉 it will work on its own for newly created templates in the newer versions but you'll have to patch your current template by adding keep_locally = true to the docker_image resource like so:

resource "docker_image" "devcontainer_builder_image" {
  name = local.devcontainer_builder_image

+  keep_locally = true
}

resource "docker_image" "devcontainer_builder_image" {
  name = local.devcontainer_builder_image

+  keep_locally = true
}

let me know if you need any help

Gaming

Programming

unable to remove docker image

Did you find this page helpful?