Envbuilder in Coder: wrong user is used when cache is enabled

Hello, when I build image for the first time, the correct user, specified in .devcontainer is used. But when the pod is restarted and image is pulled from cache, it ALWAYS uses root user which then fails, because only the specified user has private SSH key in their .ssh which can pull from remote repository. This happens regardless of ENVBUILDER_REMOTE_REPO_BUILD_MODE set to true or false. devcontainer:
{
"build": {
"dockerfile": "./Dockerfile",
"context": "."
},
"features": {
"ghcr.io/devcontainers/features/common-utils:2": {
"installZsh": "true",
"configureZshAsDefaultShell": "true",
"username": "mmolisch",
"userUid": "6043",
"userGid": "6000",
"upgradePackages": "true"
},
},
// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {
"python.defaultInterpreterPath": "/usr/local/bin/python",
"terminal.integrated.defaultProfile.linux": "zsh"
}
}
},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "pip3 install --user -r requirements.txt",
// Set `remoteUser` to `root` to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
"remoteUser": "mmolisch"
}
{
"build": {
"dockerfile": "./Dockerfile",
"context": "."
},
"features": {
"ghcr.io/devcontainers/features/common-utils:2": {
"installZsh": "true",
"configureZshAsDefaultShell": "true",
"username": "mmolisch",
"userUid": "6043",
"userGid": "6000",
"upgradePackages": "true"
},
},
// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {
"python.defaultInterpreterPath": "/usr/local/bin/python",
"terminal.integrated.defaultProfile.linux": "zsh"
}
}
},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "pip3 install --user -r requirements.txt",
// Set `remoteUser` to `root` to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
"remoteUser": "mmolisch"
}
8 Replies
Codercord
Codercord3mo ago
<#1293171628468867234>
Category
Help needed
Product
Coder OSS (v2)
Platform
Linux
Logs
Please post any relevant logs/error messages.
molitann
molitannOP3mo ago
Without cache:
#3: :bust_in_silhouette: Updated the ownership of the workspace! [421.51µs]
#4: :arrows_counterclockwise: Updating ownership of /home/mmolisch...
#4: :house_with_garden: Updated ownership of /home/mmolisch! [21.430858ms]
=== Running init command as user "mmolisch": ["/bin/sh" "-c" "#!/usr/bin/env sh\nset -eux\n# Sleep for a good long while before exiting.\n# This is to allow folks to exec into a failed workspace and poke around to\n# troubleshoot.\nwaitonexit() {\n\techo \"=== Agent script exited with non-zero code ($?). Sleeping 24h to preserve logs...\"\n\tsleep 86400\n}\ntrap waitonexit EXIT\nBINARY_DIR=\"${BINARY_DIR:-$(mktemp -d -t coder.XXXXXX)}\"\nBINARY_NAME=coder\nBINARY_URL=https://our-coder-url/bin/coder-linux-amd64\ncd \"$BINARY_DIR\"\n# Attempt to download the coder agent.\n# This could fail for a number of reasons, many of which are likely transient.\n# So just keep trying!\nwhile :; do\n\t# Try a number of different download tools, as we don not know what we\n\t# will have available.\n\tstatus=\"\"\n\tif command -v curl >/dev/null 2>&1; then\n\t\tcurl -fsSL --compressed \"${BINARY_URL}\" -o \"${BINARY_NAME}\" && break\n\t\tstatus=$?\n\telif command -v wget >/dev/null 2>&1; then\n\t\twget -q

Cloning git repo...
Cloning into 'inno-maro'...
Debian GNU/Linux 12 (bookworm)
Installing v4.93.1 of the amd64 release from GitHub.
#3: :bust_in_silhouette: Updated the ownership of the workspace! [421.51µs]
#4: :arrows_counterclockwise: Updating ownership of /home/mmolisch...
#4: :house_with_garden: Updated ownership of /home/mmolisch! [21.430858ms]
=== Running init command as user "mmolisch": ["/bin/sh" "-c" "#!/usr/bin/env sh\nset -eux\n# Sleep for a good long while before exiting.\n# This is to allow folks to exec into a failed workspace and poke around to\n# troubleshoot.\nwaitonexit() {\n\techo \"=== Agent script exited with non-zero code ($?). Sleeping 24h to preserve logs...\"\n\tsleep 86400\n}\ntrap waitonexit EXIT\nBINARY_DIR=\"${BINARY_DIR:-$(mktemp -d -t coder.XXXXXX)}\"\nBINARY_NAME=coder\nBINARY_URL=https://our-coder-url/bin/coder-linux-amd64\ncd \"$BINARY_DIR\"\n# Attempt to download the coder agent.\n# This could fail for a number of reasons, many of which are likely transient.\n# So just keep trying!\nwhile :; do\n\t# Try a number of different download tools, as we don not know what we\n\t# will have available.\n\tstatus=\"\"\n\tif command -v curl >/dev/null 2>&1; then\n\t\tcurl -fsSL --compressed \"${BINARY_URL}\" -o \"${BINARY_NAME}\" && break\n\t\tstatus=$?\n\telif command -v wget >/dev/null 2>&1; then\n\t\twget -q

Cloning git repo...
Cloning into 'inno-maro'...
Debian GNU/Linux 12 (bookworm)
Installing v4.93.1 of the amd64 release from GitHub.
With cache:
Restored DOCKER_CONFIG to
#4: :arrows_counterclockwise: Updating the ownership of the workspace...
#4: :bust_in_silhouette: Updated the ownership of the workspace! [408.689µs]
=== Running init command as user "root": ["/bin/sh" "-c" "#!/usr/bin/env sh\nset -eux\n# Sleep for a good long while before exiting.\n# This is to allow folks to exec into a failed workspace and poke around to\n# troubleshoot.\nwaitonexit() {\n\techo \"=== Agent script exited with non-zero code ($?). Sleeping 24h to preserve logs...\"\n\tsleep 86400\n}\ntrap waitonexit EXIT\nBINARY_DIR=\"${BINARY_DIR:-$(mktemp -d -t coder.XXXXXX)}\"\nBINARY_NAME=coder\nBINARY_URL=https://our-coder-url/bin/coder-linux-amd64\ncd \"$BINARY_DIR\"\n# Attempt to download the coder agent.\n# This could fail for a number of reasons, many of which are likely transient.\n# So just keep trying!\nwhile :; do\n\t# Try a number of different download tools, as we don not know what we\n\t# will have available.\n\tstatus=\"\"\n\tif command -v curl >/dev/null 2>&1; then\n\t\tcurl -fsSL --compressed \"${BINARY_URL}\" -o \"${BINARY_NAME}\" && break\n\t\tstatus=$?\n\telif command -v wget >/dev/null 2>&1; then\n\t\twget -q \"${

Cloning git repo...
Cloning into 'inno-maro'...
Host key verification failed.
Restored DOCKER_CONFIG to
#4: :arrows_counterclockwise: Updating the ownership of the workspace...
#4: :bust_in_silhouette: Updated the ownership of the workspace! [408.689µs]
=== Running init command as user "root": ["/bin/sh" "-c" "#!/usr/bin/env sh\nset -eux\n# Sleep for a good long while before exiting.\n# This is to allow folks to exec into a failed workspace and poke around to\n# troubleshoot.\nwaitonexit() {\n\techo \"=== Agent script exited with non-zero code ($?). Sleeping 24h to preserve logs...\"\n\tsleep 86400\n}\ntrap waitonexit EXIT\nBINARY_DIR=\"${BINARY_DIR:-$(mktemp -d -t coder.XXXXXX)}\"\nBINARY_NAME=coder\nBINARY_URL=https://our-coder-url/bin/coder-linux-amd64\ncd \"$BINARY_DIR\"\n# Attempt to download the coder agent.\n# This could fail for a number of reasons, many of which are likely transient.\n# So just keep trying!\nwhile :; do\n\t# Try a number of different download tools, as we don not know what we\n\t# will have available.\n\tstatus=\"\"\n\tif command -v curl >/dev/null 2>&1; then\n\t\tcurl -fsSL --compressed \"${BINARY_URL}\" -o \"${BINARY_NAME}\" && break\n\t\tstatus=$?\n\telif command -v wget >/dev/null 2>&1; then\n\t\twget -q \"${

Cloning git repo...
Cloning into 'inno-maro'...
Host key verification failed.
Phorcys
Phorcys3mo ago
hey @molitann, could you send your template over?
molitann
molitannOP3mo ago
Gist
template.tf
GitHub Gist: instantly share code, notes, and snippets.
Phorcys
Phorcys3mo ago
cc: @Cian
Cian
Cian3mo ago
Hey @molitann can you also upload your Dockerfile? I'm able to repro this without the provider in the mix, it looks like it's related to when a user gets created as part of the devcontainer customizations I'm guessing that remoteUser should probably override whatever else gets auto-detected? try with containerUser instead of remoteUser
molitann
molitannOP3mo ago
Oh yes that did the trick! Thank you I set both and the user is now correctly set even when the whole image is cached. Should be probably mentioned somewhere in the docs
"remoteUser": "mmolisch",
"containerUser": "mmolisch"
"remoteUser": "mmolisch",
"containerUser": "mmolisch"
Cian
Cian3mo ago
Thanks @molitann ! Will add to the docs.

Did you find this page helpful?