Plotly Mike
Plotly Mike
PostsComments
CCoder.com
Created by Plotly Mike on 8/29/2024 in #help
"The agent cannot authenticate until the workspace provision job has been completed."
I am trying to provision a Coder agent on a VM. I am doing this by creating a systemd unit that runs the Coder agent. However, when the unit starts, as part of a remote exec provisioner, it just keeps repeating this: 
Aug 29 07:07:09 mike9-services.redacted coder[7482]: 2024-08-29 07:07:09.252 [info]  connecting to coderd
Aug 29 07:07:09 mike9-services.redacted coder[7482]: 2024-08-29 07:07:09.257 [warn]  run exited with error ...
Aug 29 07:07:09 mike9-services.redacted coder[7482]:     error= GET https://coder.redacted/api/v2/workspaceagents/me/rpc?version=2.2: unexpected status code 401: Workspace agent not authorized.: Try logging in using >
Aug 29 07:07:09 mike9-services.redacted coder[7482]:                    Error: The agent cannot authenticate until the workspace provision job has been completed. If the job is no longer running, this agent is invalid.
Aug 29 07:07:09 mike9-services.redacted coder[7482]: 2024-08-29 07:07:09.252 [info]  connecting to coderd
Aug 29 07:07:09 mike9-services.redacted coder[7482]: 2024-08-29 07:07:09.257 [warn]  run exited with error ...
Aug 29 07:07:09 mike9-services.redacted coder[7482]:     error= GET https://coder.redacted/api/v2/workspaceagents/me/rpc?version=2.2: unexpected status code 401: Workspace agent not authorized.: Try logging in using >
Aug 29 07:07:09 mike9-services.redacted coder[7482]:                    Error: The agent cannot authenticate until the workspace provision job has been completed. If the job is no longer running, this agent is invalid.
Restarting the systemd unit has no effect. The token is being passed through from the coder_agent resource (coder_agent.main.token). With coder state pull I can see that the token on the agent resource is correct compared to the one on disk, but I cannot get this agent to associated. I've tried adding a coder_agent_instance but it doesn't help. I see an old closed issue https://github.com/coder/coder/issues/5704 as being similar, but there's no clear fix present there. Perhaps worth mentioning: - Using an AWS instance, but in a different account, so using token auth, not aws instance identity - AWS instance is created by a module that is intended to work independently of Coder; seems to pose some issues for knitting the agent together with the instance, couldn't get coder_metadata to work either - Actual instance is spun up fine and seems happy except for the agent connection
17 replies
CCoder.com
Created by Plotly Mike on 8/29/2024 in #help
Any way to view the detailed Terraform plan for a workspace update?
I am experiencing weird issues with the AWS terraform provider that I could troubleshoot if I knew what configuration had drifted when I run an update. Right now, I get output like: 
coder_agent.main: Drift detected (update)
module.network.aws_instance.services[0]: Plan to update
module.network.aws_route53_record.main[0]: Plan to update
Plan: 0 to add, 2 to change, 0 to destroy.
coder_agent.main: Drift detected (update)
module.network.aws_instance.services[0]: Plan to update
module.network.aws_route53_record.main[0]: Plan to update
Plan: 0 to add, 2 to change, 0 to destroy.
but it does not show me the actual changes to the Route53 record that it intends to make, like terraform would in normal standalone execution, and then I get this error: 
When expanding the plan for module.network.aws_route53_record.main[0] to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .records: planned set element cty.StringVal("REDACTED IP ADDRESS THAT HAS NOTHING TO DO WITH THIS CODER WORKSPACE AND IDK WHERE IT EVEN CAME FROM") does not correlate with any element in actual.
This is a bug in the provider, which should be reported in the provider's own issue tracker.
When expanding the plan for module.network.aws_route53_record.main[0] to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .records: planned set element cty.StringVal("REDACTED IP ADDRESS THAT HAS NOTHING TO DO WITH THIS CODER WORKSPACE AND IDK WHERE IT EVEN CAME FROM") does not correlate with any element in actual.
This is a bug in the provider, which should be reported in the provider's own issue tracker.
This might indeed be an upstream bug, but there should not be anything changing about the Route 53 DNS record here; so I want to figure out what's going on with the plan to avoid the situation entirely.
10 replies