Coder Workspaces Stuck in Apply Complete

After updating to the latest 2.18.x release, we're seeing the following on coder workspaces (custom kubevirt template)
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: Still creating... [30s elapsed]
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): "phase": "Running",
null_resource.start_vm[0]: (local-exec): VM started successfully
null_resource.start_vm[0]: (local-exec) Provisioning complete
null_resource.start_vm[0]: Creation complete after 32s [id=6062895267040240662]
Apply complete! Resources: 4 added, 1 changed, 4 destroyed.
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: Still creating... [30s elapsed]
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): "phase": "Running",
null_resource.start_vm[0]: (local-exec): VM started successfully
null_resource.start_vm[0]: (local-exec) Provisioning complete
null_resource.start_vm[0]: Creation complete after 32s [id=6062895267040240662]
Apply complete! Resources: 4 added, 1 changed, 4 destroyed.
It looks like the terraform build completes, the terraform process exits inside coder, but coder still thinks the workspace is building. tried rolling back versions but still no fix. is this a known issue?
16 Replies
Codercord
Codercord3w ago
<#1336062330139840602>
Category
Bug report
Product
Coder (v2)
Platform
Linux
Logs
Please post any relevant logs/error messages.
Phorcys
Phorcys3w ago
hey @bl1tzcra1g, could you send a screenshot alongside with your template?
bl1tzcra1g
bl1tzcra1gOP3w ago
test ok nice discord is back. we actually managed to figure it out. there were a number of old terraform outputs we were passing through so when we removed them it worked my colleague said "coder doesn't take terraform outputs anymore, it crashes it". interesting that there was no output. happy to provide the previous template if that helps things on your end but otherwise this can be marked as solved. appreciate the follow up
Phorcys
Phorcys3w ago
that is weird, i'd like the old template to investigate on this! also, it seems like you are using local-exec, is this so you can keep the same VM object and just start it instead of recreating a new one every time?
bl1tzcra1g
bl1tzcra1gOP3w ago
yeah exactly it's an odd template, had to work around quite a few limitations with kubevirt and coder templates but it's been rock solid for a year+ I just stepped away from my computer but I'll send it as soon as I'm back
bl1tzcra1g
bl1tzcra1gOP3w ago
sorry completely forgot lol
Phorcys
Phorcys2w ago
no worries, could you also send a screenshot of the workspace page? it seems to me that Coder might be waiting for the agent to connect
bl1tzcra1g
bl1tzcra1gOP2w ago
we have it fixed now but the workspace page was just stuck on the terraform output from above:
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: Still creating... [30s elapsed]
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): "phase": "Running",
null_resource.start_vm[0]: (local-exec): VM started successfully
null_resource.start_vm[0]: (local-exec) Provisioning complete
null_resource.start_vm[0]: Creation complete after 32s [id=6062895267040240662]
Apply complete! Resources: 4 added, 1 changed, 4 destroyed.
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: Still creating... [30s elapsed]
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): Waiting for VM to start...
null_resource.start_vm[0]: (local-exec): "phase": "Running",
null_resource.start_vm[0]: (local-exec): VM started successfully
null_resource.start_vm[0]: (local-exec) Provisioning complete
null_resource.start_vm[0]: Creation complete after 32s [id=6062895267040240662]
Apply complete! Resources: 4 added, 1 changed, 4 destroyed.
and never swapped to the screen with the agent/links to connect. we were able to confirm the agent was running via systemctl
Phorcys
Phorcys2w ago
weird, how have you fixed it?
bl1tzcra1g
bl1tzcra1gOP2w ago
this
Phorcys
Phorcys2w ago
sorry, i'm all over the place lol thanks :) is this template the fixed one or the one with the issue then?
bl1tzcra1g
bl1tzcra1gOP2w ago
all good haha, can imagine the context switching necessary for swapping b/w tickets and bug work. it's the one with the issue i can send the fixed one as well
Phorcys
Phorcys2w ago
yes please, i'll take a look to see if i can reproduce! the fixed one should help find what was the issue
Phorcys
Phorcys2w ago
i'm resolving this issue but keeping it for later
Codercord
Codercord2w ago
@Phorcys closed the thread.

Did you find this page helpful?