Unable to connect to workspace | access to terminal, code-server, ssh not working. No matching peer
Have opened an issue for the same - https://github.com/coder/coder/issues/14598
Any suggestions? Please find the details of the issue below -
Hi,
I did a POC on my local machine and was able to setup a coder platform using docker compose specs. Now am try to set it up in corporate environment so that my team can evaluate it for our use cases.
However, when I try to do in corporate environment, am unable to access the workspace container. I tried using this docker-compose spec docker-compose.yml and also tried it do with the specs mentioned in the offline deployment section. Both these methods work on my personal desktop.
However, am encountering the same error - ping(unable to find peer) in both scenarios in corporate environment. I tried running coder ping -v <workspace_name> from remote-host, client machine, from inside the containers... and all of them give the same error.
Am thinking that this is some networking error but not sure what it is. Are there any specific ports that I have to open?
Am not able to connect to my workspace containers. Any help would be appreciated.
Below are the screenshots for reference -
GitHub
Unable to connect to workspace | access to terminal / code-server ...
Hi, I did a POC on my local machine and was able to setup a coder platform using docker compose specs. Now am try to set it up in corporate environment so that my team can evaluate it for our use c...
35 Replies
<#1281871794755731476>
Category
Help needed
Product
Coder OSS (v2)
Platform
Linux
Logs
Please post any relevant logs/error messages.
have pasted them as a screenshot
hey @r1234, could you send a screenshot of the "DERP" section of the
/health page?
by the way, I have closed your GH issue so that we can focus our efforts in the Discord
hi.. thanks for the responding. Please find the screenshot below
I notice that my local deployment (the one that works) has OSHasIPv6 (green) but my corporate deployment ( not working ) has OSHasIPv6 (red)
are there any know ways of debugging connectivity with the workspace containers that we spin. I see I cannot do ssh | access its terminal | code-server etc.
@Phorcys - any help would be appreciated. if I hit a debug/health endpoint... I don't see any errors... but the ping just doesn't work...
is there any way to know which ports and which endpoints are being used for ping the operation from the client to workspace container .... all the intermediate ones... maybe that could get me somewhere.
alright, so to explain
Coder uses Tailscale between clients and the Coder workspaces
I do think it also uses it to communicate between workspaces and the control plane but don't quote me on that as I am not sure
in your case, all this being blocked basically means it can't communicate via DERP, but it should fallback to websocket
you are saying you are not able to connect to your workspaces, could you tell me how you are trying to access them and what error messages do you have (if any?)
anyways, the ping errors in your Coder logs are related to DERP, I suspect that your firewall blocks ICMP packets
maybe our networking requirements can help
Workspaces communicate to Coder using the provider
ACCESS_URL, also make sure that your workspaces can reach itIf my host, client and agents are on the same machine … then also am getting this error of now peer found
Is there any way to troubleshoot this ping error. I mean any command that I can run via cli to reproduce it… or something
I think it has to do with wgengine - am assuming that means wireguard coordinator
I see admin-v1-main have node field populated in my local setup
but it is nil in my setup in office
I think that might be the reason that it is not able to find peer
not sure what is causing the issue though and how to fix it
@r1234 some of us are at Open Source Summit EU atm so the answers will likely be delayed a bit
will get back to you next week if no one has picked it up :)
Thanks. Will ping you next week
sounds good!
just flying by, but even though it should fall back to WebSockets, you can force it
https://coder.com/docs/reference/cli/server#--derp-force-websockets
I tried that as well. Doesn’t work
Do we know how workspace agents are registered…. I mean any reason why node value would be nil in debug/coordinator in office setup as opposed to my personal setup
not sure, i did notice your one screenshot showed udp wasn't healthy/allowed. i assume what is not working is local vs code to workspace. if i remember, i thought that coordinator server process where node keys and IP addresses are shared, requires UDP to do STUN (echoing google.com for each node's address)
guess wait for Phorcys to free up
Have tried doing an offline deployment disabling direct connection and disabling stub. In that case only embedded relay should be used for connection right…
Yeah let’s wait for Phorcys to be free. Hopefully next week
if offline, you can host your own STUN server, but it is needed for coordinator to share node info between the 2 nodes (the client where VS Code or ssh in terminal is, or using coder ping etc.) and the other node (the Coder workspace)
Can you confirm you can get to the dashboard for a worskpace in a browser fine, and open web terminal and if your template has it, a web IDE like code-server?
I think no UDP is part of this too. There corporate firewall rules blocking it
I can open coder app dashboard from my laptop/client machine. However I cannot open web ide, nor terminal, nor can I do ssh in the workspace from the browser.
coder ping -v workspace name … gives the output that I shared in the very first screenshot of this thread.
So STUN is needed even if I disable direct p2p connections and use the embedded relay that comes with coder?
If that is the case … any pointers on how to setup a custom STUN server
Do you know which ports should I be checking … I mean src dest port protocol info… to see if there are any fw blockers …
So no web IDE and terminal too - as docs said, STUN, UDP is for local IDE/ssh, so this could be something broader.
To your question, no, I don't think STUN is needed if p2p/direct is disabled, and your web IDE and web terminal seems to be something. So I can't imagine your own STUN server helps anything.
Hopefully someone more intimate on networking can sugggest things.
Your access URL is set correctly too? Maybe a screenshot of the workspace dashboard would be help - like the icons appear for web terminal and code-server (if you have that installed)
I think you said you're on Kubernetes. One way to test if the workspace pod can get to the Coder control plane, is kubectl exec into the pod and curl the access url you have specified in your server config.
hey @r1234, it should indeed fall back to websockets
you should try what mark suggested
could you send the output of
coder ping -v <workspace> when in the environment that works?
could you also send a screenshot of this please?No just regular docker compose deployment
ACCESS_URL is correct and the agent can access it.
I will share the screenshots in sometime
Thanks for responding!
ok, you did
docker ps and then docker exec -it <container> /bin/bash and can curl the access URL?Yes
I can curl the access url
hey @r1234, could you try this?
and that
Apologies I could respond earlier… let me get you all the details tomorrow.
Apologies… I got stuck into some other stuff but now this poc is again gaining traction in office…
@Phorcys - As mentioned the node is nil. Which means coordinator is not able to discover it for some odd reason. But if I ssh into the workspace agent container and try to curl the access url it works… it is reachable.
Looking into code .. I found this…
Am not able to locate the code though… to see what happens when a new workspace gets provisioned. Am assuming the workspace and network node gets registered with coordinator…
Not sure how to proceed further.. kind of stuck with this. The ping fails am assuming saying no matching peer because of this reason…. But I don’t know how to fix it.
Would be up for a call … if you feel that might be more helpful… ?
Am really bad when it comes to these networking issues…
Thanks!
@sharkymark — please see the above trail … let me know if you have any suggestions that you want me to try…
HI @r1234 Could you open a new thread so that we can help you better?
Sure.