EWS01: websocket dial: failed to WebSocket dial: failed to send handshake request

I'm having this error. I already setup self signed certificate, cluster issuer and using below values. But still having the issue. Using k3s, it able to create pods but sockets are having trouble due to tls.
EWS01: websocket dial: failed to WebSocket dial: failed to send handshake request: Get "https://coder.lab.dttdata.com/api/v2/debug/ws": tls: failed to verify certificate: x509: certificate is valid for dc5503177ca435d680300ed475c2fda7.162e42cad00d0384ec662ce16b45b7d9.traefik.default, not coder.lab.dttdata.com
coder:
env:
- name: CODER_PG_CONNECTION_URL
valueFrom:
secretKeyRef:
name: coder-db-url
key: url
- name: CODER_ACCESS_URL
value: "https://coder.lab.dttdata.com"
- name: CODER_WILDCARD_ACCESS_URL
value: "*.coder.lab.dttdata.com"

service:
enable: true
type: ClusterIP
sessionAffinity: ClientIP
externalTrafficPolicy: Cluster
loadBalancerIP: ""
annotations: {}

ingress:
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
secretNames: coder-tls-secret
wildcardSecretName: coder-tls-secret

tls:
secretName:
- coder-tls-secret
coder:
env:
- name: CODER_PG_CONNECTION_URL
valueFrom:
secretKeyRef:
name: coder-db-url
key: url
- name: CODER_ACCESS_URL
value: "https://coder.lab.dttdata.com"
- name: CODER_WILDCARD_ACCESS_URL
value: "*.coder.lab.dttdata.com"

service:
enable: true
type: ClusterIP
sessionAffinity: ClientIP
externalTrafficPolicy: Cluster
loadBalancerIP: ""
annotations: {}

ingress:
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
secretNames: coder-tls-secret
wildcardSecretName: coder-tls-secret

tls:
secretName:
- coder-tls-secret
62 Replies
Codercord
Codercord•6d ago
<#1363464782174490685>
Category
Help needed
Product
Coder (v2)
Platform
N/A
Logs
Please post any relevant logs/error messages.
What operating system are you running Coder (v2) on?
Phorcys
Phorcys•6d ago
hi! the issue you're running into is because you do not have a valid certificate for this host have you defined a coder-tls-secret resource?
emircanerkul
emircanerkulOP•6d ago
yes
erkulab@erkulab:~$ kubectl get secret -n coder
NAME TYPE DATA AGE
coder-db-postgresql Opaque 2 22h
coder-db-url Opaque 1 22h
coder-tls-secret kubernetes.io/tls 3 58m
erkulab@erkulab:~$ kubectl get secret -n coder
NAME TYPE DATA AGE
coder-db-postgresql Opaque 2 22h
coder-db-url Opaque 1 22h
coder-tls-secret kubernetes.io/tls 3 58m
Phorcys
Phorcys•6d ago
how did you define it?
Phorcys
Phorcys•6d ago
okay so, is this the most up-to-date error message?
emircanerkul
emircanerkulOP•6d ago
erkulab@erkulab:~$ cat coder.certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: coder-cert
namespace: coder
spec:
secretName: coder-tls-secret
issuerRef:
name: dttdata-clusterissuer
kind: ClusterIssuer
dnsNames:
- coder.lab.dttdata.com
erkulab@erkulab:~$ cat coder.certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: coder-cert
namespace: coder
spec:
secretName: coder-tls-secret
issuerRef:
name: dttdata-clusterissuer
kind: ClusterIssuer
dnsNames:
- coder.lab.dttdata.com
yes
emircanerkul
emircanerkulOP•6d ago
one weird thing, when i did the setup i see cert is updated but now seeing it got tearific's one
No description
Phorcys
Phorcys•6d ago
the certificate is valid for dc5503177ca435d680300ed475c2fda7.162e42cad00d0384ec662ce16b45b7d9.traefik.default means that the traefik ingress controller's default certificate is in use and not the one you've set
emircanerkul
emircanerkulOP•6d ago
erkulab@erkulab:~$ kubectl get ingress -A
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
coder coder traefik coder.lab.dttdata.com,*.coder.lab.dttdata.com 192.168.1.200 80 22h
erkulab@erkulab:~$ kubectl get ingress -A
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
coder coder traefik coder.lab.dttdata.com,*.coder.lab.dttdata.com 192.168.1.200 80 22h
Phorcys
Phorcys•6d ago
you'll need to add *.coder.lab.dttdata.com to the dnsNames
emircanerkul
emircanerkulOP•6d ago
its already there
erkulab@erkulab:~$ cat coder.certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: coder-cert
namespace: coder
spec:
secretName: coder-tls-secret
issuerRef:
name: dttdata-clusterissuer
kind: ClusterIssuer
dnsNames:
- coder.lab.dttdata.com
erkulab@erkulab:~$ cat coder.certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: coder-cert
namespace: coder
spec:
secretName: coder-tls-secret
issuerRef:
name: dttdata-clusterissuer
kind: ClusterIssuer
dnsNames:
- coder.lab.dttdata.com
Phorcys
Phorcys•6d ago
no
emircanerkul
emircanerkulOP•6d ago
but somehow looks overriding?
Phorcys
Phorcys•6d ago
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: coder-cert
namespace: coder
spec:
secretName: coder-tls-secret
issuerRef:
name: dttdata-clusterissuer
kind: ClusterIssuer
dnsNames:
- coder.lab.dttdata.com
- *.coder.lab.dttdata.com
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: coder-cert
namespace: coder
spec:
secretName: coder-tls-secret
issuerRef:
name: dttdata-clusterissuer
kind: ClusterIssuer
dnsNames:
- coder.lab.dttdata.com
- *.coder.lab.dttdata.com
like this are you sure that you've applied the helm chart with these values? they look to be okay from my standpoint though keep in mind that even if it uses the proper certificate it won't be trusted as it's a self-signed certificate, so health checks won't work and the agent won't be able to connect unless you use a valid cert
emircanerkul
emircanerkulOP•6d ago
yes i did, let me update again so Yea i trusted manuelly in my device but not sure if i need to do smth in pods
Phorcys
Phorcys•6d ago
any reason you can't use valid certificates from let's encrypt/zerossl? you do, but it's a hassle
emircanerkul
emircanerkulOP•6d ago
No need to add more complexity 😄 yea i can i have domain and could via cloudflare
Phorcys
Phorcys•6d ago
cert-manager
Cloudflare
cert-manager configuration: ACME DNS-01 challenges using Cloudflare DNS
Phorcys
Phorcys•6d ago
cert-manager will emit certificates and resolve dns challenges via the Cloudflare API
emircanerkul
emircanerkulOP•6d ago
hmmm.. this could be reason
Phorcys
Phorcys•6d ago
it most definitely is, but it's also that your ingress controller seems to be ignoring the certificate anyways here's an example
---
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
creationTimestamp: null
name: cloudflare-api-token-secret
namespace: cert-manager
spec:
encryptedData:
api-token: [...]
template:
metadata:
creationTimestamp: null
name: cloudflare-api-token-secret
namespace: cert-manager
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-production
namespace: cert-manager
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: acme-production-issuer-key
email: <email for letsencrypt>
solvers:
- dns01:
cloudflare:
email: <cloudflare email>
apiTokenSecretRef:
name: cloudflare-api-token-secret
key: api-token
---
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
creationTimestamp: null
name: cloudflare-api-token-secret
namespace: cert-manager
spec:
encryptedData:
api-token: [...]
template:
metadata:
creationTimestamp: null
name: cloudflare-api-token-secret
namespace: cert-manager
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-production
namespace: cert-manager
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: acme-production-issuer-key
email: <email for letsencrypt>
solvers:
- dns01:
cloudflare:
email: <cloudflare email>
apiTokenSecretRef:
name: cloudflare-api-token-secret
key: api-token
you don't need to use sealed secrets though
emircanerkul
emircanerkulOP•6d ago
First i want to solve why it mangle certificates and not use what i said before i was uusing
ingress:
enable: true
className: ""
# TODO
host: "coder.mydomain.com"
# TODO
wildcardHost: "*.coder.mydomain.com"
annotations: {}
tls:
enable: false
secretNames: ""
wildcardSecretName: ""
ingress:
enable: true
className: ""
# TODO
host: "coder.mydomain.com"
# TODO
wildcardHost: "*.coder.mydomain.com"
annotations: {}
tls:
enable: false
secretNames: ""
wildcardSecretName: ""
then switched according to https://coder.com/docs/admin/setup#tls--reverse-proxy
coder:
tls:
secretName:
- coder-tls

# Alternatively, if you use an Ingress controller to terminate TLS,
# set the following values:
ingress:
enable: true
secretName: coder-tls
wildcardSecretName: coder-tls
coder:
tls:
secretName:
- coder-tls

# Alternatively, if you use an Ingress controller to terminate TLS,
# set the following values:
ingress:
enable: true
secretName: coder-tls
wildcardSecretName: coder-tls
Phorcys
Phorcys•6d ago
check if your certificate was issued successfully kubectl describe certificate coder-cert it'll default to use the ingress controller's invalid cert until the cert is actually issued properly
emircanerkul
emircanerkulOP•6d ago
it is
No description
Phorcys
Phorcys•6d ago
what if you do
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: coder-cert
namespace: coder
spec:
secretName: coder-tls-secret
issuerRef:
name: dttdata-clusterissuer
kind: ClusterIssuer
commonName: coder.lab.dttdata.com
dnsNames:
- coder.lab.dttdata.com
- "*.coder.lab.dttdata.com"
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: coder-cert
namespace: coder
spec:
secretName: coder-tls-secret
issuerRef:
name: dttdata-clusterissuer
kind: ClusterIssuer
commonName: coder.lab.dttdata.com
dnsNames:
- coder.lab.dttdata.com
- "*.coder.lab.dttdata.com"
not sure if that'd change anything though
emircanerkul
emircanerkulOP•6d ago
I just did and applied
Phorcys
Phorcys•6d ago
yeah no it shouldn't because the secret exists
emircanerkul
emircanerkulOP•6d ago
with
- "*.coder.lab.dttdata.com"
- "*.coder.lab.dttdata.com"
quote required
erkulab@erkulab:~$ kubectl get certificate -A
NAMESPACE NAME READY SECRET AGE
coder coder-cert True coder-tls-secret 77m
erkulab@erkulab:~$ kubectl get certificate -A
NAMESPACE NAME READY SECRET AGE
coder coder-cert True coder-tls-secret 77m
I would revert old style, tls inside ingress not in same level. I suspect that
emircanerkul
emircanerkulOP•6d ago
It fixed that issue. IDK why doc says wrong config.. https://coder.com/docs/admin/setup#tls--reverse-proxy helm tls value for ingress look not correct. Correct one is this
ingress:
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
wildcardSecretName: coder-tls-secret
secretNames: coder-tls-secret
ingress:
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
wildcardSecretName: coder-tls-secret
secretNames: coder-tls-secret
No description
emircanerkul
emircanerkulOP•6d ago
Okay now still having same issue.
No description
emircanerkul
emircanerkulOP•6d ago
So the plan is use cloudflare right? Couldnt be anything else? @Phorcys kubernates should smart enough to propagate self signed one to trust any other pods now having another issue.
EWS01: websocket dial: failed to WebSocket dial: failed to send handshake request: Get "https://coder.lab.dttdata.com/api/v2/debug/ws": tls: failed to verify certificate: x509: certificate signed by unknown authority
EWS01: websocket dial: failed to WebSocket dial: failed to send handshake request: Get "https://coder.lab.dttdata.com/api/v2/debug/ws": tls: failed to verify certificate: x509: certificate signed by unknown authority
I just added both tls under coder helm values, also under ingress (thought why not both :D) I prefer to find way to trust my self authority instead going cloudflare way okay CODER_TLS_ALLOW_INSECURE_CIPHERS didnt to the thing. I should fix instead skip or go alternatives
emircanerkul
emircanerkulOP•6d ago
Tataaaa
No description
emircanerkul
emircanerkulOP•6d ago
quotefrom some ai channel: AI never sleeps 😄 Here it is incase someone also need
coder:
env:
- name: CODER_PG_CONNECTION_URL
valueFrom:
secretKeyRef:
name: coder-db-url
key: url
- name: CODER_ACCESS_URL
value: "https://coder.lab.dttdata.com"
- name: CODER_WILDCARD_ACCESS_URL
value: "*.coder.lab.dttdata.com"
- name: CODER_TLS_ENABLE
value: "true"

certs:
secrets:
- name: coder-tls-secret # The name of your TLS secret
key: tls.crt # The key within the secret containing the public certificate (acting as CA)

service:
enable: true
type: ClusterIP
sessionAffinity: ClientIP
externalTrafficPolicy: Cluster
loadBalancerIP: ""
annotations: {}


# tls:
# secretName:
# - coder-tls-secret

ingress:
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
wildcardSecretName: coder-tls-secret
secretNames: coder-tls-secret
coder:
env:
- name: CODER_PG_CONNECTION_URL
valueFrom:
secretKeyRef:
name: coder-db-url
key: url
- name: CODER_ACCESS_URL
value: "https://coder.lab.dttdata.com"
- name: CODER_WILDCARD_ACCESS_URL
value: "*.coder.lab.dttdata.com"
- name: CODER_TLS_ENABLE
value: "true"

certs:
secrets:
- name: coder-tls-secret # The name of your TLS secret
key: tls.crt # The key within the secret containing the public certificate (acting as CA)

service:
enable: true
type: ClusterIP
sessionAffinity: ClientIP
externalTrafficPolicy: Cluster
loadBalancerIP: ""
annotations: {}


# tls:
# secretName:
# - coder-tls-secret

ingress:
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
wildcardSecretName: coder-tls-secret
secretNames: coder-tls-secret
emircanerkul
emircanerkulOP•6d ago
More errors more pains 😄 Idk solving things make me feel good i guess. 😄
No description
Phorcys
Phorcys•5d ago
it's valid coder.tls means the TLS termination that Coder will serve, and ingress.tls is what the ingress serves i should've noticed it but it would've been correcti f there was no ingress in front of coder no need to move it somewhere else as it makes it harder to follow what's going on
emircanerkul
emircanerkulOP•5d ago
yea noticed having both are unnecessary. the think fixed is putting cert but having other issues hope you have time to check it
Phorcys
Phorcys•5d ago
are you still using self signed certs?
emircanerkul
emircanerkulOP•5d ago
looks u remeved those messages yes usıng and works good
Phorcys
Phorcys•5d ago
you have to edit your template's image to trust those certs
emircanerkul
emircanerkulOP•5d ago
at least dont see errors anymore, handshakes are good
Phorcys
Phorcys•5d ago
is there really no way for you to use valid certificates? it will solve this issue self signed is not great
emircanerkul
emircanerkulOP•5d ago
putting cert is fixed. And having other kind issues now but you looks removed those chats, i searched old messages and put mine in related one
Phorcys
Phorcys•5d ago
i know, but the issue you're having is that the workspace cannot connect to your Coder instance because it doesn't trust the certs i only closed the other threads but I saw your messages this happens because of the invalid certs
emircanerkul
emircanerkulOP•5d ago
soo,
certs:
secrets:
- name: coder-tls-secret # The name of your TLS secret
key: tls.crt # The key within the secret containing the public certificate (acting as CA)
certs:
secrets:
- name: coder-tls-secret # The name of your TLS secret
key: tls.crt # The key within the secret containing the public certificate (acting as CA)
this also not enough?
Phorcys
Phorcys•5d ago
no
emircanerkul
emircanerkulOP•5d ago
it made dissappeard those errors
No description
Phorcys
Phorcys•5d ago
yes, but it's different the Kubernetes template spawns a pod and runs the Coder agent install script the agent install script fetches the Coder agent from your Coder deployment and then tries to connect to that deployment
emircanerkul
emircanerkulOP•5d ago
hmm okay then looking on it if there are no easy option for those
Phorcys
Phorcys•5d ago
your workspace's pod does not trust the certificate either so you can either install valid certs, which is the best option or you can edit the image that the pod is running to allow your self signed cert, but that is quite a hassle for reference here's a thread about this -> https://discord.com/channels/747933592273027093/1352111189328396370 the easy option is to configure your cert-manager with cloudflare
emircanerkul
emircanerkulOP•5d ago
okay thank you will go with that. Looks no easy twig there, i was thinking if anything main.tf there make this work
emircanerkul
emircanerkulOP•5d ago
After using https://github.com/cloudflare/origin-ca-issuer and succesfully creating it weirdly it do not get the domains right.
coder:
env:
- name: CODER_PG_CONNECTION_URL
valueFrom:
secretKeyRef:
name: coder-db-url
key: url

- name: CODER_ACCESS_URL
value: "https://coder.lab.dttdata.com"
- name: CODER_WILDCARD_ACCESS_URL
value: "*.coder.lab.dttdata.com"

service:
enable: true
type: ClusterIP
sessionAffinity: ClientIP
externalTrafficPolicy: Cluster
loadBalancerIP: ""
annotations: {}

ingress:
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
secretNames: coder-lab-dttdata-tls
wildcardSecretName: coder-lab-dttdata-tls
coder:
env:
- name: CODER_PG_CONNECTION_URL
valueFrom:
secretKeyRef:
name: coder-db-url
key: url

- name: CODER_ACCESS_URL
value: "https://coder.lab.dttdata.com"
- name: CODER_WILDCARD_ACCESS_URL
value: "*.coder.lab.dttdata.com"

service:
enable: true
type: ClusterIP
sessionAffinity: ClientIP
externalTrafficPolicy: Cluster
loadBalancerIP: ""
annotations: {}

ingress:
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
secretNames: coder-lab-dttdata-tls
wildcardSecretName: coder-lab-dttdata-tls
GitHub
GitHub - cloudflare/origin-ca-issuer: cert-manager issuer for Origi...
cert-manager issuer for Origin CA. Contribute to cloudflare/origin-ca-issuer development by creating an account on GitHub.
No description
emircanerkul
emircanerkulOP•5d ago
also added annotations but nothing changed
ingress:
annotations:
cert-manager.io/issuer: prod-issuer
cert-manager.io/issuer-kind: OriginIssuer
cert-manager.io/issuer-group: cert-manager.k8s.cloudflare.com
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
secretNames: coder-lab-dttdata-tls
wildcardSecretName: coder-lab-dttdata-tls
ingress:
annotations:
cert-manager.io/issuer: prod-issuer
cert-manager.io/issuer-kind: OriginIssuer
cert-manager.io/issuer-group: cert-manager.k8s.cloudflare.com
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
secretNames: coder-lab-dttdata-tls
wildcardSecretName: coder-lab-dttdata-tls
any tip @Phorcys Arent there other coder related good hearted people other than you 😄 btw i think propageting trusted cert into other pods is a bug more than a feature request
Phorcys
Phorcys•4d ago
there are, but i'm usually the only one around here on week-ends and today is easter so there's less people around have you tried with this instead? this is related to the origin CA, i think it's a different thing
emircanerkul
emircanerkulOP•4d ago
ah i see, happy easter. Hmm yea those terms.. need to learn.. i'll go and use acme dns instead origin ca. I thought could be better to use somekind pluging everyday learning new thing and never end 😄 today was doing watch?v=FijNKp3Zdus Disassembly easy but had some hard time while Assembly part
emircanerkul
emircanerkulOP•4d ago
works good but didnt get why i dont green certificate
No description
emircanerkul
emircanerkulOP•4d ago
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: certificate
namespace: coder
spec:
secretName: default-tls
issuerRef:
name: cloudflare-clusterissuer
kind: ClusterIssuer
dnsNames:
- lab.dttdata.com
- '*.lab.dttdata.com'
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: certificate
namespace: coder
spec:
secretName: default-tls
issuerRef:
name: cloudflare-clusterissuer
kind: ClusterIssuer
dnsNames:
- lab.dttdata.com
- '*.lab.dttdata.com'
and in helm values
...
ingress:
annotations:
cert-manager.io/issuer: prod-issuer
cert-manager.io/issuer-kind: OriginIssuer
cert-manager.io/issuer-group: cert-manager.k8s.cloudflare.com
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
secretNames: default-tls
wildcardSecretName: default-tls
...
...
ingress:
annotations:
cert-manager.io/issuer: prod-issuer
cert-manager.io/issuer-kind: OriginIssuer
cert-manager.io/issuer-group: cert-manager.k8s.cloudflare.com
enable: true
host: "coder.lab.dttdata.com"
wildcardHost: "*.coder.lab.dttdata.com"
annotations: {}
tls:
enable: true
secretNames: default-tls
wildcardSecretName: default-tls
...
okay all works good. I thought
dnsNames:
- lab.dttdata.com
- '*.lab.dttdata.com'
dnsNames:
- lab.dttdata.com
- '*.lab.dttdata.com'
was enough. Spesified sub domain
Phorcys
Phorcys•4d ago
great, can we close the issue now?
emircanerkul
emircanerkulOP•4d ago
Sure, thank you for all helps
Phorcys
Phorcys•4d ago
glad you got it figured out! feel free to ask any other questions :-)
Codercord
Codercord•4d ago
@Phorcys closed the thread.

Did you find this page helpful?