SSL issue while reaching kubernetes dashboard

I'm configuring a kubernetes cluster (using microk8s) and cert-manager. Certificates work fine to traefik dashboard and to other sites but I have an issue with kubernates dashboard as it's already server via SSL (service is on port 443) and I currently expose a Let'sEncrypt certificate.

When I try to reach the dashboard the dashboard's pod logs:

2021/06/17 07:43:20 http: TLS handshake error from 213.215.191.83:39484: remote error: tls: bad certificate
2021/06/17 07:43:23 http: TLS handshake error from 213.215.191.83:39500: remote error: tls: bad certificate

The IngressRoute I'm using is:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: dashboard-k8s
  namespace: kube-system
spec:
  entryPoints:
    - websecure
  routes:
    - match: "Host(`k.example.com`)"
      kind: Rule
      services:
      - name: kubernetes-dashboard
        port: 443
  tls:
    secretName: k8s-dashboard-http-cert

I have no clear idea of how should SSL work in this scenario.
Would it be safe to have dashboard served on 80 and SSL terminated on traefik?

There are multiple ways to expose Dashboard through traefik.

• Traefik terminates HTTPS from client and as client opens new HTTPS request to Dashboard.

+-------------+       HTTPS       +-------------+        HTTPS      +-------------+ 
     |  Client     -----(session 1)-----  Traefik    ------(session 2)---- Dashboard   | 
     +-------------+                   +-------------+                   +-------------+

This is what you are using. The problem with your configuration is, on second session, Traefik will verify Dashboard's TLS certificate.

When Dashboard certificate is not signed by any of CA in Traefik list, it closes connection, then Dashboard raises error message.

If you want to use this scenario, you need to skip Dashboard certificate verification, or add Dashboard certificate to Traefik list of trusted CA.

Below configuration works by skipping Dashboard certificate verification.

apiVersion: traefik.containo.us/v1alpha1
    kind: ServersTransport
    metadata:
      name: mytransport
      namespace: kube-system
    spec:
      serverName: "k.example.com"
      insecureSkipVerify: true
    ---
    apiVersion: traefik.containo.us/v1alpha1
    kind: IngressRoute
    metadata:
      name: dashboard-k8s
      namespace: kube-system
    spec:
      entryPoints:
        - websecure
      routes:
        - match: "Host(`k.example.com`)"
          kind: Rule
          services:
          - name: kubernetes-dashboard
            port: 443
            serversTransport: mytransport
      tls:
        secretName: k8s-dashboard-http-cert

• Traefik passes the TLS traffic to Dashboard without any processing.

+-------------+                   +-------------+                   +-------------+ 
     |  Client     -------HTTPS-------Traefik(passthrough)---------------- Dashboard   | 
     +-------------+                   +-------------+                   +-------------+

In this scenario, Traefik does not do any TLS termination, the traffic passes as it is to Dashboard.

This works for me.

apiVersion: traefik.containo.us/v1alpha1
    kind: IngressRouteTCP
    metadata:
      name: dashboard-k8s
      namespace: kube-system
    spec:
      entryPoints:
        - websecure
      routes:
        - match: HostSNI(`k.example.com`)
          services:
          - name: kubernetes-dashboard
            namespace: kubernetes-dashboard
            port: 443
      tls:
        passthrough: true

• Traefik terminates HTTPS from client and open new plain HTTP request to Dashboard.

+-------------+                   +-------------+                   +-------------+ 
     |  Client     -------HTTPS---------  Traefik    ---------HTTP-------- Dashboard   | 
     +-------------+                   +-------------+                   +-------------+

In this scenario, Dashboard need to provide plain HTTP termination.

I am not sure whether Dashboard support it.

StackExchange DevOps Q#14179, answer score: 7