Handling request based on source ip with traefik in k8s

I have a k8s cluster (three vms on my own hardware; no aws, google cloud, ...) that uses traefik (https://traefik.io/) as a reverse proxy to address services/deployments in the background.

For this I use the deployment-variant from this part of the documentation: https://docs.traefik.io/user-guide/kubernetes/#deploy-trfik-using-a-deployment-or-daemonset

Now I have multiple applications deployed in the cluster, which all have some ingresses assigned, which are read by the traefik-ingress-controller. Some of those applications are internal ones, like kibana or the traefik-web-ui, and some others are external, like the applications themselves. I distinguish those two by having different dns entries (like https://dashboard.internal.mycoolapp.com and https://app1.external.mycoolapp.com) and the internal dns is not resolved from the outside world (=the internet) whereas the external is (like from google dns).

That's for the setup. Now, let's come to the problem:

A couple of days ago, I thought: What happens, if I create a wildcard dns entry for *.internal.mycoolapp.com on a machine, that is outside my network, and just resolve it to the same ip(s) the external dns entry resolves to. Et voila, my internal services are accessible from the outside!

So this is, of course, a state which is not acceptable. So I'm searching for solutions on this.

What first came to mind was to block all incoming requests on rules for internal services, if the host ip of the requester is outside of our network:

...
kind: Ingress
metadata:
  name: app1
  namespace: default
  annotations:
    traefik.ingress.kubernetes.io/whitelist-source-range: "10.0.0.0/8"
    ingress.kubernetes.io/whitelist-x-forwarded-for: "true"
...

Theoretically this should work. But as I found out later on, before reaching the traefik-ingress-controller, all requests are handled by kube-proxy and their host addresses are translated to local addresses ((S)NAT), so every request has an internal host addre

I think your ingress rule is fine but I think your Traefik service needs to be changed. Check out this article https://kubernetes.io/docs/tutorials/services/source-ip/

I think what you are looking for is nodePort with externalTrafficPolicy set to local.

This should allow Traefik to enforce whitelist-source-range annotation on the ingress.

EDIT:

I was going to go into more detail but it was so old I thought you might have already found a solution. Ingress controllers are not allowed to bind to 80 and 443 but most organizations are using the load balancer service to load balance traffic to the Ingress controller and then out to the kubernetes services and then to the pods.

DNS    HostName Routing       Port routing
LB --> Ingress nodePort --> Service ClusterIP --> Pod

I don't usually use the cloud providers LB. So usually I will wildcard my dns *.mycluster.com and point that to nginx. In nginx I will then have a couple of configs to redirect that traffic to either the ingress controller or some of my services that are running nodePort because they are fragile.

Hope this helps

StackExchange DevOps Q#4441, answer score: 1