Kubernetes Ingress and Gateway with Traefik
7 minute read
The tutorial was written and tested using version 38.0.2 of Traefik’s Helm chart. Please use the latest version instead.
Also, please set up and use TLS for your workloads. Since TLS is not part of this tutorial the related configuration is commented out in the code snippets below.
Install
Create an empty Helm values.yaml file. Traefik runs out‑of‑the‑box, so the file is not required for the initial deployment. However, the following sections will add custom configuration to it.
touch values.yaml
Set the target Kubernetes cluster, for example by pointing the KUBECONFIG environment variable to the kubeconfig file.
export KUBECONFIG=/path/to/kubeconfig/file
Install Traefik into the traefik namespace:
helm install traefik oci://ghcr.io/traefik/helm/traefik \
--create-namespace \
--namespace=traefik \
--values=values.yaml \
--version=38.0.2 \
--wait
Further reading …
Upgrade
Whenever you modify values.yaml, upgrade the release with the following command:
helm upgrade traefik oci://ghcr.io/traefik/helm/traefik \
--namespace=traefik \
--values=values.yaml \
--version=38.0.2 \
--wait
Configure
Reserve the Floating IP Address
If you wish to keep the load balancer’s IP address, add the loadbalancer.openstack.org/keep-floatingip annotation to your values file.
# values.yaml
service:
annotations:
loadbalancer.openstack.org/keep-floatingip: "true"
When you already have a reserved IP address, use it like this:
# values.yaml
service:
annotations:
loadbalancer.openstack.org/keep-floatingip: "true"
loadbalancer.openstack.org/load-balancer-address: "192.0.2.255" # Example IP address.
spec:
loadBalancerIP: "192.0.2.255" # Example IP address.
Further reading …
Use Proxy Protocol
Enabling the proxy protocol allows applications to see the client’s original IP address instead of the load balancer’s internal IP address. Add the annotation to the service and configure each entry point (web / websecure) in the values file:
# values.yaml
ports:
web:
proxyProtocol:
trustedIPs:
- "10.250.0.0/16"
websecure:
proxyProtocol:
trustedIPs:
- "10.250.0.0/16"
service:
annotations:
loadbalancer.openstack.org/proxy-protocol: v2
In the trustedIPs fields, please enter the internal IP address of your load balancer — or the range of possible internal IP addresses it can use. At present, this is the range shown above, which also matches the “Nodes CIDR” that is displayed to you in the PSKE dashboard.
Further reading …
Disable Data Collection
Traefik collects anonymous usage data by default and periodically checks for new releases. If you prefer to disable these features, add the following:
# values.yaml
global:
checkNewVersion: false # Default: true
sendAnonymousUsage: false # Default: false
Further reading …
Configure Logging
Switching the log format to JSON and raising the threshold can simplify log aggregation and reduce noise.
# values.yaml
logs:
general:
format: json # Default: common
level: ERROR # Default: INFO
Further reading …
Enable Access Logs
Traefik access logs are not mandatory in typical deployments but can be useful for troubleshooting or auditing. Enable them with the following settings – with or without filters:
# values.yaml
logs:
access:
enabled: true # Default: false
format: json # Default: common
filters:
minduration: "5" # Default: ""
statuscodes: "400" # Default: ""
Further reading …
Disable Path Filtering and Sanitization
By default Traefik blocks certain URL‑encoded characters and sanitizes request paths. Disable this behavior only if you have a compelling reason to allow those characters.
# values.yaml
ports:
web:
http:
encodedCharacters:
allowEncodedSlash: true # Default: false
allowEncodedBackSlash: true # Default: false
allowEncodedNullCharacter: true # Default: false
allowEncodedSemicolon: true # Default: false
allowEncodedPercent: true # Default: false
allowEncodedQuestionMark: true # Default: false
allowEncodedHash: true # Default: false
sanitizePath: false # Default: true
websecure:
http:
encodedCharacters:
allowEncodedSlash: true # Default: false
allowEncodedBackSlash: true # Default: false
allowEncodedNullCharacter: true # Default: false
allowEncodedSemicolon: true # Default: false
allowEncodedPercent: true # Default: false
allowEncodedQuestionMark: true # Default: false
allowEncodedHash: true # Default: false
sanitizePath: false # Default: true
Further reading …
Use
Prepare a Test Deployment
Create a file named httpbin.yaml with the following content:
# httpbin.yaml
apiVersion: v1
kind: Namespace
metadata:
name: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
namespace: httpbin
spec:
replicas: 1
selector:
matchLabels:
app: httpbin
template:
metadata:
labels:
app: httpbin
spec:
containers:
# Please use an up-to-date version of the container.
- image: ghcr.io/mccutchen/go-httpbin:2.20
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 8080
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
resources:
limits:
memory: 50Mi
requests:
cpu: 50m
livenessProbe:
tcpSocket:
port: 8080
readinessProbe:
httpGet:
path: /status/200
port: 8080
scheme: HTTP
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
namespace: httpbin
labels:
app: httpbin
service: httpbin
spec:
ports:
- name: http
port: 8080
targetPort: 8080
selector:
app: httpbin
Deploy the namespace, deployment, and service:
kubectl apply --filename=httpbin.yaml
Create a Kubernetes Ingress
Append the ingress specification to the end of httpbin.yaml:
# httpbin.yaml
---
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
name: httpbin
namespace: httpbin
spec:
ingressClassName: traefik
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: httpbin
port:
number: 8080
# tls:
# - hosts:
# - example.com
# secretName: httpbin-tls
Apply the Ingress resource:
kubectl apply --filename=httpbin.yaml
Test it with curl:
EXTERNAL_IP=$(kubectl --namespace=traefik get svc/traefik \
--output=jsonpath='{.status.loadBalancer.ingress[0].ip}')
curl --header Host:example.com --verbose "http://${EXTERNAL_IP:?}/status/200"
Further reading …
Add Basic Auth to the Ingress Resource
Create a secret and a Traefik middleware that secure the endpoint with Basic Auth:
# httpbin.yaml
---
kind: Secret
apiVersion: v1
metadata:
name: basic-auth
namespace: httpbin
type: Opaque
data:
# Username: admin
# Password: insecure
# Please use a secure password instead!
auth: YWRtaW46JGFwcjEkWU5iT0trNVkkNlVaa2pKN1dneUpVYWcvYXlqdzE3Lgo=
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: basic-auth
namespace: httpbin
spec:
basicAuth:
secret: basic-auth
Attach the middleware to the Ingress by annotating it. The annotation format is <namespace>-<middleware>@kubernetescrd.
# httpbin.yaml
metadata:
annotations:
traefik.ingress.kubernetes.io/router.middlewares: httpbin-basic-auth@kubernetescrd
Further reading …
Add an IP Allow List to the Ingress Resource
Define an IP allow list middleware:
# httpbin.yaml
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: ip-allow-list
namespace: httpbin
spec:
ipAllowList:
sourceRange:
- 192.0.2.255 # Example
- 198.51.100.0/24 # Example
Again annotate the Ingress to use the middleware:
# httpbin.yaml
metadata:
annotations:
traefik.ingress.kubernetes.io/router.middlewares: httpbin-ip-allow-list@kubernetescrd
You can apply both middlewares simultaneously by separating them with commas:
# httpbin.yaml
metadata:
annotations:
traefik.ingress.kubernetes.io/router.middlewares: |-
httpbin-basic-auth@kubernetescrd,httpbin-ip-allow-list@kubernetescrd
Further reading …
Create a Kubernetes Gateway
Activate Traefik’s Gateway provider in values.yaml and upgrade the release:
# values.yaml
providers:
kubernetesGateway:
enabled: true # Default: false
gateway:
enabled: false # Not used in this tutorial so we can disable it. Default: true
Add a Gateway and an HTTPRoute to httpbin.yaml:
# httpbin.yaml
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: httpbin
namespace: httpbin
spec:
gatewayClassName: traefik
listeners:
- allowedRoutes:
namespaces:
from: Same
name: web # This is the name of Traefik's HTTP endpoint.
port: 8000 # This is the default port number of Traefik's HTTP endpoint.
protocol: HTTP
# - allowedRoutes:
# namespaces:
# from: Same
# name: websecure # This is the name of Traefik's HTTPS endpoint.
# port: 8443 # This is the default port number of Traefik's HTTPS endpoint.
# protocol: HTTPS
# tls:
# certificateRefs:
# - kind: Secret
# name: httpbin-tls
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: httpbin
namespace: httpbin
spec:
parentRefs:
- name: httpbin
namespace: httpbin
hostnames:
- example.com
rules:
- backendRefs:
- name: httpbin # This is the name of your Kubernetes service.
namespace: httpbin
port: 8080 # This is the port number of your Kubernetes service.
matches:
- path:
type: PathPrefix
value: /
Deploy the Gateway and HTTPRoute:
kubectl apply --filename=httpbin.yaml
If you previously created an Ingress for the same route, delete it to avoid confusion:
kubectl --namespace=httpbin delete --ignore-not-found ingress/httpbin
Verify that the Gateway is reachable:
EXTERNAL_IP=$(kubectl --namespace=traefik get svc/traefik \
--output=jsonpath='{.status.loadBalancer.ingress[0].ip}')
curl --header Host:example.com --verbose "http://${EXTERNAL_IP:?}/status/200"
Attach middlewares to the HTTPRoute as filters:
# httpbin.yaml
filters:
- type: ExtensionRef
extensionRef:
group: traefik.io
kind: Middleware
name: ip-allow-list
- type: ExtensionRef
extensionRef:
group: traefik.io
kind: Middleware
name: basic-auth
Further reading …
Uninstall
Before removing Traefik, decide whether you need to keep the floating IP address of the load balancer.
Either reserve the floating IP address …
helm upgrade traefik oci://ghcr.io/traefik/helm/traefik \
--namespace=traefik \
--reuse-values \
--set-string=service.annotations.loadbalancer\\.openstack\\.org/keep-floatingip=true
Take note of the IP so you can reuse it later:
kubectl --namespace=traefik get svc/traefik \
--output=jsonpath='{.status.loadBalancer.ingress[0].ip}'
Or release the floating IP address …
helm upgrade traefik oci://ghcr.io/traefik/helm/traefik \
--namespace=traefik \
--reuse-values \
--set-string=service.annotations.loadbalancer\\.openstack\\.org/keep-floatingip=false
Uninstall the Traefik release:
helm uninstall traefik --namespace=traefik
Before deleting the Traefik CRDs, list any resources that still use them:
kubectl api-resources --api-group=traefik.io --output=name |
tr '\n' ',' | sed -E 's/,$//' |
xargs --no-run-if-empty kubectl get --all-namespaces
kubectl api-resources --api-group=hub.traefik.io --output=name |
tr '\n' ',' | sed -E 's/,$//' |
xargs --no-run-if-empty kubectl get --all-namespaces
If each of the above commands reports “No resources found” – or if you are certain you do not need the resources anymore – delete the CRDs:
kubectl api-resources --api-group=traefik.io --output=name |
xargs --no-run-if-empty kubectl delete crds
kubectl api-resources --api-group=hub.traefik.io --output=name |
xargs --no-run-if-empty kubectl delete crds
Finally, remove the namespace that contained Traefik:
kubectl delete namespace traefik