Istio Service Mesh Security in Practice: mTLS Mutual Auth and Traffic Encryption
Configure Istio service-mesh mTLS mutual authentication and AuthorizationPolicy to encrypt Kubernetes east-west traffic, authenticate workload identity, and enforce fine-grained access control. For production cluster service hardening.
Scope
Applies to microservices on Kubernetes when east-west traffic carries sensitive data (orders, users, payments), audit requires verifiable service identity, or you want to replace a homegrown crypto middleware. Use Istio service mesh mTLS plus authorization policy for unified security control. Config below targets Istio 1.18+ and Kubernetes 1.24+.
Prerequisites
- Istio installed via istioctl or Helm; istiod healthy
- Target namespace can enable sidecar auto-injection (`istio-injection=enabled` label)
- kubectl cluster access to create PeerAuthentication / AuthorizationPolicy CRDs
How it works
Istio injects an Envoy sidecar into each Pod; all service traffic goes through the sidecar. istiod issues SPIFFE identity certs per workload. Sidecars encrypt with mTLS and mutually verify identity. PeerAuthentication sets mTLS mode: `PERMISSIVE` (plaintext still accepted) or `STRICT` (mutual TLS required). AuthorizationPolicy adds L7 authz beyond the network layer (source identity, HTTP method, path). Cluster default is PERMISSIVE. Production should move to STRICT plus authz policy.
Procedure
Step 1: Enable namespace auto-injection
`# Enable sidecar auto-injection for the target namespace
kubectl label namespace default istio-injection=enabled
# Redeploy test apps (Bookinfo example can be skipped; use your own services)
kubectl rollout restart deployment -n default`
Step 2: Enforce STRICT mTLS
Create a PeerAuthentication resource and switch every service in the namespace to mandatory mutual TLS:
`cat Allow only specified service accounts to reach the target service, and restrict HTTP methods:
`cat If a service exposes both an internal business port and an external health-check port, set mTLS mode per port so health checks are not broken:
`cat Use Prometheus metrics to watch cluster mTLS adoption and request failure rate; catch plaintext traffic early:
`# Fraction of non-mTLS sidecar-to-sidecar requests (should approach 0) istio_requests_total{reporter="destination",connection_security_policy!="mutual_tls"}
# 403 authorization denials — tune AuthorizationPolicy istio_requests_total{response_code="403"}
# Open Grafana Istio dashboards for inter-service security posture istioctl dashboard grafana`
### Step 6: Verify certificates and mTLS state
`# Inspect TLS negotiation between services istioctl authn tls-check deployment/order-api-v1 default
# Inspect sidecar-issued certs and rotation time istioctl proxy-config secret order-api-v1 -o json | jq .`
## Verification
`# 1. Capture traffic and confirm sidecar-to-sidecar encryption on 8443 (default mTLS port) kubectl exec -it deploy/order-api-v1 -c istio-proxy -- \ tcpdump -i eth0 -nn -c 20 port 8080
# 2. Authz check: unauthorized source should get 403 kubectl exec -it deploy/unauthorized-app -- \ curl -s -o /dev/null -w "%{http_code}\n" http://order-api:8080/api/orders/
# 3. After mTLS, plaintext port must not be reachable directly (sidecar rejects) kubectl exec -it deploy/payment-v1 -- curl http://order-api:8080 2>&1 | head`
## FAQ
### Q1: Mass call failures after switching to STRICT?
Usually the target has no sidecar, or still speaks plaintext. Confirm every involved namespace has injection on and completed a rolling restart. During triage, set PeerAuthentication back to PERMISSIVE, then cut to STRICT once all sidecars are ready.
### Q2: Does mTLS duplicate NetworkPolicy?
No. mTLS handles transport encryption and identity. NetworkPolicy handles L3/L4 source allowlists. Stack them: NetworkPolicy for reachability, then mTLS + AuthorizationPolicy for service identity and method-level access.
### Q3: Do sidecar certs need manual renewal?
No. istiod rotates workload certs every 24h by default; restart the process to load the new cert. If certs look wrong, check istiod logs, NTP, and CA config.
### Q4: Off-cluster services or third-party APIs cannot do mTLS. What then?
Declare those destinations `mesh-external` with a Sidecar resource; traffic leaves as plaintext (or TLS origination). Prefer egress TLS plus a destination-host allowlist. Keep STRICT mTLS between in-mesh services to minimize plaintext.
## Summary
Istio service mesh lands zero-trust on east-west traffic via automatic mTLS + AuthorizationPolicy: STRICT encrypts and authenticates; fine-grained authz shrinks lateral movement. Roll out per namespace, PERMISSIVE then STRICT, watch mTLS coverage on the dashboards, and harden the whole cluster incrementally.