Service Mesh

Introduction to Service Mesh

Service Mesh is nothing but an underlying networking layer in a microservices environment to enhance their communication with security and observability. This offers encrypted protocols for communication and tracing the networking pattern and load balancing-like features, which are missing by default in microservices networking, so app developers had to write

these pieces of code along with the business logic inside the application. Now service mesh separates this networking overhead from them and implements it as sidecar containers with the application container named Data Plane and manages this with a separate control plane.

Diagram

Advantages and Working of Service Mesh

  1. Managing Routing Requests

  2. Load balancing

  3. Handling Retries and Timeouts

  4. Implementing encryption authentication

  5. Monitoring and tracing

  6. Handling internal mTLS certificates

Managing Routing Requests

As every communication request now goes through the local proxy container first and the control plane is managing that, it can now control the routing of traffic. This helps during deployment by supporting various deployment strategies like canary deployment and A/B or blue-green deployments.

Istio VirtualService example


http:

  - match:

      - uri:

          prefix: /v1/

    route:

      - destination:

          host: user-service

          subset: v1

Load balancing

For this Data-Plane (sidecar proxy container), Service Mesh can now discover all instances (endpoints) of a service. As these sidecar proxies use algorithms like round-robin, least requests, or random to distribute traffic If one instance is slow or down, the proxy can avoid it automatically.


loadBalancer:

  simple: ROUND_ROBIN

Handling Retries and Timeouts

Each proxy also enforces timeout policies by defining how long to wait and retry logic by how many times to retry and when. and this is now configured via the control plane, not in the app code.


retries:

  attempts: 3

  perTryTimeout: 2s

timeout: 5s

Implementing encryption authentication

All microservices can now communicate with the secured, encrypted TLS protocol. The control plane will handle the certificates, and only trusted services can communicate. Service identity verification/authentication can also work by controlling that the frontend can talk to the user service but not the payment service.


action: ALLOW

rules:

  - from:

      - source:

          principals: ["cluster.local/ns/frontend/sa/frontend-service-account"]

Monitoring and tracing

Now as the control plane gets every routing request that passes through the sidecar proxy container, it can trace all the microservices connecting to each other and can monitor them.

by collecting metrics like latency, errors, throughput, etc. It can also integrate with monitoring tools like Prometheus, Grafana, and Datadog by sending logs to them.

Handling internal TLS certificates

The control plane acts as the certificate authority (CA) for the mesh, so it has a root certificate and private key, which is used to sign intermediate or service certificates, and this root certificate is trusted across the entire mesh. This CA can be either internal (self-managed by the mesh) or external by integrating with HashiCorp Vault or AWS PCA.

As each service (pod, container) in the mesh is assigned a service identity, usually tied to a Kubernetes service account. Now When a new workload starts, its sidecar proxy connects to the control plane’s Certificate Authority service and requests a certificate via an agent, like Istio’s istio-agent, and proves who it is using its service account token. After that, the control plane verifies it with the Kubernetes API and issues a short-lived X.509 certificate containing the workload’s identity. The proxy stores this certificate and private key securely (usually in memory). These certificates are short-lived (often 24 hours), and the control plane automatically renews and rotates them before expiry, and this rotation happens transparently while traffic continues to flow.

Now when service A calls service B, both A’s and B’s sidecar proxies perform a mutual TLS handshake, and each sends its certificate, and each verifies the other’s certificate against the mesh root CA or external CA. Now the session is fully encrypted with TLS certificates, which will get an automatic renewal.