< img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=3131724&fmt=gif" />
Feynman, Felix
Published on:2022-03-04    The number of views:

Canary Release in Kubernetes with Nginx Ingress

An increasing number of applications are running in cloud-native environments nowadays. Application developers have to effectively develop, build, test, and release their cloud-native applications while minimizing errors in code and reducing impacts on users. One way to address this challenge is to implement canary releases.

This article gives a quick look on the concept of canary release, and then focuses on how to implement a canary release in Kubernetes with Nginx Ingress.

What is Canary Release

Overview

Canary release is an application deployment method that makes new application features available to a limited portion of users. Canary release is widely adopted in the process of continuous delivery for a variety of benefits, such as:

  • Rollout control: Rollout control can be realized over the new application version by allowing a small subset of users to use the canary release before a complete rollout.
  • Easy rollback: Rollback can be quickly implemented in case of any bugs or issues.
  • Real-world testing: With canary release, real-world testing can be conducted in a small scale without exposing all users to potential bugs or issues.

Canary Release with Nginx Ingress

Nginx Ingress has brought a new feature of canary annotations. You can use these canary annotations to configure multiple backend services for your gateway entry and to control traffic allocations. After you set nginx.ingress.kubernetes.io/canary: "true" for an Ingress, you can use the following annotations for canary rules:

  • nginx.ingress.kubernetes.io/canary-by-header
  • nginx.ingress.kubernetes.io/canary-by-header-value
  • nginx.ingress.kubernetes.io/canary-weight
  • nginx.ingress.kubernetes.io/canary-by-cookie

Note

The above rules will be assessed according to the priority sequence: canary-by-header > canary-by-cookie > canary-weight. For more information, see this document.

Generally, the above rules can be divided into the following two categories:

  • Weight-based canary release rules

    weight-based-canary

  • User-based canary release rules

    user-based-canary

Canary Release in Kubernetes with Nginx Ingress

Now, how to implement a canary release in Kubernetes with Nginx Ingress? Let's do a weight-based one together.

Prerequisites

To prepare a Kubernetes cluster, you can refer to this blog. In this article, let's set up a Kubernetes cluster with KubeSphere installed.

Enable Cluster Gateway

  1. Log in to the KubeSphere web console as admin. Go to Platform > Cluster Management > Cluster Settings > Gateway Settings and click Enable Gateway on the Cluster Gateway tab.

  2. In the displayed dialog box, select the NodePort access mode for the gateway and click OK.

  3. You can view the details of the gateway on the page.

    cluster-gateway

Create an application

  1. Let's create a namespace first.

    kubectl create ns ingress-demo
    
  2. Create a YAML file production.yaml with the following content.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: production
      labels:
        app: production
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: production
      template:
        metadata:
          labels:
            app: production
        spec:
          containers:
          - name: production
            image: mirrorgooglecontainers/echoserver:1.10
            ports:
            - containerPort: 8080
            env:
              - name: NODE_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              - name: POD_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              - name: POD_NAMESPACE
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
              - name: POD_IP
                valueFrom:
                  fieldRef:
                    fieldPath: status.podIP
       
    ---
       
    apiVersion: v1
    kind: Service
    metadata:
      name: production
      labels:
        app: production
    spec:
      ports:
      - port: 80
        targetPort: 8080
        protocol: TCP
        name: http
      selector:
        app: production
    
  3. Run the following command to apply it.

    kubectl apply -f production.yaml -n ingress-demo
    

Create an ingress

Since the application is ready now, let's create an Ingress for it.

  1. Create a file production.ingress with the following content.

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: production
      annotations:
        kubernetes.io/ingress.class: nginx
    spec:
      rules:
      - host: kubesphere.io
        http:
          paths:
          - backend:
              serviceName: production
              servicePort: 80
    
  2. Run the following command to apply it.

    kubectl apply -f production.ingress -n ingress-demo
    
  3. Run the following command to access the application.

    $ curl --resolve kubesphere.io:32141:192.168.2.2 kubesphere.io:32141
       
    Hostname: production-5d84fbdb58-964mq
       
    Pod Information:
     node name:	qkcp
     pod name:	production-5d84fbdb58-964mq
     pod namespace:	ingress-demo
     pod IP:	10.233.96.107
       
    Server values:
     server_version=nginx: 1.13.3 - lua: 10008
       
    Request Information:
     client_address=10.233.96.105
     method=GET
     real path=/
     query=
     request_version=1.1
     request_scheme=http
     request_uri=http://kubesphere.io:8080/
       
    Request Headers:
     accept=*/*
     host=kubesphere.io:32141
     user-agent=curl/7.29.0
     x-forwarded-for=192.168.2.2
     x-forwarded-host=kubesphere.io:32141
     x-forwarded-port=80
     x-forwarded-proto=http
     x-forwarded-scheme=http
     x-real-ip=192.168.2.2
     x-request-id=5f525b4511d9b4ba7f86f2ed9c90ab5f
     x-scheme=http
       
    Request Body:
     -no body in request-
    

    Note

    192.168.2.2 is the gateway address of the project ingress-demo and 32141 is the node port. You need to replace them with the actual values from your environment.

Create a canary version of the application

Next, let's create a canary version of the application to implement a canary release.

  1. Create a YAML file canary.yaml with the following content.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: canary
      labels:
        app: canary
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: canary
      template:
        metadata:
          labels:
            app: canary
        spec:
          containers:
          - name: production
            image: mirrorgooglecontainers/echoserver:1.10
            ports:
            - containerPort: 8080
            env:
              - name: NODE_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              - name: POD_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              - name: POD_NAMESPACE
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
              - name: POD_IP
                valueFrom:
                  fieldRef:
                    fieldPath: status.podIP
       
    ---
       
    apiVersion: v1
    kind: Service
    metadata:
      name: canary
      labels:
        app: canary
    spec:
      ports:
      - port: 80
        targetPort: 8080
        protocol: TCP
        name: http
      selector:
        app: canary
    
  2. Run the following command to apply it.

    kubectl apply -f canary.yaml -n ingress-demo
    

Set a weight-based canary release

  1. Create a file weighted-canary.ingress with the following content.

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: canary
      annotations:
        kubernetes.io/ingress.class: nginx
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-weight: "30"
    spec:
      rules:
      - host: kubesphere.io
        http:
          paths:
          - backend:
              serviceName: canary
              servicePort: 80
    
  2. Run the following command to apply it.

    kubectl apply -f weighted-canary.ingress -n ingress-demo
    
  3. Run the following command to verify the weight-based canary release.

    Note

    Although we set 30% traffic to the canary version of the application, the actual traffic ratio may fluctuate to a small extent.
    for i in $(seq 1 10); do curl -s --resolve kubesphere.io:32141:192.168.2.2 kubesphere.io:32141 | grep "Hostname"; done
    

    The output may look as follows:

    Hostname: canary-c66cf4f8c-zntfs
    Hostname: production-5d84fbdb58-964mq
    Hostname: production-5d84fbdb58-964mq
    Hostname: production-5d84fbdb58-964mq
    Hostname: production-5d84fbdb58-964mq
    Hostname: production-5d84fbdb58-964mq
    Hostname: production-5d84fbdb58-964mq
    Hostname: canary-c66cf4f8c-zntfs
    Hostname: production-5d84fbdb58-964mq
    Hostname: canary-c66cf4f8c-zntfs
    
  4. The weight-based canary release is implemented. If you want to use other annotations for canary rules, you can add annotations to the ingress on the KubeSphere web console.

    add-annotation

Canary release on KubeSphere

Based on Istio, KubeSphere provides users with necessary control to deploy canary services. All the canary configurations can be made on its wizard web console. For more information, see Canary Release.

canary-on-console

canary-details

Recap

This article hopes to help you understand how to implement canary release in Kubernetes with Nginx Ingress through real-world practices. As cloud-native technologies continue to gain momentum, canary release will be helpful when it comes to improving application development workflow.

Reference

Nginx Ingress Controller - Annotations

Canary Deployments on Kubernetes without Service Mesh

Canary deployment with ingress-nginx

close

Receive the latest news, articles and updates from KubeSphere