v2.1
v2.0
v1.0
  1. Release Notes
    1. Release Notes - 2.1.1Latest
    1. Release Notes - 2.1.0
    1. Release Notes - 2.0.2
    1. Release Notes - 2.0.1
    1. Release Notes - 2.0.0
  1. Introduction
    1. Introduction
    1. Features
    1. Architecture
    1. Advantages
    1. Glossary
  1. Installation
    1. Introduction
      1. Intro
      2. Port Requirements
      3. Kubernetes Cluster Configuration
    1. Install on Linux
      1. All-in-One Installation
      2. Multi-Node Installation
      3. High Availability Configuration
      4. Air Gapped Installation
      5. StorageClass Configuration
      6. Enable All Components
    1. Install on Kubernetes
      1. Prerequisites
      2. Install on K8s
      3. Air Gapped Installation
      4. Install on GKE
    1. Pluggable Components
      1. Pluggable Components
      2. Enable Application Store
      3. Enable DevOps System
      4. Enable Logging System
      5. Enable Service Mesh
      6. Enable Alerting and Notification
      7. Enable Metrics-server for HPA
      8. Verify Components Installation
    1. Upgrade
      1. Overview
      2. All-in-One
      3. Multi-node
    1. Third-Party Tools
      1. Configure Harbor
      2. Access Built-in SonarQube and Jenkins
      3. Enable built-in Grafana Installation
      4. Load Balancer plugin in Bare Metal - Porter
    1. Authentication Integration
      1. Configure LDAP/AD
    1. Cluster Operations
      1. Add or Cordon Nodes
      2. High Risk Operations
      3. Uninstall KubeSphere
  1. Quick Start
    1. 1. Getting Started with Multi-tenancy
    1. 2. Expose your App Using Ingress
    1. 3. Compose and Deploy Wordpress to K8s
    1. 4. Deploy Grafana Using App Template
    1. 5. Job to Compute π to 2000 Places
    1. 6. Create Horizontal Pod Autoscaler
    1. 7. S2I: Publish your App without Dockerfile
    1. 8. B2I: Publish Artifacts to Kubernete
    1. 9. CI/CD based on Spring Boot Project
    1. 10. Jenkinsfile-free Pipeline with Graphical Editing Panel
    1. 11. Canary Release of Bookinfo App
    1. 12. Canary Release based on Ingress-Nginx
    1. 13. Application Store
  1. DevOps
    1. Pipeline
    1. Create SonarQube Token
    1. Credentials
    1. Set CI Node for Dependency Cache
    1. Set Email Server for KubeSphere Pipeline
  1. User Guide
    1. Configration Center
      1. Secrets
      2. ConfigMap
      3. Configure Image Registry
  1. Logging
    1. Log Query
  1. Developer Guide
    1. Introduction to S2I
    1. Custom S2I Template
  1. API Documentation
    1. API Documentation
    1. How to Access KubeSphere API
  1. Troubleshooting
    1. Troubleshooting Guide for Installation
KubeSphere®️ 2020 All Rights Reserved.

Graphical CI/CD Pipeline without Jenkinsfile

Edit

We have demonstrated how to create a Jenkinsfile-based pipeline for Spring Boot Project in the last tutorial. It requires users are familiar with Jenkinsfile. However, Jenkinsfile is a non-trivial thing to learn and some people even do not know about it. Therefore, unlike the last tutorial, we are going to show how to create a CI/CD pipeline without Jenkinsfile by visually editing the workflow through KubeSphere console.

Objective

We will use the graphical editing panel in KubeSphere console to create a pipeline, which automates the processes and release the sample project to Kubernetes development environment. If you have tried the Jenkinsfile-based pipeline, the build steps for this tutorial are easy to understand. The sample project in this tutorial is same to the one that we used in the last tutorial.

Prerequisites

Pipeline Overview

The sample pipeline includes the following six stages.

Pipeline

To elaborate every stage:

  • Stage 1. Checkout SCM: Pull the GitHub repository code;
  • Stage 2. Unit test: The pipeline will continue running the next stage only if the unit test is passed;
  • Stage 3. Code Analysis: Configure SonarQube for static code quality check and analysis;
  • Stage 4. Build and Push: Build the image and push the it to DockerHub with tag snapshot-$BUILD_NUMBER where $BUILD_NUMBER is the serial number of the pipeline active list;
  • Stage 5. Artifacts: Generate the artifact (jar package) and save it;
  • Stage 6. Deploy to DEV: Deploy the project to the development environment. It requires an approval in this stage. An email will be sent after the deployment is successful.

Hands-on Lab

Step 1: Create Credentials

We need to create three credentials for DockerHub, Kubernetes and SonarQube respectively. If you have finished the last lab Create a Jenkinsfile-based Pipeline for Spring Boot Project, you already have the credentials created. Otherwise, please refer to create credentials to create them that are used in the pipeline.

Create Credentials

Step 2: Create Project

The sample pipeline will deploy the sample to Kubernetes namespace, thus we need to create a project in KubeSphere. If you do not finish the last lab, please refer to the step to create a project named kubesphere-sample-dev by using project-admin, then invite the account project-regular into this project and assign the role of operator to this account.

Step 3: Create Pipeline

Follow the steps below to create a pipeline using graphical editing panel.

Fill in the basic information

3.1. In the DevOps project, select the Pipeline on the left and click Create.

Create Pipeline

3.2. In the pop-up window, name it graphical-pipeline, click Next.

Advanced Settings

3.3. Keep clicking Add Parameter to add three string parameters as follows. These parameters will be used in the Docker command of the pipeline. Click Create when you are done.

Parameter Type Name Default Value Description
String REGISTRY The sample repository address is docker.io. Image Registry
String DOCKERHUB_NAMESPACE Fill in your DockerHub account which can also be the Organization name under the account. DockerHub Namespace
String APP_NAME Fill the application name with devops-sample. Application Name

Advanced Settings

Step 4: Editing pipeline

This pipeline consists of six stages. We will demonstrate the steps and tasks in each stage.

Stage I: Pull Source Code (Checkout SCM)

The graphical editing panel includes two areas, i.e., canvas on the left and content on the right. It will generate Jenkinsfile after creating a pipeline in the panel, which is much more user-friendly for developers.

Note: Pipeline includes scripted pipeline and declarative pipeline, and the panel supports declarative pipeline. For pipeline syntax, see Jenkins Documentation.

4.1.1. As follows, select node from the drop-down list of agent type in the content area, input maven in the label.

Note: The agent is used to define execution environment. The agent directive tells Jenkins where and how to execute the pipeline or a specific stage. Please refer to Jenkins Agent for further information.

Select Agent

4.1.2. In the canvas area, click the + button to add a stage. Click the box with title No Name that encloses the box Add Step, name it Checkout SCM in the content area on the right of the panel.

Checkout SCM

4.1.3. Click Add Step. Select git from the content area. For now, fill in the pop-up window as follows:

  • Url: Input GitHub repository URL https://github.com/kubesphere/devops-java-sample.git. Please replace the url with your own repository.
  • Credential ID: Leave it blank as it is for using a private repository.
  • Branch: Leave it as blank. Blank means default to master.

When you are done, click OK to save it and you will see the first stage created.

GitHub repository

Stage II: Unit Test

4.2.1. Click + on the right of the stage Checkout SCM to add another stage for performing a unit test in the container, name it Unit Test.

Unit Test

4.2.2. Click Add Step and select container, name it maven, then click OK.

maven

4.2.3. In the content area, click Add nesting steps in the maven container created above to add a nested step. Then select shell and enter the following command in the pop-up window:

mvn clean -o -gs `pwd`/configuration/settings.xml test

Click OK to save it.

maven container

Stage III: Code Analysis

4.3.1. Same as above, click + on the right of the stage Unit Test to continue adding a stage for configuring SonarQube, which is used to perform static code quality analysis in the container, name it Code Analysis.

Code Analysis

4.3.2. Click Add Step in Code Analysis, and select container,name it maven,then click OK.

Code Analysis

4.3.3. Click Add nesting steps in the maven container created above to add a nested step and select withCredentials, Select the previously created credential ID sonar-token and input SONAR_TOKEN in the text variable, then click OK.

withCredentials

4.3.4. In the task withCredential on the right, click Add nesting steps (the first one),then select withSonarQubeEnv, leave the default name sonar, click OK to save it.

Code Analysis

withSonarQubeEnv

4.3.5. Click Add nesting steps (the first one) in the withSonarQubeEnv. Then select shell on the right, enter the following commands in the pop-up window for SonarQube branch and authentication, and click OK to save the information.

mvn sonar:sonar -o -gs `pwd`/configuration/settings.xml -Dsonar.branch=$BRANCH_NAME -Dsonar.login=$SONAR_TOKEN

SonarQube branch

4.3.6. Click on the Add nesting steps (the third one) on the right, select timeout. Input 1 to time, and select Hours in unit.

Click OK to save it.

SonarQube timeout

4.3.7. In the timeout, click Add nesting steps (the first one). Then select waitforSonarQubeGate and keep the default Start the follow-up task after inspection in the popup window.

Click OK to save it.

waitforSonarQubeGate

Stage IV: Build and Push the Image

4.4.1. Similarly, click + on the right of the stage of Code Analysis to add another stage to build and push images to DockerHub, name it Build and Push.

4.4.2. Click Add Step and select container,name it maven,then click OK.

maven container

4.16. Click Add nesting steps in the contain maven, and select shell on the right, enter the following command in the pop-up window:

mvn -o -Dmaven.test.skip=true -gs `pwd`/configuration/settings.xml clean package

4.4.3. Then continue to click Add nesting steps on the right, select shell in the pop-up window, enter the following command to build a Docker image based on the Dockerfile:

Please DO NOT miss the dot . at the end of the command.

docker build -f Dockerfile-online -t $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BUILD_NUMBER .

Build Docker image

Click OK to save it.

4.4.4. Similarly, click Add nesting steps again and select withCredentials on the right. Fill in the pop-up window as follows:

Note: Considering the security, the account information are not allowed to be exposed in plaintext in the script.

  • Credential ID:Select the DockerHub credentials you created, e.g. dockerhub-id
  • Password variable:Enter DOCKER_PASSWORD
  • Username variable:Enter DOCKER_USERNAME

Click OK to save the it.

DockerHub credentials

4.4.5. Click Add nesting steps (the first one) in the withCredentials step created above, select shell and enter the following command in the pop-up window, which is used to log in Docker Hub:

echo "$DOCKER_PASSWORD" | docker login $REGISTRY -u "$DOCKER_USERNAME" --password-stdin

Click OK to save the it.

docker login

4.4.6. As above, click Add nesting steps in the withCredentials step again, choose shell and enter the following command to push the SNAPSHOT image to DockerHub:

docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BUILD_NUMBER

docker push

Stage V: Generate Artifact

4.5.1. Click + on the right of the Build and Push stage, here we add another stage to save artifacts. This example uses the jar package and name it Artifacts.

Save Artifacts

4.5.2. Click Add Step in Artifacts stage, select archiveArtifacts. Enter target/*.jar in the pop-up window, which is used to set the archive path of artifact in Jenkins.

Click OK to save the it.

Artifacts

Stage VI: Deploy to Dev

4.6.1. Click + on the right of the stage Artifacts to add the last stage, name it Deploy to Dev. This stage is used to deploy resources to development environment, namely, the project of kubesphere-sample-dev.

4.6.2. Click Add Step in Deploy to Dev, select input and enter @project-admin in the pop-up window, assigning account project-admin to review this pipeline.

Click OK to save the it.

4.6.3. Click Add Step on the right,select kubernetesDeploy. Fill in the pop-up window as below and click Confirm to save the information:

  • Kubeconfig: select demo-kubeconfig
  • Configuration file path: Enter deploy/no-branch-dev/** which is the related path of the Kubernetes yaml.

Click OK to save the it.

Deploy to Kubernetes

4.6.4. Similarly, click Add Step to send an email notification to the user after the pipeline runs successfully, select mail and fill in the information.

Note: Make sure you have configured email server in ks-jenkins. Please refer to Jenkins email configuration. If not yet, skip this step and you still can run this pipeline.

At this point, the total six stages of the pipeline have been edited completely, click Confirm → Save, it will generate Jenkinsfile as well.

Complete Pipeline

Step 5: Run Pipeline

5.1. The pipeline created by the graphical editing panel needs to be manually run. Click Run, you can see the three string parameters defined in the third step. Click OK to start this pipeline.

Run Pipeline

5.2. You can see the status of the pipeline in the Activity list. Click Activity to view the detailed running status.

5.3. Enter the first activity to view detailed page.

View detailed page

Note: If the previous steps are running correctly, you can see that the pipeline has successfully run to the last stage in a few minutes. Since we set the review step and specify the account project-admin as the reviewer. Therefore, we need to switch to use project-admin to manually review and approve it.

5.4. Log out, and log in with account project-admin. Enter into the pipeline graphical-pipeline of the DevOps project that we used above. Drill into Activity to view the running status. You can see the pipeline has run to the Deploy to DEV stage. Click Proceed to approve it.

Activity

Step 6: View Pipeline

6.1. Log back the account project-regular. After a few minutes, the pipeline runs successfully. Click Activity list in the pipeline to view the current running pipeline serial number. This page shows the running status of each stage in the pipeline.

View Pipeline

6.2. Click Show Logs on the top right of the current page to inspect the logs. The pop-up window shows the specific logs, running status and time of each stage. Click on a specific stage and expand its specific log on the right. You can debug any problems based on the logs which also can be downloaded to your local file for further analysis.

Show Logs

Step 7: Check Code Quality

Back to the Activity page, click Code quality to check the analysis of the code quality for the demo project, which is provided by the SonarQube. The sample code is simple and does not show bugs or vulnerabilities. Click on the SonarQube icon on the right to access SonarQube. Please refer to Access SonarQube to log in.

Check Code Quality

View the Quality Report at SonarQube

Quality report

Step 8: Download Artifacts

Enter the first activity and select Artifacts. You can find the artifact of jar package generated by the pipeline, and you can download it by clicking the icon.

Download Artifacts

Step 9: Verify the Kubernetes Resource

If every stage of the pipeline runs successfully, the Docker image will be automatically built and pushed to your DockerHub account. Finally, the project is deployed to the Kubernetes with a deployment and a service automatically.

9.1. Enter the project kubesphere-sample-dev, click Application Workloads → Workloads to see that ks-sample-dev has been created successfully.

Environment Address Namespace Deployment Service
Dev http://{$Virtual IP}:{$8080} or http://{$Intranet/Public IP}:{$30861} kubesphere-sample-dev ks-sample-dev ks-sample-dev

View Deployment

View Deployment

9.2. Navigate to Service list, you can find the corresponding service has been created. The NodePort exposed by the service is30861 in this example.

View Service

View Service

9.3. Now verify the images pushed to DockerHub. You can see that devops-sample is the value of APP_NAME, while the tag is the value of SNAPSHOT-$BUILD_NUMBER, and $BUILD_NUMBER is the serial number of the activity within pipeline. This tag has also been used in deployment ks-sample-dev.

View DockerHub

View DockerHub

9.4. Since we set an email notification in the pipeline, thus we can verify the email in the mailbox.

Email notification

Step 10: Access the Sample Service

We can access the sample service using command or access in browser. For example, you can use the web kubectl by using account admin as follows:

# curl {$Virtual IP}:{$Port} or curl {$Node IP}:{$NodePort}
curl 10.233.4.154:8080
Really appreciate your star, that's the power of our life.

Congratulation! You have been familiar with using graphical editing panel to visualize your CI/CD workflow.