How we implemented Reusable CI/CD Pipeline using Git and Tekton.

Our goal is to create a cloud-native CICD pipeline using a simple voting app in NodeJS. The application will just be a backend service with an endpoint. Blog expects you to have prior knowledge about Kubernetes, Docker, and NodeJS.

Pre Requisites

• Minikube.
• Knative Installed on k8’s cluster.
• Google container Registry.
• Tekton Pipeline Installed on the cluster.
• Tekton Triggers installed on the cluster.
• Node JS Project.
• Play with Docker (optional).

Step 1: Install minikube

Follow instructions relevant to your environment by Installing Minikube.

Step 2: Installing Knative

We will just need knative serving component as we will not be using any event-based applications to invoke our service.

Step 3: Google Container Registry

You can create a free account on Google Cloud and have a registry set up in minutes.

Step 4: Tekton Trigger

Tekton trigger is another project created for integration webhook functionality to trigger your pipeline and manage the resources. Tekton Trigger Installation.

Step 5: Tekton Pipeline

Follow the simple steps on Tekton Github Project Tekton Pipeline

Step 6: NodeJS App

Refer: https://github.com/kailashyogeshwar85/knative-nodeapp

It has Knative service specification.

Architecture

Architecture

Components Breakdown

Namespaces: Namespace is like a classroom for segregation of resources eg: The developer can have a dev namespace, QA team can have a namespace, in short, refer to as ns.

Service Accounts: Service account are similar to the user with some predefined permissions to view, edit, list K8’s resources. We will have 2 SA. First will be used for WebhookService and Pipeline component.

Secrets: Secrets are the K8’s resource for saving sensitive data like passwords, tokens. We will have 2 secrets. First will be used for pushing an image to GCR and 2 will be used by Knative during service deployment to launch your pod.

WebhookService: It is a tekton trigger component.

Pipeline: It is a tekton pipeline components.

Service FaaS: It is a final service that will be created which we can also refer to as FaaS (Function as a service) similar to AWS Lambda.

Jobs: Jobs are tasks that you can create in the K8s cluster to perform some operations on the service available inside a cluster. So we will use it to scale up and scale down the magic of Kubernetes. Wow, Sounds cool!!

Additional Components:

• Jaegar: Tool for Distributed Tracing of FaaS
• Tekton Dashboard: For managing and monitoring your pipeline.
• Grafana/Prometheus: Metrics Visualization of your FaaS.

Implementation

Before implementing the below components make sure to follow the below steps.

• Step1: Always create namespaces first.
• Step2: Then create Secrets as secrets are namespace specific and you cannot share them across namespaces.
• Step3: Create SA’s as it should have permission to use the secrets.
• Step 4: Apply CRD’s.

Webhook Service

It is a Tekton trigger component that has three components:

• EventListener: Service responsible for receiving the incoming request. In short a webhook service. Listens for request on port 8000.
• Trigger Binding: Tekton Trigger component that will extract the params from incoming request payload.
• Trigger Template: Tekton Trigger component is the main glue that acts as a bridge between Tekton Trigger & Tekton Pipeline.

Event Listener

Event Listener is the tekton trigger component that will deploy a pod which will provide the endpoint to be used for triggering pipeline.

Event listener will be available only within the cluster to make it available outside you will have to forward the port to the internal port using below command.

Specs

Trigger Binding

Trigger Binding is a tekton trigger component used for binding params from request payload to trigger params.

trigger.binding.yaml

Trigger Template

The trigger template is responsible for providing the params received from the trigger binding component to the pipeline as params.

NOTE: Params in a template should be same as params in trigger binding specification.

template.yaml

Setup Pipeline

Let's create a pipeline first and then map it to the tekton trigger component.

Step 1

We need to create a secret that will be used by knative when launching a pod by pulling images.

K8’s Secrets

• gcr-image-pull: docker-registry secret for pulling images in github-app-faas namespace used by our Knative Service aka FaaS.
• gcr-image-push: for pushing images to Registry used by kaniko tekton tasks in github-deployment namespace.

gcr-pull-image.yaml

pull-secret.yaml

NOTE: Pull secret should be docker-registry credentials.

Use the below snippet to create a docker-registry secret using bash. We need to patch the default account of a namespace as it will be used for pulling images.

image-pull-secret.yaml

Similarly, create a secret for gcr-push-image.yaml in github-deployment namespace.

Service Accounts

We will need a service account for both Tekton Trigger & Pipelines. Service account helps to organize which resources can be created by which Service Account in short SA. Make sure you have proper roles assigned to your service account else your service will be in the Crash step and you will have to debug by inspecting knative serving component logs.

Service Account For Pipeline

pipeline_sa.yaml

Let’s create a Resource CRD

Resources

Tekton has its own resource types. Below are some of them

• Git Resource: Use for cloning a repository
• Pull Request Resource: When used as an input to a task will provide PR related information to the steps.
• Image Resource: It represent a docker image living in a remote registry. eg: asia.gcr.io/PROJECT_NAME/alpine:latest

More Resources: https://github.com/tektoncd/pipeline/blob/master/docs/resources.md

Pipeline Resource:

A resource that will be provided to pipeline tasks as an input to perform steps.

Pipeline

A pipeline is a series of tasks that will be carrying out some tasks eg: build an image from source.

Github Deployment Pipeline

github-pipeline.yaml

Walk Through

• In the above pipeline, we are defining a resource of type: git using resources directive.
• Then we are defining params with default values just like variables that we will be receiving from a trigger template.
• Then we define our tasks build-source and reference it using the name source-to-image and then we provide params to it as well as the resources that are required for building and pushing docker image.

Task

Tasks are steps in your pipeline. Each task is executed in an isolated pod which makes it more performant and loosely coupled.

NOTE: every resource used in a tekton pipeline is mounted to /workspace directory followed by resource name. So git resource with name my-code will be available at path /workspace/my-code.

TaskSpec: source-to-image

Demo

demo

Lessons Learnt

This post has been in draft for more than 2 months and finally, I have determined to publish it.

Never be late in creating a blog involving Kubernetes as packages are constantly changing and on some other day you will find out that the same specification does not work anymore.

Lastly, plan out your side projects that's how you will learn to design and implement the real systems.

Github Repository: https://github.com/kailashyogeshwar85/knative-tekton-cicd