Next Generation CI/CD mit OpenShift

Pipelines, Projekt Tekton und GitOps

Daniel Brintzinger Sebastian Tiggelcamp Ortwin Schneider
Senior Solution Architect DevOps Engineer Solution Architect
Red Hat Consol Red Hat
1
 Agenda

What we’ll
discuss today

Introduction to Project Tekton,

OpenShift Pipelines & GitOps

Live Demo

Lessons Learned and Roadmap

2
 Introduction to
Project Tekton,
OpenShift Pipelines
& GitOps

3
 CONFIDENTIAL Designator

What is
CI/CD?

4
 WHAT IS CI/CD? GENERAL DISTRIBUTION

Continuous Integration and Continuous Delivery (CI/CD)

GIT COMMIT RELEASE

Code Run Debug Build Int Test Package Deploy Stage ...

LOCAL DEVELOPMENT CONTINUOUS INTEGRATION CONTINUOUS DELIVERY

Kubernetes Kubernetes Kubernetes

5
 WHAT IS CI/CD? GENERAL DISTRIBUTION

What is Cloud-Native CI/CD?

Containers Serverless DevOps

Built for container apps and Runs serverless with no CI/CD Designed with microservices
runs on Kubernetes engine to manage and maintain and distributed teams in mind

6
 WHAT IS CI/CD? GENERAL DISTRIBUTION

Why Cloud-Native CI/CD?

Traditional CI/CD Cloud-Native CI/CD

Designed for Virtual Machines Designed for Containers and Kubernetes

Require IT Ops for CI engine maintenance Pipeline as a service with no Ops overheard

Plugins shared across CI engine Pipelines fully isolated from each other

Plugin dependencies with undefined update cycles Everything lifecycled as container images

No interoperability with Kubernetes resources Native Kubernetes resources

Admin manages persistence Platform manages persistence

Config baked into CI engine container Configured via Kubernetes ConfigMaps

7
 WHAT IS CI/CD? GENERAL DISTRIBUTION

Why Cloud-Native CI/CD?

Traditional CI/CD Cloud-Native CI/CD

Designed for Virtual Machines Designed for Containers and Kubernetes

Require IT Ops for CI engine maintenance Pipeline as a service with no Ops overheard

Plugins shared across CI engine Pipelines fully isolated from each other

Plugin dependencies with undefined update cycles Everything lifecycled as container images

No interoperability with Kubernetes resources Native Kubernetes resources

Admin manages persistence Platform manages persistence

Config baked into CI engine container Configured via Kubernetes ConfigMaps

8
 CONFIDENTIAL Designator

OpenShift
Pipelines

9
 GENERAL DISTRIBUTION

OpenShift Pipelines

Kubernetes-native Serverless CI/CD Run pipelines in Standard and Web, CLI, and
declarative with no single server isolated containers with portable to any Visual Studio
Pipelines with to share and all required Kubernetes Code and IDE
Tekton maintain dependencies platform plugins

10
 OPENSHIFT PIPELINES GENERAL DISTRIBUTION

An open-source project for providing a set of shared and standard

components for building Kubernetes-style CI/CD systems

Governed by the Continuous Delivery Foundation

Contributions from Google, Red Hat, Cloudbees, IBM, Pivotal and many more
11
 OPENSHIFT PIPELINES GENERAL DISTRIBUTION

OpenShift Pipelines Architecture

Developer CodeReady Workspaces

Dev Console Tekton CLI Visual Studio Code
(Eclipse Che)
Tools

API

OpenShift Pipelines

CI/CD Operator Extensions Integrations Tasks

Core

Tekton Pipelines

Kubernetes OpenShift

12
 TEKTON CONCEPTS GENERAL DISTRIBUTION

Tekton Concepts

Step Pipeline Resource Condition

Run commands in a container Inputs and outputs to tasks An check that can determine
with volumes, env vars, etc and pipelines (git, image, etc) if a task should be executed

Task Task Run Catalog

A list of steps that run An invocation of a task with An collection of reusable tasks
sequentially in the same pod inputs and outputs

Pipeline Pipeline Run Triggers

A graph of tasks executed in An invocation of a pipeline A Tekton sub-project to start
a certain order with inputs and outputs pipelines based on events

13
 TEKTON CONCEPTS GENERAL DISTRIBUTION

Steps

● Run command or script in a container - name: build

image: maven:3.6.0-jdk-8-slim
● Kubernetes container spec command: [“mvn”]
args: [“install”]
○ Env vars
○ Volumes
○ Config maps - name: parse-yaml
image: python3
○ Secrets
script:|-
#!/usr/bin/env python3
...

14
 TEKTON CONCEPTS GENERAL DISTRIBUTION

Task

● Defines a unit of work to be executed

Task

● A list of steps to run sequentially

Step
● Step containers run in the task pod
Step

● Has inputs, outputs and parameters Step

● Workspaces and results for sharing data Step

● Can run independent of pipelines

15 Example Tasks: Maven Install, AWS CLI, Kubectl Deploy, Security Scan, etc
 GENERAL DISTRIBUTION

kind: Task
metadata:
name: maven
spec:

Maven Task params:

- name: goal
Example type: string
default: package
steps:
- name: mvn
image: maven:3.6.0-jdk-8-slim
command: [ mvn ]
args: [ $(params.goal) ]

16
 TEKTON CONCEPTS GENERAL DISTRIBUTION

Pipeline

● Define Tasks execution order (graph) Pipeline

● Inputs and parameters Task

● Retries tasks
Task
Task
● Conditional task execution
● Workspaces for sharing data between tasks Task

● Reusable across projects

18
 TEKTON CONCEPTS GENERAL DISTRIBUTION

Sharing Data Between Tasks

Task: results
● Task exposes data as variables
● Suitable for small pieces of data
● Examples: commit id and branch name Pipeline

Task: workspaces Task Task

● Shared volumes between tasks
○ Persistent volumes
/workspace/myw
○ Config maps
PersistentVolumeClaim
○ Secrets
● Suitable for large data
● Examples: code, binaries, reports
19
 TEKTON CONCEPTS GENERAL DISTRIBUTION

Conditions
kind: Condition
metadata:

● Defines a single checks

name: deployment-exists
spec:
params:
● Used in conditional tasks - name: appName
check:
● Parameterized image: openshift-cli
script: oc get deployment $(params.app)
● Command run in container kind: Pipeline

○ True: exit code - 0

spec:
tasks:
- name: run-tests
○ False: non-zero exit code taskRef: { image: ui-test-runner }
conditions:
- conditionRef: deployment-exists
params:
- {name: appName, value: api }
20
 TEKTON CONCEPTS GENERAL DISTRIBUTION

PipelineResource

● Inputs and outputs of tasks and pipelines

○ git repository Pipeline

○ image in a registry
PipelineResource
○ cluster credentials Git

○ storage PipelineResource
Image
○ ...and mo
● Decoupled from pipeline definition
● Reusable across pipelines
22
 TEKTON CONCEPTS GENERAL DISTRIBUTION

Triggers
Run pipelines based on events like HTTP webhooks on commit, pull request, etc

Filter or modify webhook payload

e.g. match branch or files changes

Interceptors

webhook
EventListener PipelineRun

TriggerBinding TriggerTemplate

How to map webhook JSON What resources to create (e.g.

payload to variables PipelineRun) using the variables
23
from payload mapping
 Task Catalogs
A growing collection of reusable tasks

25
 TEKTON CONCEPTS GENERAL DISTRIBUTION

OpenShift Pipelines Architecture

Define pipeline
Run pipelines
pipeline-pod-a
Pipeline PipelineRun
pipeline-pod-b
Task Task TaskRun TaskRun

pipeline-pod-c
Pipeline Controllers
(Tekton, ext, ...)
PipelineResource

26
 GENERAL DISTRIBUTION

Tekton Pipeline Example

Code
analysis
PIPELINE Deploy to Integration Deploy to
Build JAR
DEV cluster Tests STAGE cluster
Build app
image

PIPELINE dev cluster stage cluster

RUN git url registry url
cred cred

Container DEV STAGE

Registry Cluster Cluster

27

TASK PIPELINE RESOURCE

 GENERAL DISTRIBUTION

Install OpenShift Pipelines Operator

28
 OPENSHIFT PIPELINES GENERAL DISTRIBUTION

29
 OPENSHIFT PIPELINES GENERAL DISTRIBUTION

30
 OPENSHIFT PIPELINES GENERAL DISTRIBUTION

31
 OPENSHIFT PIPELINES
Pipeline Builder to compose Pipelines from Task library GENERAL DISTRIBUTION

32
 GENERAL DISTRIBUTION

Manage Pipelines with Tekton CLI

33
 GENERAL DISTRIBUTION

Tekton Pipelines VS Code Extension

34
 CONFIDENTIAL Designator

GitOps

35
 What is GitOps?

Git as the source of truth for the state of

the entire system. Changes in Git drive
changes in the system

Repeatable Predictable Auditable Accessible

37
 GitOps Workflow

desired current
state state
Deployment Deployment
Service Service
Route Route
ConfigMap ConfigMap
Pipeline Pipeline
Task Task
... ...

38
 GitOps Workflow

desired reconcile current

state state
Deployment Deployment
Service Service
Route Route
ConfigMap ConfigMap
Pipeline Pipeline
Task Task
... ...

39
 GitOps Workflow

desired current
state oc apply -k
state
(kustomize)

40
 GitOps Workflow

desired current
state oc apply -k
state
(kustomize)

git commit
git merge
41
 GitOps Tooling

Argo CD Kustomize
https://argoproj.github.io/argo-cd
https://kustomize.io/
 Argo CD - What is It?

Argo CD is a declarative, GitOps continuous

delivery tool for Kubernetes.

● Easily deploy applications in a

declarative way
● Synchronizes cluster state with git
repos
● Works with a variety of Kubernetes
deployment tools including:
○ Helm
○ Kustomize
○ Ksonnet/Jsonnet
○ Directories of yaml
● It is not a CI tool
 Argo CD - Synchronizing

Poll/Push

Change in git Pushed to Argo CD Check Synchronize

Sync Status
 Live Demo

46
 Lessons Learned
and Roadmap

47
 Lessons Learned GENERAL DISTRIBUTION

Learnings
● We used Tekton in some internal projects
● Feels native because it is!

48
 Lessons Learned GENERAL DISTRIBUTION

Learnings

● Feels uncommon at first

● A lot of writing bash scripts and building container
images
● Debug? Metrics? Human Input? SCM Connect?
● Sometimes not that easy to write tasks generic

49
 Lessons Learned GENERAL DISTRIBUTION

Learnings

● Moderate UI experience

○ VSCode plugin, good but...

○ Tekton Dashboard, good but...
○ OpenShift Dev Console (Pipeline
Builder), good but...
50
 Lessons Learned GENERAL DISTRIBUTION

Learnings

● OLM update broke Tekton Operator

● Handling Task Results
● Some CRDs went to beta, Pipeline Resources -> Future?
● A lot of resources left after a build (Pods, PVCs etc.). Where
is my build history?
● Triggering Subpipelines -> Results?

51
 Lessons Learned GENERAL DISTRIBUTION

Why it’s great?

● Cloud native! Declarative config, easy to share, easy to understand
● Environment promotion through GitOps
● Member of cd.foundation and part of Jenkins X, Kabanero, Relay…
● Task Catalog and new: Tekton Hub
● Foundational platform for Continuous Delivery
● Focus on core API -> enable 3rd party tooling
● Reusable!! Serverless!
● Will be a supported “product” within OpenShift!!

52
 GENERAL DISTRIBUTION

OpenShift Dev Preview 4

Webhooks
GA
Disconnected install

Pipelines Default tasks

RBAC UX improvements (cont.)
Proxy support
Image stream support

Roadmap Tekton CLI (more commands)

VSCode Plugin (cont.)
Support for finally clauses
Enhanced pipeline resources
Console Dev (view, graph, logs) Console - catalog integration

May 2020

Dec 2019 2HCY20

Tech Preview
Webhook event filtering
More tasks in the catalog
Console - generate pipelines
Console - task snippets
Console - pipeline builder
Workspace and sharing artifacts
Tekton CLI (more commands)
VSCode Plugin (cont.)
 CONFIDENTIAL Designator

Thank you linkedin.com/company/red-hat

youtube.com/user/RedHatVideo
Red Hat is the world’s leading provider of s

enterprise open source software solutions.

facebook.com/redhatinc
Award-winning support, training, and consulting

services make
twitter.com/RedHat
Red Hat a trusted adviser to the Fortune 500.

54