Übersicht on IT-CNE Cloud-Native Core

1.1 Tasks: Setup

Mon, 01 Jan 0001 00:00:00 +0000

Task 1.1.1: Getting Started

In Bearbeitung.

Task 1.1.2: Install Prometheus

Mit ArgoCD

In Bearbeitung.

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: <user>-app-of-apps
  namespace: argocd
spec:
  destination:
    namespace: argocd
    server: https://kubernetes.default.svc
  project: default
  source:
    repoURL: 'https://gitea.training.cluster.acend.ch/<user>/prometheus-training-lab-setup'
    path: apps/
    targetRevision: main
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - Replace=true

kubectl argocd create -f user-app-of-apps.yaml

Deploy Prometheus

Wir verwenden diese Helm-Chart: https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/README.md

helm install -f kube-prom-stack-values.yaml -n monitoring-system prometheus-stack oci://ghcr.io/prometheus-community/charts/kube-prometheus-stack

# kube-prom-stack-values.yaml
prometheusOperator:
  enabled: true
prometheus:
  enabled: true
  ingress:
    enabled: true
    # ingressClassName: "traefik"
    hosts:
      - prometheus.app.lernvirt.ch

thanosRuler:
  enabled: true
  ingress:
    enabled: true
    # ingressClassName: "traefik"
    hosts:
      - thanosruler.app.lernvirt.ch
grafana:
  enabled: true
  ingress:
    enabled: true
    # ingressClassName: "traefik"
    hosts:
      - grafana.app.lernvirt.ch

alertmanager:
  enabled: true
  ingress:
    enabled: true
    # ingressClassName: "traefik"
    hosts:
      - alertmanager.app.lernvirt.ch

Configure Prometheus

As mentioned in the introduction, configuring a Prometheus on Kubernetes can be done using the Prometheus Operator. We basically need to specify a Prometheus custom resource and the Operator will do its work. In the prometheus custom resource’s spec block you can find various configuration options:

2.1 Tasks: Thanos Querier

Mon, 01 Jan 0001 00:00:00 +0000

To have centralized access to the metrics of both the shared infrastructure Prometheus instance as well as your own userwokload Prometheus instance, we will install the Thanos Querier in this lab. This will give us a central point of view for the following prometheus instances:

Infrastructure Prometheus (K8S Metrics, Node Metrics, …) in the global monitoring namespace
Userworkload Prometheus (the stack we can use for our application metrics) in the <user>-monitoring namespace

Task 2.1.1: Install Thanos Querier

To install the Thanos Querier, change query.enabled to true in the values.yaml of your user-monitoring Helm release. ArgoCD will automagically install all needed components in your namespace.

2.2 Tasks: Explore metrics

Mon, 01 Jan 0001 00:00:00 +0000

In this lab you are going to explore various metrics, which your Prometheus server is collecting.

Task 2.2.1: Prometheus/Thanos web UI

As we have now attached the Thanos Querier to your own Prometheus as well as the infrastructure Prometheus, we will use its UI to have a global view.

Get a feel for how to use the Thanos Querier web UI. Open the web UI and navigate to the Graph menu (right on top in the grey navigation bar next to Stores).

2.3 Tasks: PromQL

Mon, 01 Jan 0001 00:00:00 +0000

In this lab you are going to learn a bit more about PromQL (Prometheus Query Language).

PromQL is the query language that allows you to select, aggregate and filter the time series data collected by prometheus in real time.

Note

Execute the queries in the Thanos Querier UI.

PromQL can seem overwhelming. It may take a little time to get used to it. There may be different approaches to solve the tasks. Our solution is just one possibility.

3.1 Tasks: Grafana intro

Mon, 01 Jan 0001 00:00:00 +0000

Task 3.1.1: Install Grafana

Similar to the basic setup, we are just going to update our configuration of the ArgoCD application to install and create our Grafana instance. Update your monitoring application (charts/user-monitoring/values.yaml) and update the grafana.enabled flag to true:

charts/user-monitoring/values.yaml:

user: <user> # Replace me
# prometheus
prometheus:
  enabled: true
# thanos-query
query:
  enabled: true
# grafana
grafana:
  enabled: true
# blackboxexporter
blackboxexporter:
  enabled: false
# pushgateway
pushgateway:
  enabled: false
# alertmanager
alertmanager:
  enabled: false
# thanos-ruler
ruler:
  enabled: false

Verify the installation and sync process in the ArgoCD UI. When the application is synchronized successfully navigate to your freshly created Grafana instance: https://grafana.app.lernvirt.ch. Use the admin account to login in. Skip the password change.

3.2 Tasks: Grafana dashboards

Mon, 01 Jan 0001 00:00:00 +0000

Task 3.2.1: Import a dashboard

As a first Task we import an existing dashboard to our grafana. There is a huge collection of predefined dashboards available on https://grafana.com/grafana/dashboards.

In this task we learn how to import existing dashboards, which we can also use as reference to write our own queries and dashboards.

Choose one or more of the preexisting dashboards from https://grafana.com/grafana/dashboards and import them into your grafana. Use the daterange, interval picker on the top right to change the timerange (between 5 minutes and 10 minutes) of the displayed mertics.

4.1 Tasks: Blackbox exporter

Mon, 01 Jan 0001 00:00:00 +0000

Task 4.1.1: Install Blackbox exporter

Similar to the basic setup, we are just going to update our configuration of the ArgoCD application to install the Blackbox exporter. Update your monitoring application (charts/user-monitoring/values.yaml) and update the blackboxexporter.enabled flag to true:

charts/user-monitoring/values.yaml:

user: <user> # Replace me
# prometheus
prometheus:
  enabled: true
# thanos-query
query:
  enabled: true
# grafana
grafana:
  enabled: true
  datasources:
  - name: prometheus
    access: proxy
    editable: false
    type: prometheus
    url: http://prometheus-operated:9090
# blackboxexporter
blackboxexporter:
  enabled: true
# pushgateway
pushgateway:
  enabled: false
# alertmanager
alertmanager:
  enabled: false
# thanos-ruler
ruler:
  enabled: false

Commit and push the changes.

4.2 Tasks: Pushgateway

Mon, 01 Jan 0001 00:00:00 +0000

Task 4.2.1 - Install and configure Pushgateway

Update your monitoring application (charts/user-monitoring/values.yaml) and update the blackboxexporter.enabled flag to true:

charts/user-monitoring/values.yaml:

user: <user> # Replace me
# prometheus
prometheus:
  enabled: true
# thanos-query
query:
  enabled: true
# grafana
grafana:
  enabled: true
  datasources:
  - name: prometheus
    access: proxy
    editable: false
    type: prometheus
    url: http://prometheus-operated:9090
# blackboxexporter
blackboxexporter:
  enabled: true
# pushgateway
pushgateway:
  enabled: true
# alertmanager
alertmanager:
  enabled: false
# thanos-ruler
ruler:
  enabled: false

Commit and push the changes and verify whether the deployment worked correctly.

6.1 Tasks: Instrumenting

Mon, 01 Jan 0001 00:00:00 +0000

Task 6.1.1: Get the source code

git clone https://github.com/acend/prometheus-training-go-instrumentation.git

Change into the freshly cloned git repository

cd prometheus-training-go-instrumentation

TODO: create Dockerfile This application creates a simple webserver which listens at http://localhost:8083 and returns the string Prometheus Training after a random interval between 0 and 1000 milliseconds. You can test the app by building it as a docker image using docker build -t prometheus-training-go-instrumentation:local . and then running the built image using docker run --rm -p 8083:8083 prometheus-training-go-instrumentation:local. You can then (in a separate terminal) issue a request using the command curl http://localhost:8083.

6.1 Tasks: Instrumenting

Mon, 01 Jan 0001 00:00:00 +0000

Task 6.1.1: Spring Boot Example Instrumentation

Using the micrometer metrics facade in Spring Boot Applications lets us collect all sort of metrics within a Spring Boot application. Those metrics can be exported for Prometheus to scrape by a few additional dependencies and configuration.

Let’s have a deeper look at how the instrumentation of a Spring Boot application works. For that we can use the prometheus-training-spring-boot-example application located at https://github.com/acend/prometheus-training-spring-boot-example. To make the application collect metrics and provide a Prometheus endpoint we now need to simply add the following two dependencies in the pom.xml file, where it says :

7.1 Tasks: Enable and configure Alertmanager

Mon, 01 Jan 0001 00:00:00 +0000

Task 7.1.1: Install Alertmanager and Thanosruler

Update your monitoring application (charts/user-monitoring/values.yaml) and update the alertmanager.enabled and ruler.enabled flag to true:

charts/user-monitoring/values.yaml:

user: <user> # Replace me
# prometheus
prometheus:
  enabled: true
# thanos-query
query:
  enabled: true
# grafana
grafana:
  enabled: true
  datasources:
  - name: prometheus
    access: proxy
    editable: false
    type: prometheus
    url: http://prometheus-operated:9090
# blackboxexporter
blackboxexporter:
  enabled: true
# pushgateway
pushgateway:
  enabled: true
# alertmanager
alertmanager:
  enabled: true
# thanos-ruler
ruler:
  enabled: true

Commit and push the changes.

Verify the installation and sync process in the ArgoCD UI. Or execute the following command:

7.2 Tasks: Alertmanager

Mon, 01 Jan 0001 00:00:00 +0000

Task 7.2.1: Configure Slack as alert receiver

As part of our lab setup we already configured an alert receiver for Alertmanager alerts.

We deployed the Mail catcher, which is a very simple component. We can send Emails to the server, and those are then displayed in the Web UI. The Emails are not send anywhere, and this setup should only be used for demo purposes.

When we enabled the alertmanager configuration in the charts/user-monitoring/values.yaml we also deployed the following AlertmanagerConfig Custom Resource

7.3 Tasks: Alertrules and alerts

Mon, 01 Jan 0001 00:00:00 +0000

Note

For doing the alerting lab it’s useful to have a “real” application so that alerts can be provoked. You will use the demo app installed in your monitoring-demo namespace for this purpose.

The example app exposes metrics which are already scraped from lab one.

{job="example-web-python"}

The Prometheus Operator allows you to configure Alerting Rules (PrometheusRules). This enables Kubernetes users to configure and maintain alerting rules for their projects. Furthermore it is possible to treat Alerting Rules like any other Kubernetes resource and lets you manage them in Helm or Kustomize. A PrometheusRule has the following form:

Web terminal

Mon, 01 Jan 0001 00:00:00 +0000

Using OpenShift’s web terminal might be more convenient for you as it doesn’t require you to install kubectl locally on your computer.

Note

If you do change your mind, head right over to 1.1.1. Local usage.

First of all, open your browser. Then, log in on OpenShift’s web console using the URL and credentials provided by your trainer.

Task 2: Initialize terminal

Warning

Make sure to create a dedicated project for the web terminal!

4.3 Tasks: Exporter as a sidecar

Mon, 01 Jan 0001 00:00:00 +0000

Task 4.3.1: Deploy a database and use a sidecar container to expose metric

As we’ve learned in Lab 4 - Prometheus exporters when applications do not expose metrics in the Prometheus format, there are a lot of exporters available to convert metrics into the correct format. In Kubernetes this is often done by deploying so called sidecar containers along with the actual application.

This lab demonstrates that with a mariadb database. In the first step, we are going to install a plain mariadb with a Secret (username password to access the database), a Service and the Deployment.

5.1 Tasks: kube-prometheus metrics

Mon, 01 Jan 0001 00:00:00 +0000

Task 5.1.1: Memory usage of Prometheus

Task description:

Display the memory usage of both Prometheus pods
Use a filter to just display metrics from your prometheus containers

Use the [Thanos Querier web UI](https://thanos-query.app.lernvirt.ch to execute the queries.

Note

Search for a metric with memory_working_set in its name

Hints

container_memory_working_set_bytes{pod=~"prometheus-prometheus-0", container="prometheus",namespace=~"<user>.*"}

Task 5.1.2: Kubernetes pod count

Task description:

Display how many pods are currently running on your Kubernetes platform

Hints

There are different ways to archive this. You can for example query all running containers and group them by pod and namespace.

count(sum(kube_pod_container_status_running == 1) by (pod,namespace,cluster))

You may also sum() all running pods on your Kubernetes nodes

sum(kubelet_running_pods)

5.2 Tasks: Prometheus Operator

Mon, 01 Jan 0001 00:00:00 +0000

5.2.1: Prometheus storage

By default, the Prometheus operator stack does not persist the data of the deployed monitoring stack. Therefore, any pod restart would result in a reset of all data. Let’s learn about persistence for Prometheus.

Task description:

See this example of how to configure storage for Prometheus

To get the current values, either look at the file in git or use kubectl -n $USER-monitoring get prometheus prometheus -oyaml.

Hints

To enable storage for our Prometheus we would simply add the following section to our Prometheus definition.

...
spec:
  ...
  securityContext:
    fsGroup: 65534
  storage:
    disableMountSubPath: true
    volumeClaimTemplate:
      spec:
        resources:
          requests:
            storage: 10Gi
  ...

Windows

Mon, 01 Jan 0001 00:00:00 +0000

Installation for Windows

Follow the steps outlined in https://kubernetes.io/docs/tasks/tools/install-kubectl/.

In case the installation from the official package repositories didn’t work (or a specific version is needed) the static binary can be downloaded and put into the following path:

C:\Kubernetes\

In Windows, the PATH can be set in the advanced system settings. It depends on the version:

Note: Windows Quick Hack

Copy the kubectl binary directly into the folder C:\Windows.

YAML

Mon, 01 Jan 0001 00:00:00 +0000

YAML Ain’t Markup Language (YAML) is a human-readable data-serialization language. YAML is not a programming language. It is mostly used for storing configuration information.

Note

Data serialization is the process of converting data objects, or object states present in complex data structures, into a stream of bytes for storage, transfer, and distribution in a form that can allow recovery of its original structure.

As you will see a lot of YAML in our Kubernetes basics course, we want to make sure you can read and write YAML. If you are not yet familiar with YAML, this introduction is waiting for you. Otherwise, feel free to skip it or come back later if you meet some less familiar YAML stuff.

macOS

Mon, 01 Jan 0001 00:00:00 +0000

Installation for macOS

Follow the steps outlined in https://kubernetes.io/docs/tasks/tools/install-kubectl/.

In case the installation from the official package repositories didn’t work (or a specific version is needed) the static binary can be downloaded and put into the following path:

~/bin

File mode

The kubectl binary has to be executable:

cd ~/bin
chmod +x kubectl

PATH variable

In macOS, the directory ~/bin should already be part of the PATH variable. In case kubectl is placed in a different directory, you can change the PATH variable with the following command:

Linux

Mon, 01 Jan 0001 00:00:00 +0000

Installation for Linux

Follow the steps outlined in https://kubernetes.io/docs/tasks/tools/install-kubectl/.

In case the installation from the official package repositories didn’t work (or a specific version is needed) the static binary can be downloaded and put into the following path:

~/bin

File mode

The kubectl binary has to be executable:

cd ~/bin
chmod +x kubectl

PATH variable

In Linux, the directory ~/bin should already be part of the PATH variable. In case kubectl is placed in a different directory, you can change the PATH variable with the following command:

Console login

Mon, 01 Jan 0001 00:00:00 +0000

First of all, open your browser. Then, log in on OpenShift’s web console using the URL and credentials provided by your trainer.

In order to log in on the command line, copy the login command from the web console.

To do that, open the Web Console and click on your username that you see at the top right, then choose Copy Login Command.

Verification

Mon, 01 Jan 0001 00:00:00 +0000

Verify the installation

You should now be able to execute kubectl in the command prompt. To test, execute:

kubectl version

You should now see something like (the version number may vary):

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:58:59Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
...

If you don’t see a similar output, possibly there are issues with the PATH variable.

Warning

Make sure to use at least version 1.16.x for your kubectl

Optional Kubernetes power tools

Mon, 01 Jan 0001 00:00:00 +0000

Optional Kubernetes power tools

kubectx and kubens are two handy shell scripts which let you easily switch between Kubernetes contexts and namespaces. See https://github.com/ahmetb/kubectx for detailed instructions.

Installation of kubectx and kubens:

curl https://raw.githubusercontent.com/ahmetb/kubectx/master/kubectx -o ~/bin/kubectx
curl https://raw.githubusercontent.com/ahmetb/kubectx/master/kubens -o ~/bin/kubens
chmod +x ~/bin/kubectx ~/bin/kubens

kube-ps1 is another helpful shell script which adds the current context and namespace to the shell prompt: https://github.com/jonmosco/kube-ps1

fzf is yet another handy helper tool when you have to deal with a lot of contexts or namespaces by adding an interactive menu to kubectx and kubens: https://github.com/junegunn/fzf

Other ways to work with OpenShift

Mon, 01 Jan 0001 00:00:00 +0000

Other ways to work with OpenShift

If you don’t have access to a running OpenShift development environment (anymore), there are several options to get one.

OpenShift Developer Sandbox: 30 days of no-cost access to a shared cluster on OpenShift
OpenShift Local: A local OpenShift environmennt running on your machine
OKD single node installation: OKD (OpenShift community edition) single node installation

Next steps

When you’re ready to go, head on over to the labs and begin with the training!

6.1 Deploy your awesome application

Mon, 01 Jan 0001 00:00:00 +0000

Using the generated and modified Helm chart, we are going to deploy our own awesome application.

Task 6.1.1: Deploy the chart

Let’s deploy our awesome application. Therefore we need to adjust ingress configuriation in the values file.

ingress:
  enabled: true
  hosts:
    - host: helm-complex-chart-<namespace>.webconsole.app.lernvirt.ch
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls:
    - hosts:
        - helm-complex-chart-<namespace>.webconsole.app.lernvirt.ch

Solution

Note

Don’t forget to replace webconsole.app.lernvirt.ch with the value provided by the trainer.

To create a release from our chart, we run the following command within our chart directory:

6.2 Template the database backend

Mon, 01 Jan 0001 00:00:00 +0000

In this lab we are going to templates the resources that are necessary to deploy a MariaDB database as a backend to our example-web-python application. Before we start creating those templates we want to have a look at a couple of best practices.

Resource names

When looking at the app templates of our mychart chart, the name of the resource is always defined by a helper function:

name: {{ include "helm-complex-chart.fullname" . }}

Resources within the same Namespace and of the same resource type must have unique names. Since a Helm chart can be instantiated multiple times in different releases, it’s best to include the release name as part of the resource name:

6.3 Backend as a Dependency

Mon, 01 Jan 0001 00:00:00 +0000

In the previous lab we’ve manually added a database deployment to our awesome application. Instead of creating your own templates and deployments, we can also integrate other helm charts as dependencies. We have seen this already in lab 3 but are now going to go into more detail in this lab.

Helm Dependencies

Helm dependencies allow you to integrate other helm charts within your helm chart. It helps to reduce code duplication and to centralize certain functionality and to simply use well maintained helm charts in your deployments.

6.4 Test

Mon, 01 Jan 0001 00:00:00 +0000

Task 6.4.1: Add a test to your chart

As we learned in the previous section, Helm gives us the availability to run automated tests during the Helm deployment. Now it’s time to write our first test. The test should meet the following requirements:

Image: mariadb
Env Variables:
- MYSQL_DATABASE_HOST from secret
- MYSQL_DATABASE_USER from secret
- MYSQL_DATABASE_PASSWORD from secret
Command: mysql --host=$MYSQL_DATABASE_HOST --user=$MYSQL_DATABASE_USER --password=$MYSQL_DATABASE_PASSWORD
Annotation: helm.sh/hook: test

Open the test file templates/tests/test-connection.yaml and start to modify it.

6.5 Helm hooks

Mon, 01 Jan 0001 00:00:00 +0000

In the previous lab we learned how to deploy our python example application and connect it to a MariaDB. In this chapter we are going to look at a mechanism of Helm called hooks, which let’s chart developers intervene / interact at certain points in a release’s life cycle.

Task 6.5.1: Chart Hooks

At some point when developing helm charts we would like to interact or alter certain behaviour of our deployed resources. For example in this usecase we would like to have test data available in our database when releasing our chart. With the hook functionality of helm we can alter the life cycle and intervene at certain points. Hooks are regular templates specially annotated that causes Helm to utilize them differently. The tests used in the previous chapter are a special case of helm hooks. The following hooks are available for you to alter the behaviour of the chart:

6.6 Deploy your Chart with different environment configurations

Mon, 01 Jan 0001 00:00:00 +0000

One of the most typical challenges when deploying an application is the handling of different deployment environments.

A typical use case is to deploy your application into a Development and Production environment. Helm is a great solution for handling this challenge. In this lab we are going to show you how to prepare your Helm releases for different environments.

Task 6.6.1 Create a devlopment release

Let us start with a new value file for the Development environment. Create a new file called values-dev.yaml and add following content to the file

6.7 Optional: Publish your chart

Mon, 01 Jan 0001 00:00:00 +0000

In this lab we will learn how to publish our Helm chart and make it accessible to the public by using GitHub pages.

Note

To work through this lab, you need a personal GitHub account. See Hosting chart repositories for a lot of different strategies how to serve a Helm repository.

GitHub pages provides an easy way to expose static files over HTTP(S) to the public. Everyone with a GitHub account can use this feature. The files to expose must be located in the docs/ subdirectory or on a separate branch which can be configured in the repository settings.

7.1 Refactor 1

Mon, 01 Jan 0001 00:00:00 +0000

Task 7.1.1: Make the Ingress resource more configurable

After cloning the repository and inspecting the templates already created for you, you will notice some potential for improvement. For example the hostname of the applications should not be hard-coded. When deploying to multiple environments you will run into conflicts.

Modify the consumer and producer ingress templates and extract following variables to make them configurable:

Extract .spec.rules.host as value
Extract .spec.tls.hosts[0] as value, use the same value as above

Note

If you want to test the chart locally you can execute following command

helm template -s templates/consumer-ingress.yaml ./helm-basic-chart --debug | cat -n -

7.2 Refactor 2

Mon, 01 Jan 0001 00:00:00 +0000

If you take a closer look at your Chart you will still recognize some weak spots. For example the producer and consumer will look like a lot of code duplication. We don’t like code duplication at all! The only big difference is the "consumer" or "producer" pre- or suffixed everywhere.

Just for fun: How much lines of code are actually different?

Hint / Solution


bc -l <<< "$(diff -U0 templates/consumer-deployment.yaml templates/producer-deployment.yaml | wc -l)"/2

When considering the differences and how they affect the service, we can easily see the flaw of this Chart. The entire Chart is a duplication. Both services could use the same Chart and just be two instances / releases!

7.3 Composition

Mon, 01 Jan 0001 00:00:00 +0000

Instead of instantiating the Chart two times manually by hand, we can also define a composition of two Charts in a single Chart. In other 2words we create a new Helm Chart and add the Chart we just wrote twice as a dependency to the chart.

In Helm we can make a composition of already existing Charts and own templates and define it as a new Chart. This is a very common use case if we mix self created resources with own templated resources.

8.1 Tasks: Relabeling

Mon, 01 Jan 0001 00:00:00 +0000

Relabeling in Prometheus can be used to perform numerous tasks using regular expressions, such as

adding, modifying or removing labels to/from metrics or alerts,
filtering metrics based on labels, or
enabling horizontal scaling of Prometheus by using hashmod relabeling.

It is a very powerful part of the Prometheus configuration, but it can also get quite complex and confusing. Thus, we will only take a look at some basic/simple examples.

There are four types of relabelings:

8.2 Tasks: Recording Rules

Mon, 01 Jan 0001 00:00:00 +0000

Prometheus recording rules allow you to precompute queries at a defined interval (global.evaluation_interval or interval in rule_group) and save them to a new set of time series.

In this lab you are going to create your first own recording rules. Recording rules are very useful when it comes to queries, which are very complex and take a long time to compute. The naming convention dictates to use the following format when naming recording rules level:metric:operation. Additional information regarding naming best-practices can be found here.

8.3 Tasks: Troubleshoot Kubernetes Service Discovery

Mon, 01 Jan 0001 00:00:00 +0000

Task 8.3.1: Troubleshooting Kubernetes Service Discovery

We will now deploy an application with an error in the monitoring configration.

Deploy Loki in your namespace by adding the following files to your git repo in the user-demo/ folder:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: loki
  name: loki
spec:
  replicas: 1
  selector:
    matchLabels:
      app: loki
  template:
    metadata:
      labels:
        app: loki
    spec:
      containers:
      - image: mirror.gcr.io/grafana/loki:latest
        name: loki

---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: loki
  name: loki
spec:
  ports:
    - name: http
      port: 3100
      protocol: TCP
      targetPort: 3100
  selector:
    app: loki
  type: NodePort

---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    app.kubernetes.io/name: loki
  name: loki
spec:
  endpoints:
    - interval: 30s
      port: http
      scheme: http
      path: /metrics
  selector:
    matchLabels:
      prometheus-monitoring: 'true'

Add, commit and push changes to your git repository and let ArgoCD sync your app again.

Helm overview

Mon, 01 Jan 0001 00:00:00 +0000

Ok, let’s start with Helm. First, you have to understand the following 3 Helm concepts: Chart, Repository and Release.

A Chart is a Helm package. It contains all of the resource definitions necessary to run an application, tool, or service inside of a Kubernetes cluster. Think of it like the Kubernetes equivalent of a Homebrew formula, an Apt dpkg, or a Yum RPM file.

A Repository is the place where charts can be collected and shared. It’s like Perl’s CPAN archive or the Fedora Package Database, but for Kubernetes packages.

CLI installation

Mon, 01 Jan 0001 00:00:00 +0000

This guide shows you how to install the helm CLI tool. helm can be installed either from source or from pre-built binary releases. We are going to use the pre-built releases. helm binaries can be found on Helm’s release page for the usual variety of operating systems.

Warning

If you do this training in our acend web based environment, no installation is required.

Task 1.2.12.2.1: Install CLI

Install the CLI for your Operating System

Create a chart

Mon, 01 Jan 0001 00:00:00 +0000

In this lab we are going to create our very first Helm chart and deploy it.

Task 1.2.12.3.7: Create Chart

First, let’s create our chart. Open your favorite terminal and make sure you’re in the workspace for this lab, e.g. cd ~/<workspace-kubernetes-training>:

helm create mychart

You will now find a mychart directory with the newly created chart. It already is a valid and fully functional chart which deploys an nginx instance. Have a look at the generated files and their content. For an explanation of the files, visit the Helm Developer Documentation. In a later section you’ll find all the information about Helm templates.

Generic Chart setup

Mon, 01 Jan 0001 00:00:00 +0000

In the following labs we are going to create our first Helm Charts with the help of Baloise’s Generic Chart and deploy them.

Baloise’s Generic Helm Chart is meant as a template and easy starting point to deploy common Kubernetes resource manifests. By declaring the Generic Chart as a dependency of your own Chart, you can make use of all the features the Generic Chart offers.

Task 2: Setup the dependency

So first, let’s create your own Chart. Open your favorite terminal and make sure you’re in the workspace for this lab, e.g. cd ~/<workspace-kubernetes-training>:

A first example using the Generic Chart

Mon, 01 Jan 0001 00:00:00 +0000

You’re now all set to begin with a first example!

Task 4: Create a `values.yaml` file

Still inside your mychart Helm Chart directory, open the already existing values.yaml file. Inside you’ll find a host of defined parameters. Delete them all.

Instead, fill in the following content:

first-example-app:
  replicaCount: 1
  image:
    repository: REGISTRY-URL/example/nginx-sample
    tag: latest
    pullPolicy: IfNotPresent
  ingress:
    controller: Route
    clusterName: CLUSTER-NAME
  network:
    http:
      servicePort: 8080
      ingress:
        clusterName: CLUSTER-NAME
  readinessProbe:
    httpGet:
      path: /
      port: 8080
    initialDelaySeconds: 5
    timeoutSeconds: 1
  resources:
    requests:
      cpu: 10m
      memory: 16Mi
    limits:
      cpu: 200m
      memory: 32Mi

Task 5: A first test

Before applying anything to the cluster, you should test if the current values have the desired effect. In order to do so, execute the following command:

Complex example

Mon, 01 Jan 0001 00:00:00 +0000

In this extended lab, we are going to deploy an existing, more complex application with a Helm chart from the Artifact Hub.

Artifact Hub

Check out Artifact Hub where you’ll find a huge number of different Helm charts. For this lab, we’ll use the WordPress chart by Bitnami, a publishing platform for building blogs and websites.

WordPress

As this WordPress Helm chart is published in Bitnami’s Helm repository, we’re first going to add it to our local repo list:

Generic Chart usage

Mon, 01 Jan 0001 00:00:00 +0000

You have now seen how to set up and use the Generic Chart. Now it’s your turn!

Task 1: Setup

Repeat the steps from Generic Chart setup in order to create a new Chart.

Note

Note the alias: line inside Chart.yaml. You can change this value to whatever you’d like, but you need to use the same name as first line inside your values.yaml!

This is also how you can use the Generic Chart multiple times if you have more than one app/component.

Mon, 01 Jan 0001 00:00:00 +0000

Tag 01 - Einführung

Modulübersicht

Kursplan

Inhaltsverzeichnis

[TOC]

Tagesziel

Modulübersicht
Einführung in DevOps
- Was ist DevOps?
- DevOps Lifecycles
- Technologieübersicht
Projekt Kickoff

Theorie

Willkommen zum ersten Tag unseres DevOps-Kurses! Heute legen wir das Fundament für unser gemeinsames Verständnis von DevOps und starten mit unserem Praxisprojekt - der Modernisierung des Online-Shops von TechStyle.

DevOps ist mehr als nur ein technisches Konzept - es ist eine Kultur, eine Denkweise und eine Sammlung von Praktiken, die darauf abzielen, die Zusammenarbeit zwischen Entwicklung (Development) und Betrieb (Operations) zu verbessern. In den kommenden Wochen werden wir diese Prinzipien nicht nur theoretisch erlernen, sondern sie praktisch in unserem durchgängigen Projekt anwenden.

Mon, 01 Jan 0001 00:00:00 +0000

Tag X - Beschreibung Tag

[TOC]

Modulübersicht

Kursplan

Tagesziel

Ziel 1
- Ziel 1.1
- Ziel 1.2
Ziel 2

Theorie

Theorie Teil 1

Grundlegende Konzepte
Wichtige Definitionen
Historischer Kontext

Theorie Teil 2

Erweiterte Methoden
Best Practices
Aktuelle Entwicklungen

Praxis

Praxis - Auftrag 1

Installieren Sie die benötigte Software
Konfigurieren Sie die Umgebung
Führen Sie den ersten Test durch

Praxis - Auftrag 2

Erstellen Sie ein neues Projekt
Implementieren Sie die besprochenen Funktionen
Dokumentieren Sie Ihre Ergebnisse

Mon, 01 Jan 0001 00:00:00 +0000

Kubernetes

Umgebung zum Modul: CNC - Cloud-native Core.

Mit Control-Plane verbinden und microk8s add-node ausführen. Die Ausgabe ist dann jeweils, mittels Voranstellung von sudo, auf dem Worker auszuführen.

ssh ubuntu@${ip}

microk8s add-node --token-ttl 3600
exit

ssh ubuntu@${ip_01}
sudo <Ausgabe von oben>
exit

Die obigen Befehle sind für jeden Worker zu wiederholen.

Dashboard

Das Kubernetes Dashboard ist wie folgt erreichbar.

https://${fqdn}:18443

Mon, 01 Jan 0001 00:00:00 +0000

Unterrichtsressourcen

[[TOC]]

Überblick über die Unterrichtsressourcen

Ressourcen in Unterordnern ablegen und in dieser Übersichtsseite sinnvolle Hinweise und Verlinkungen ergänzen

Infrastruktur erstellen

git clone https://gitlab.com/ch-tbz-wb/Stud/cnc
cd cnc/2_Unterrichtsressourcen

terraform init

terraform plan

terraform apply -parallelism=1 -auto-approve

Hinweis: Terraform verwendet 10 Threads das ist zuviel für das MAAS.

Für mehrere Studenten

export TF_VAR_host_no=10
for stud in 01 02 03 04 05 06 07 08 09 10 11 12
do
    terraform workspace new ws${stud}
    terraform workspace select ws${stud}
    terraform init
    terraform apply -parallelism=1 -auto-approve
    TF_VAR_host_no=$((TF_VAR_host_no + 6))
done

Hinweis: falls MAAS sich mit Fehler meldet, einloggen in den entsprechende KVM-Host und VMs mittels virsh list --all und virsh undefine xxx manuell weglöschen.

Übersicht on IT-CNE Cloud-Native Core

1.1 Tasks: Setup

Task 1.1.1: Getting Started

Task 1.1.2: Install Prometheus

Mit ArgoCD

Deploy Prometheus

Configure Prometheus

2.1 Tasks: Thanos Querier

Task 2.1.1: Install Thanos Querier

2.2 Tasks: Explore metrics

Task 2.2.1: Prometheus/Thanos web UI

2.3 Tasks: PromQL

3.1 Tasks: Grafana intro

Task 3.1.1: Install Grafana

3.2 Tasks: Grafana dashboards

Task 3.2.1: Import a dashboard

4.1 Tasks: Blackbox exporter

Task 4.1.1: Install Blackbox exporter

4.2 Tasks: Pushgateway

Task 4.2.1 - Install and configure Pushgateway

6.1 Tasks: Instrumenting

Task 6.1.1: Get the source code

6.1 Tasks: Instrumenting

Task 6.1.1: Spring Boot Example Instrumentation

7.1 Tasks: Enable and configure Alertmanager

Task 7.1.1: Install Alertmanager and Thanosruler

7.2 Tasks: Alertmanager

Task 7.2.1: Configure Slack as alert receiver

7.3 Tasks: Alertrules and alerts

Web terminal

Task 1: Login on the web console

Task 2: Initialize terminal

4.3 Tasks: Exporter as a sidecar

Task 4.3.1: Deploy a database and use a sidecar container to expose metric

5.1 Tasks: kube-prometheus metrics

Task 5.1.1: Memory usage of Prometheus

Task 5.1.2: Kubernetes pod count

5.2 Tasks: Prometheus Operator

5.2.1: Prometheus storage

Windows

Installation for Windows

YAML

macOS

Installation for macOS

File mode

PATH variable

Linux

Installation for Linux

File mode

PATH variable

Console login

Task 4: Login on the web console

Task 5: Login on the command line

Verification

Verify the installation

Optional Kubernetes power tools

Optional Kubernetes power tools

Other ways to work with OpenShift

Other ways to work with OpenShift

Next steps

6.1 Deploy your awesome application

Task 6.1.1: Deploy the chart

Solution

6.2 Template the database backend

Resource names

6.3 Backend as a Dependency

Helm Dependencies

6.4 Test

Task 6.4.1: Add a test to your chart

6.5 Helm hooks

Task 6.5.1: Chart Hooks

6.6 Deploy your Chart with different environment configurations

Task 6.6.1 Create a devlopment release

6.7 Optional: Publish your chart

7.1 Refactor 1

Task 7.1.1: Make the Ingress resource more configurable

7.2 Refactor 2

7.3 Composition

8.1 Tasks: Relabeling

8.2 Tasks: Recording Rules

Task 4: Create a `values.yaml` file