vCluster

The following demonstrates the usage of vcluster to provide a preview environment for each Merge Request of a GitLab project.

Prerequisites

We need a Kubernetes cluster, which can be created following these instructions. We also need the kubectl binary configured with the cluster’s kubeconfig, and the helm binary.

About our Sample application

We consider this dumb simple application. A live version is available at https://shapes.techwhale.io/.

Connecting GitLab to our Kubernetes cluster

We defined a configuration for a GitLab agent named techwhale creating the configuration file at the root of the project’s source code. This defines the permissions and access levels for the agent.

ci_access:
  projects:
    - id: shape-it/www

In this example, the ci_access section specifies that the techwhale agent has access to the GitLab project identified by shape-it/www. This means the agent can perform actions related to this project when triggered by CI/CD jobs.

Next we start the connection to the cluster step from the “Operate / Kubernetes Cluster” menu

We select the agent named techwhale.

The helm command to install the GitLab agent is returned.

We run this command in the cluster. After a few seconds, the GitLab connects to the cluster.

When a CI job is triggered, the GitLab runner will use the techwhale agent to securely communicate with the connected cluster.

DNS configuration

We use Exoscale DNS service to manage the exoscale.dev domain name used in this example. Each Merge Request will have its own dedicated MR_IDENTIFIER.mrs.exoscale.dev subdomain. The DNS resolution is done using the wildcard domain *.mrs.exoscale.dev which resolves to the IP Address of the Traefik Load Balancer Service.

First we get the IP address of this Load Balancer:

$ kubectl get svc -n traefik
NAME      TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                      AGE
traefik   LoadBalancer   10.105.239.127   91.92.153.116   80:30539/TCP,443:31423/TCP   2h

Next we add the A record in the DNS provider

Definition of the CI jobs

The helm repository contains the Helm packaging of our sample application. .gitlab-ci.yaml specifies the actions performed each time a commit is made to the main branch:

graph TD;
    A[Commit to branch main]-->B[New Semantic Version created];
    B-->C[New tag pushed to Git repository];
    C-->D[Update version in Chart.yaml];
    D-->E[Build and push new Chart];

The www repository contains the source code of the application, which is a simple Flask app that displays a colored shape. The .gitlab-ci.yaml file specifies the actions performed each time a commit is made on the main branch or when a Merge Request is used.

• Actions performed when a commit is made to the main branch

graph TD;
    A[Commit to branch main]-->B[New Semantic Version created];
    B-->C[New image created and pushed to the registry];
    C-->D[Update image tag in values.yaml in helm repo];
    C-->E[Update appVersion Chart.yaml in helm repo];

• Actions performed when a Merge Request is either created of updated

graph TD;
    A[Create / Update a Merge Request]-->B[New image created with this tag and push to the registry];
    B-->C[vcluster created / updated];
    C-->D[New image deployed in the vcluster];
    D-->E[Comment added to the MR when application is ready];

• Actions performed when a MR is merged to the main branch

graph TD;
    A[Merge Request merged to main branch]-->B[Deletion of the vcluster];
    A-->cvC[Removal of temporary images];

Testing the entire flow

In this section we put everything into action to illustrate the usage of vcluster within GitLab CI. The objective is to create a preview environment for each Merge Request and deploy the MR’s code in it.

Installing the infrastructure components

We install Traefik and Cert-Manager in the host cluster. These components will be shared with the virtual cluster.

• Traefik

helm repo add traefik https://traefik.github.io/charts
helm install traefik traefik/traefik --version 33.0.0 -n traefik --create-namespace

• Cert manager

Installation of cert-manager:

helm repo add cert-manager https://charts.jetstack.io
helm install cert-manager cert-manager/cert-manager --set crds.enabled=true --version 1.16.1 -n cert-manager --create-namespace

Definition of a ClusterIssuer in charge of issuing the Certificates against Let’s Encrypt CA

cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    email: devops@techwhale.io
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: acme-account-key
    solvers:
    - http01:
       ingress:
         class: traefik
EOF

Deploying the application

First we deploy the application using Argo CD:

helm upgrade --install shapes oci://registry-1.docker.io/lucj/shapeit \
  --version v1.0.7 \
  --set ingress.enabled=true \
  --set ingress.annotations.cert-manager\\.io\\/cluster-issuer=letsencrypt \
  --set ingress.className=traefik \
  --set ingress.tls.host=shapes.exoscale.dev \
  --set ingress.tls.secretName=certs \
  --set ingress.hosts[0].host=shapes.exoscale.dev \
  --set ingress.hosts[0].paths[0].path=/ \
  --set ingress.hosts[0].paths[0].pathType=ImplementationSpecific \
  -n shapes --create-namespace

Then we make sure the application’s Pod is running in the dedicated namespace:

kubectl get po -n shapes
NAME                              READY   STATUS    RESTARTS   AGE
shapes-shapeit-6495574dd8-6mwbh   1/1     Running   0          1m

The application is exposed on https://shapes.exoscale.dev

Creating a Merge Request

• creating a new branch

git branch change_default_shape && git checkout change_default_shape

• changing the code

import os
import socket
from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')
def index():
    # Set default shape / color if not provided
    shape_type = os.environ.get('SHAPE_TYPE', 'square')      # Change circle to square
    shape_color = os.environ.get('SHAPE_COLOR', '#3498db')

    # Get container details
    hostname = socket.gethostname()
    ip_address = socket.gethostbyname(hostname)
...

• pushing the changes

git add code/app.py
git commit -m 'use square by default'
git push origin change_default_shape

• creating the Merge Request

From the GitLab web UI, we create the Merge Request

It automatically triggers the CI

When the application is ready, the CI adds the URL to access the environment

Using this URL we can visualize the changes done in the current Merge Request (the circle turned into a square)

Under the scenes, we can verify a new vcluster was created

$ vcluster list

        NAME     | NAMESPACE | STATUS  | VERSION | CONNECTED |  AGE
  ---------------+-----------+---------+---------+-----------+--------
    vcluster-mr4 | mr4       | Running | 0.20.0  |           | 4m19s

We can also see a new image is present in the Docker Hub lucj/shapes repository

Updating the Merge Request

• changing the code

import os
import socket
from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')
def index():
    # Set default shape / color if not provided
    shape_type = os.environ.get('SHAPE_TYPE', 'triangle')      # Change shape
    shape_color = os.environ.get('SHAPE_COLOR', '#FF7043')     # Change color

    # Get container details
    hostname = socket.gethostname()
    ip_address = socket.gethostbyname(hostname)
...

git add code/app.py
git commit -m 'use coral triangle by default'
git push origin change_default_shape

• viewing update of the Merge Request

The Merge Request triggers the CI

A new comment is added when the application is ready

The URL allows to access the application and visualize the changes

Checking the new image in the Docker Hub lucj/shapes repository

Merging the MR

The final step is to merge this Merge Request to the main branch.

Under the scenes we can see the vcluster was deleted

$ vcluster list

    NAME | NAMESPACE | STATUS | VERSION | CONNECTED | AGE
  -------+-----------+--------+---------+-----------+------

Also, the temporary images were removed from DockerHub.