Use Labels and Node Selectors with Oracle Cloud Native Environment

Introduction

The ability to influence how Kubernetes schedules Pods to provide the best performance, reduce running costs, and more easily simplify cluster management is a vital skill for an administrator to master. Many Kubernetes clusters have nodes with varying capabilities, for example:

GPU card (for machine learning applications)
SSD disk (for applications requiring fast data access)
High-end CPU (for CPU-intensive tasks)

Administrators use several ways to influence how the Kubernetes scheduler deploys applications to specific nodes. This tutorial covers using labels and node selectors, which are the simplest way to assign Pods to nodes.

Objectives

In this tutorial, you will learn:

How to review and set labels on nodes
How to define and use a node selector to influence an application deployment

Prerequisites

Installation of Oracle Cloud Native Environment
- a single control node and two worker nodes

Deploy Oracle Cloud Native Environment

Note: If running in your own tenancy, read the linux-virt-labs GitHub project README.md and complete the prerequisites before deploying the lab environment.

Open a terminal on the Luna Desktop.

Clone the linux-virt-labs GitHub project.

git clone https://github.com/oracle-devrel/linux-virt-labs.git

Change into the working directory.
```
cd linux-virt-labs/ocne2
```

Install the required collections.

ansible-galaxy collection install -r requirements.yml

Deploy the lab environment.
```
ansible-playbook create_instance.yml -e localhost_python_interpreter="/usr/bin/python3.6" -e install_ocne_rpm=true -e create_ocne_cluster=true -e "ocne_cluster_node_options='-n 1 -w 2'"
```
The free lab environment requires the extra variable local_python_interpreter, which sets ansible_python_interpreter for plays running on localhost. This variable is needed because the environment installs the RPM package for the Oracle Cloud Infrastructure SDK for Python, located under the python3.6 modules.
The default deployment shape uses the AMD CPU and Oracle Linux 8. To use an Intel CPU or Oracle Linux 9, add -e instance_shape="VM.Standard3.Flex" or -e os_version="9" to the deployment command.
Important: Wait for the playbook to run successfully and reach the pause task. At this stage of the playbook, the installation of Oracle Cloud Native Environment is complete, and the instances are ready. Take note of the previous play, which prints the public and private IP addresses of the nodes it deploys and any other deployment information needed while running the lab.

Confirm the Number of Nodes

It helps to know the number and names of nodes in your Kubernetes cluster.

Open a terminal and connect via SSH to the ocne instance.
```
ssh oracle@<ip_address_of_node>
```
List the nodes in the cluster.
```
kubectl get nodes
```
The output confirms both worker nodes are in a Ready state.

What are Labels?

Kubernetes labels are defined using key-value pairs that provide a way to organize Kubernetes objects such as Pods, Nodes, Deployments, etc. The values assigned to the key-value pairs are completely free-form and do not have any predefined meaning. However, they must comply with the following restrictions for key-label values:

Must be 63 characters or less (they can even be empty)
Unless empty, begin and end with an alphanumeric character ([a-z0-9A-Z])
Could contain dashes (-), underscores (_), dots (.), and alphanumerics between

Review Existing Node Labels

The flexibility allowed in naming the key-pair values provides complete freedom of choice when choosing how to define them, for example, assigning a label identifying a Pod or node as production or even assigning them to specific applications. So, before creating and assigning new labels, let's review any existing labels on your cluster.

Show the existing labels on each node in a cluster.
```
kubectl get nodes --show-labels
```
The most commonly pre-populated labels present by default on nodes in a Kubernetes cluster are:

Show the existing labels for a specific node.

kubectl label --list nodes ocne-worker-1

Example Output:

beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ocne-worker-1
kubernetes.io/os=linux
beta.kubernetes.io/arch=amd64

You can list the labels for several nodes simultaneously by appending each node name to this command, separated by a space.

Apply a New Label to a Node

Apply a new label to a worker node.
```
kubectl label node ocne-worker-1 disktype=ssd
```
The values used for the key-value pair are free-form and used by the administrator to represent one of the attributes available on the node. Once defined, the Kubernetes scheduler uses the labels to influence how Pods deploy to specific nodes.

Confirm the application of the label.

kubectl get nodes --show-labels | grep disktype

Example Output:

ocne-worker-1    Ready    <none>          13m   v1.28.3+3.el8   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,disktype=ssd,kubernetes.io/arch=amd64,kubernetes.io/hostname=ocne-worker-1,kubernetes.io/os=linux

Apply labels simultaneously to several nodes.

kubectl label nodes ocne-worker-1 ocne-worker-2 environment=production

Check both nodes for the label.

kubectl label --list nodes ocne-worker-1 ocne-worker-2

Example Output:

Listing labels for Node./ocne-worker-1:
 disktype=ssd
 beta.kubernetes.io/os=linux
 environment=production
 kubernetes.io/arch=amd64
 kubernetes.io/hostname=ocne-worker-1
 kubernetes.io/os=linux
 beta.kubernetes.io/arch=amd64
Listing labels for Node./ocne-worker-2:
 beta.kubernetes.io/os=linux
 environment=production
 kubernetes.io/arch=amd64
 kubernetes.io/hostname=ocne-worker-2
 kubernetes.io/os=linux
 beta.kubernetes.io/arch=amd64

How to Overwrite a Node Label

What happens if a label needs to be re-labeled or the wrong value is assigned to a key? Well, the label's value can be changed.

Change the previously applied key-value pair (originally environment=production) on the ocne-worker-2 node.
```
kubectl label --overwrite nodes ocne-worker-2 environment=staging
```

Confirm the label is updated.

kubectl label --list nodes ocne-worker-2

Example Output:

beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
environment=staging
kubernetes.io/arch=amd64
kubernetes.io/hostname=ocne-worker-2
kubernetes.io/os=linux

Notice that the label changed to environment=staging.

How to Remove Labels from a Node

What happens if a label needs to be removed instead of altered? You can delete labels from a node or nodes as required.

Remove a label from the node by assigning the key without any value.
```
kubectl label nodes ocne-worker-2 environment-
```
Confirm that you removed the label.
```
kubectl label --list nodes ocne-worker-2
```
Example Output:
```
kubernetes.io/arch=amd64
kubernetes.io/hostname=ocne-worker-2
kubernetes.io/os=linux
beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
```
You now have covered the basics of managing Kubernetes labels. Next, learn how an administrator uses them to help target deployments to specific nodes using a node selector.

What is a Node Selector?

A node selector references a previously defined label in the deployment YAML file and instructs the Kubernetes scheduler to deploy only to nodes with a matching label in the Kubernetes cluster. In other words, Kubernetes will only schedule a deployment onto nodes with matching labels.

You may know about the nodeName field and wonder if it is the same as the nodeSelector field. It is not; nodeName differs from nodeSelector because nodeSelector uses the Kubernetes scheduler to deploy an application to a node with a matching label. Remember, you can apply a label to several nodes. On the other hand, nodeName is a manual scheduler that uses the defined node name and deploys it to the only node in the cluster with that name.

Define and Apply a Node Selector

Create a deployment file and include a node selector matching one of the previously defined labels in the deployment manifest file.

cat << EOF | tee -a nginx-pod.yml > /dev/null

apiVersion: v1
kind: Pod
metadata:
  name: nginx-test
  labels:
    env: default
spec:
  containers:
  - image: ghcr.io/oracle/oraclelinux9-nginx:1.20
    name: nginx-test
  nodeSelector:
    disktype: ssd
EOF

Deploy the application.
```
kubectl apply -f nginx-pod.yml
```

Check which node Nginx deployed to.

kubectl get pods -n default -o wide

Example Output:

NAME         READY   STATUS    RESTARTS   AGE     IP           NODE             NOMINATED NODE   READINESS GATES
nginx-test   1/1     Running   0          2m46s   10.244.1.3   ocne-worker-1   <none>           <none>

Notice that Kubernetes scheduled the Nginx deployment to the ocne-worker-1 node with the label disktype:ssd as expected.

Next Steps

Understanding how to control application deployments is a key skill for administrators to acquire. All nodes in a cluster can have different labels to provide a way to influence how and where deployments get scheduled by combining labels and node selectors.

How does this differ from just using a node name instead? Using a node selector provides slightly more flexibility than just the node name because a label can repeat across many nodes across the cluster. Instead, the node name within a cluster is unique to a specific node, avoiding a potential issue if the specified node fails, goes offline, or runs out of resources and prevents creating a single point of failure. Using a node selector has developed further into node affinity to provide a more powerful way to control how your applications deploy with more options on your Kubernetes cluster.

That concludes the walkthrough by introducing labels and node selectors and showing how they can help manage your application deployments.