Use ConfigMaps and Secrets with Oracle Cloud Native Environment
Introduction
A key advantage of containers is knowing your software executes on any computer without problems. Many applications use environment variables to control the application's runtime behaviour, such as an API Key or environment variables. They may be hard-coded into a container, but doing so makes switching environments between a Production and Development environment difficult. Kubernetes addresses this by moving the application's configuration into an object called a ConfigMap comprised of a series of key-value pairs injected into the application's environment at runtime. The application then consumes these transparently as though they were environment values in their runtime environment.
This tutorial shows how to create and use ConfigMaps and Secrets with Oracle Cloud Native Environment. Both ConfigMaps and Secrets are used to define configuration data and have many similarities. You will start by covering how to use ConfigMaps, and then cover the use of Secrets.
Objectives
In this lab, you'll learn how to:
- Use ConfigMaps and Secrets with Oracle Cloud Native Environment.
Prerequisites
Minimum of a 3-node Oracle Cloud Native Environment cluster:
- Operator node
- Kubernetes control plane node
- Kubernetes worker node
Each system should have Oracle Linux installed and configured with:
- An Oracle user account (used during the installation) with sudo access
- Key-based SSH, also known as password-less SSH, between the hosts
- Installation of Oracle Cloud Native Environment
Deploy Oracle Cloud Native Environment
Note: If running in your own tenancy, read the linux-virt-labs
GitHub project README.md and complete the prerequisites before deploying the lab environment.
Open a terminal on the Luna Desktop.
Clone the
linux-virt-labs
GitHub project.git clone https://github.com/oracle-devrel/linux-virt-labs.git
Change into the working directory.
cd linux-virt-labs/ocne
Install the required collections.
ansible-galaxy collection install -r requirements.yaml
Deploy the lab environment.
ansible-playbook create_instance.yaml -e ansible_python_interpreter="/usr/bin/python3.6"
The free lab environment requires the extra variable
ansible_python_interpreter
because it installs the RPM package for the Oracle Cloud Infrastructure SDK for Python. The location for this package's installation is under the python3.6 modules.Important: Wait for the playbook to run successfully and reach the pause task. The Oracle Cloud Native Environment installation is complete at this stage of the playbook, and the instances are ready. Take note of the previous play, which prints the public and private IP addresses of the nodes it deploys.
Looking at ConfigMap's
A ConfigMap is used to store non-confidential data in key-value pairs. The resulting ConfigMap(s) are subsequently available for any deployed Pods to utilize as environment variables, command-line arguments or as configuration files in a volume. For this reason, an administrator needs to know what they are and how to use them.
Important Notice: Using a ConfigMap does not provide any secrecy or encryption. Do not use a
ConfigMap
for any confidential data. Use aSecret
, or another external solution to keep the data private.Another restriction to their use is that the ConfigMap file is limited to a maximum size of 1MB. Larger datasets should use alternate storage methods, such as databases.
Create a ConfigMap
Whilst most ConfigMaps are created using a YAML file with the kubectl create -f
command, it is also possible to create a ConfigMap directly from kubectl
itself. Only use direct entry when defining a small number of key-pairs.
Open a terminal and connect via SSH to ocne-control-01 node.
ssh oracle@<ip_address_of_ol_node>
Use direct entry to define a key-pair.
kubectl create configmap mytest --from-literal=Country=USA
Where: The
key-value
pair stored in the ConfigMap is: 'Country=USA'Example Output:
[oracle@ocne-control-01 ~]$ kubectl create configmap mytest --from-literal=Country=USA configmap/mytest created
Confirm creation of the ConfigMap.
kubectl get configmap mytest
Example Output:
[oracle@ocne-control-01 ~]$ kubectl get configmap mytest NAME DATA AGE mytest 1 22s
Describe the ConfigMap.
kubectl describe configmap mytest
Example Output:
[oracle@ocne-control-01 ~]$ kubectl describe configmap mytest Name: mytest Namespace: default Labels: <none> Annotations: <none> Data ==== Country: ---- USA BinaryData ==== Events: <none>
Enter multiple key-pairs.
Most ConfigMaps consist of several entries. This is how to enter multiple key-pairs.
kubectl create configmap mytest01 --from-literal=Country=USA --from-literal=State=ME --from-literal=City=Portland
Example Output:
[oracle@ocne-control-01 ~]$ kubectl create configmap mytest01 --from-literal=Country=USA --from-literal=State=ME --from-literal=City=Portland configmap/mytest01 created
Describe the ConfigMap.
kubectl describe configmap mytest01
Example Output:
[oracle@ocne-control-01 ~]$ kubectl describe configmap mytest01 Name: mytest01 Namespace: default Labels: <none> Annotations: <none> Data ==== City: ---- Portland Country: ---- USA State: ---- ME BinaryData ==== Events: <none>
Inspect the definition in YAML format.
kubectl get configmap mytest01 -o yaml
Example Output:
[oracle@ocne-control-01 ~]$ kubectl get configmap mytest01 -o yaml apiVersion: v1 data: City: Portland Country: US State: ME kind: ConfigMap metadata: creationTimestamp: "2023-09-20T15:17:24Z" name: mytest02 namespace: default resourceVersion: "23860" uid: b425b856-9b22-4cf4-9daa-9c1f4aea39e7
Check what ConfigMaps are currently defined system-wide.
kubectl get configmap
Example Output:
[oracle@ocne-control-01 ~]$ kubectl get configmap NAME DATA AGE kube-root-ca.crt 1 5h20m mytest 1 132m mytest01 1 52m
Did you notice the ConfigMap - kube-root-ca.crt listed? This is created as part of the installation process and is used internally by Oracle Cloud Native Environment. It should not be altered or deleted.
(Optional) Inspect the contents of the kube-root-ca.crt ConfigMap.
kubectl get configmap kube-root-ca.crt -o yaml
Example Output:
[oracle@ocne-control-01 ~]$ kubectl get configmap kube-root-ca.crt -o yaml apiVersion: v1 data: ca.crt: | -----BEGIN CERTIFICATE----- MIIC/jCCAeagAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl cm5ldGVzMB4XDTIzMDkyMDEwMzk0N1oXDTMzMDkxNzEwMzk0N1owFTETMBEGA1UE AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKWO PCSCOAjr74K/H8wO+TgGxP6mbiYOvYkkmXpKmd25paO+sOgEsFDpzd2IJrF62wdF XOMPmFuKqBabxLD9BFxnu5NQGCXmQzSKjOxPTy+tIYMIoCSiqy5uNTtOV/2HOOhP a0e1vcYVuCTMlJXzvCh8w90LTDzHH6vk5IanhwbNgVb3L7IyOKeE02foqp7Yfo2t 8KQElJNBjtZediHXWArHTI5m7fuLZy+QzFFDn/lCiwHB64gwNwvvaPJ158q2K49K cw3v222zzo6/RQv3nFCoq45LjdyrjHCbHHic5r2ZZH0oSnA7foKq2j+SGJ9w3KjC 1oiGGzx3BSM3JWl38GUCAwEAAaNZMFcwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB /wQFMAMBAf8wHQYDVR0OBBYEFM+0ErJ+VJfYmbN5usVZwIK9kPScMBUGA1UdEQQO MAyCCmt1YmVybmV0ZXMwDQYJKoZIhvcNAQELBQADggEBACAJ5RnQk0Qm771JXSrP U1ksIXO1K5BvZKUOCRuQowgyMWLF2ndFccvyDYwEjRflHC6spDOfgEcWgOtS2aYS QmkMr9W07yfVgZXUxmtSOPmpKLXLgf+auCGaR5LQ70gGEqocqY3KDK0EKdZuj47y WLahJ6PqVtzy6gkwNRz9/EWFbO12zMiHsyzO9wksZO5q3bm27pdOLe0tQpg4RNL4 dx2XVTg2ruvqLypdkQWAis+VHJF72JA8GVjfWGboCd6nYSfdKEzhNlvTqRi/j/uF 7kywrKLX3O0iQYzrwnbFZYY+FaXLTP1Q7jSnNEaqjxrCin2x9zXiVpifjZG5cza2 /yE= -----END CERTIFICATE----- kind: ConfigMap metadata: annotations: kubernetes.io/description: Contains a CA bundle that can be used to verify the kube-apiserver when using internal endpoints such as the internal service IP or kubernetes.default.svc. No other usage is guaranteed across distributions of Kubernetes clusters. creationTimestamp: "2023-09-20T10:40:23Z" name: kube-root-ca.crt namespace: default resourceVersion: "355" uid: 4576026a-0632-4e43-99b4-be199863c84f
Create a ConfigMap from a YAML File
Rather than creating a ConfigMap declaratively from the command line, it is often both easier and faster to define the key-value pairs within a YAML manifest file in advance.
Create the ConfigMap YAML file manifest.
cat << 'EOF' > ~/data-config.yaml apiVersion: v1 kind: ConfigMap metadata: name: data-config data: database_url: jdbc:oracle:thin:@localhost:1521:<SID> user: oracle EOF
Create the ConfigMap.
kubectl apply -f data-config.yaml
Using a ConfigMap as Environment Variables
A ConfigMap, once declared, can be used by one or more Pods deployed into the same namespace. The key-value pairs of a ConfigMap can be used as an environment variable within the Pod by using the envFrom.configMapRef
variable to inject the key-value pairs as environment variables. Next you will deploy a Pod and inject the key-pair values from the ConfigMap just created into it's environment.
Create the Pod definition.
cat << 'EOF' > ~/web-pod.yaml apiVersion: v1 kind: Pod metadata: name: web-pod spec: containers: - image: nginx:1.25.0 name: app envFrom: - configMapRef: name: data-config EOF
Hint: Look at the bottom of the file to see the
envFrom.configMapRef
variable.Create the deployment.
kubectl apply -f web-pod.yaml
Inspect the Pod's environment to confirm that environment variables are present.
You can do this by executing a remote command to retrieve the environment defined with
kubectl
like this.Note: It may be necessary to refresh this a few times before it works.
kubectl exec web-pod -- env
Example Output:
kubectl exec web-pod -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=xterm HOSTNAME=web-pod database_url=jdbc:oracle:thin:@localhost:1521:<SID> user=oracle KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443 KUBERNETES_PORT_443_TCP_PROTO=tcp KUBERNETES_PORT_443_TCP_PORT=443 KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1 KUBERNETES_SERVICE_HOST=10.96.0.1 KUBERNETES_SERVICE_PORT=443 KUBERNETES_SERVICE_PORT_HTTPS=443 KUBERNETES_PORT=tcp://10.96.0.1:443 NGINX_VERSION=1.25.0 NJS_VERSION=0.4.1 PKG_RELEASE=1~buster HOME=/root
Note: You should be able to see the ConfigMap variables have been successfully injected into the Pod's environment variables, as confirmed by these values:
... database_url=jdbc:oracle:thin:@localhost:1521:<SID> user=oracle ...
However, this does not match the naming convention used by the Pod's other environment variables, which is to capitalize variables. You could modify the original ConfigMap YAML definition file - which might impact other running services. Or you could redefine the keys used to inject the ConfigMap key-value pairs into the Pod by using the
valueFrom
attribute in the Pod's definition YAML file. The next step demonstrates how to do this.Delete the original Pod deployment.
kubectl delete -f web-pod.yaml
Create the Pod definition.
cat << 'EOF' > ~/web-pod01.yaml apiVersion: v1 kind: Pod metadata: name: web-pod spec: containers: - image: nginx:1.25.0 name: app env: - name: DATABASE_URL valueFrom: configMapKeyRef: name: data-config key: database_url - name: USERNAME valueFrom: configMapKeyRef: name: data-config key: user EOF
Create the deployment.
kubectl apply -f web-pod01.yaml
Confirm the environment variables now match the typical conventions used for environment variables in this Pod's environment.
kubectl exec web-pod -- env
Example Output:
kubectl exec web-pod -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=xterm HOSTNAME=web-pod DATABASE_URL=jdbc:oracle:thin:@localhost:1521:<SID> USER=oracle KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443 KUBERNETES_PORT_443_TCP_PROTO=tcp KUBERNETES_PORT_443_TCP_PORT=443 KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1 KUBERNETES_SERVICE_HOST=10.96.0.1 KUBERNETES_SERVICE_PORT=443 KUBERNETES_SERVICE_PORT_HTTPS=443 KUBERNETES_PORT=tcp://10.96.0.1:443 NGINX_VERSION=1.25.0 NJS_VERSION=0.4.1 PKG_RELEASE=1~buster HOME=/root
Note: The environment variables are capitalized as expected, as confirmed by these values:
... DATABASE_URL=jdbc:oracle:thin:@localhost:1521:<SID> USER=oracle ...
Mounting a ConfigMap as a Volume
A ConfigMap mounted as a Volume is useful when processing larger datasets of key-value pairs. This method allows the application to read the key-value pairs from the filesystem.
Delete the original Pod deployment.
kubectl delete -f web-pod01.yaml
Create a new Pod definition.
cat << 'EOF' > ~/nginx-pod.yaml apiVersion: v1 kind: Pod metadata: name: nginx-pod spec: containers: - image: nginx:1.25.0 name: app volumeMounts: - name: foo mountPath: /etc/config readOnly: true volumes: - name: foo configMap: name: data-config EOF
Create the deployment.
kubectl apply -f nginx-pod.yaml
Open an interactive shell to confirm the environment variables now match the typical conventions used for environment variables.
kubectl exec -it nginx-pod -- /bin/sh
Confirm the mount path (
/etc/config
) defined in the Pod deployment descriptor are listed as files. First of all, check the expected ConfigMap keys are showing.ls -1 /etc/config
Example Output:
# ls -1 /etc/config database_url user
Confirm each file's contents correspond to the value defined in the ConfigMap.
Check the
database_url
key contains the expected value.cat /etc/config/database_url
Example Output:
# cat /etc/config/database_url jdbc:oracle:thin:@localhost:1521:<SID>#
Check the
user
key contains the expected valuefiles are showing.cat /etc/config/user
Example Output:
# cat /etc/config/user oracle#
Exit the Pod's interactive shell.
exit
That confirms the ConfigMap is mounted as a Volume.
(Optional) Editing a ConfigMap
Note: This section assumes you know how to use
vi
. If you don't know how to use thevi
editor, please feel free to skip the example in this section.
One of the drawbacks of using environment variables, or command-line parameters with Pods is the inability to update them while the application is running. However, using a ConfigMap that has been mounted as Volume provides the ability to update the configuration values without having to recreate, or restart, the Pod. So when you update the underlying Configmap, all mounted Volumes referencing the ConfigMap are also updated.
Important Note It can take from a few seconds to several minutes after updating a ConfigMap mounted as a Volume for the updated values to be accessible within the running application.
Edit the mounted ConfigMap.
kubectl edit configmap data-config
Example Output:
# Please edit the object below. Lines beginning with a '#' will be ignored, # and an empty file will abort the edit. If an error occurs while saving this file will be # reopened with the relevant failures. # apiVersion: v1 data: database_url: jdbc:oracle:thin:@localhost:1521:<SID> user: oracle kind: ConfigMap metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","data":{"database_url":"jdbc:oracle:thin:@localhost:1521:\u003cSID\u003e","user":"oracle"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"backend-config","namespace":"default"}} creationTimestamp: "2023-09-21T12:52:07Z" name: backend-config namespace: default resourceVersion: "40737" uid: 7b01dab8-f985-4d05-8e87-51babb412614 ~ ~ ~ ~ "/tmp/kubectl-edit-2548943921.yaml" 18L, 762C
Note: Notice that the command opened the default editor for the system, which in the lab environment is
vi
.Update the
user: oracle
key-value pair.Follow the steps below:
- Enter edit mode (press the
i
key), and navigate to the second part of the `user: oracle key-value pair. - Change the value
oracle
toelcaro
- Save and exit the
vi
editor (Enter:wq!
, then pressEnter
)
Example Output
[oracle@ocne-control-01 ~]$ kubectl edit configmap backend-config configmap/backend-config edited
- Enter edit mode (press the
Open an interactive shell to confirm the environment variables now match the typical conventions used for environment variables.
kubectl exec -it nginx-pod -- /bin/sh
Confirm that the
user
key-value pair just edited shows the new value entered into the ConfigMap.Note: You may need to wait upto 3-5 minutes before the new value shows.
cat /etc/config/user
Example Output:
# cat /etc/config/user elcaro#
Exit the Pod's interactive shell.
exit
That confirms the ConfigMap's
user
key-value pair has been updated and is accessible to the Pod using it.
Looking at Kubernetes Secrets
Kubernetes Secrets, like ConfigMaps, are designed to store information in a key-value pair using the same sources.
However, Secrets are intended to store confidential information, albeit using a basic level of security because they are Base64 encoded. Making them easily decoded by a malicious attacker. Explaining how to improve the encryption provided by Kubernetes Secrets is beyond the scope of this lab, but will be covered in the future. For now, let's take a look at the simplest form of Kubernetes Secrets.
Create a Secret
Kubernetes Secrets are created by the same methods already used when creating ConfigMaps, and so will be familiar. The first step will be to create a Secret from the command line using literal values.
Create a Secret using a literal value.
kubectl create secret generic test --from-literal=pwd=P@ssw0rd
Example Output:
[oracle@ocne-control-01 ~]$ kubectl create secret generic test --from-literal=pwd=P@ssw0rd secret/test created
Confirm the Secret has created and stored the value entered.
kubectl describe secret test
Example Output:
[oracle@ocne-control-01 ~]$ kubectl describe secret test Name: test Namespace: default Labels: <none> Annotations: <none> Type: Opaque Data ==== pwd: 8 bytes
The Secret is created but does not show what is stored. Notice that it shows:
Type: Opaque
, what does that mean? This is the default Secret type assigned when the specific type is not provided in a Secret Configuration file (for more information look here ).Confirm the Secret has stored the value entered.
kubectl get secret test -o yaml
Example Output:
[oracle@ocne-control-01 ~]$ kubectl get secret test -o yaml apiVersion: v1 data: pwd: UEBzc3cwcmQ= kind: Secret metadata: creationTimestamp: "2023-09-22T12:12:02Z" name: test namespace: default resourceVersion: "5872" uid: 59526821-faff-414c-956a-23f308990115 type: Opaque
Notice the difference in how the
key-value
pair is displayed. This is because the contents of the Secret's values are Base64-encoded, whereas those stored in a ConfigMap are in clear text. The next step shows how to decode the stored value to confirm it matches the entered value.Decode the Base64-encoded value stored in the newly created Secret.
echo -n 'UEBzc3cwcmQ=' | base64 --decode
Example Output:
[oracle@ocne-control-01 ~]$ echo -n 'UEBzc3cwcmQ=' | base64 --decode P@ssw0rd[oracle@ocne-control-01 ~]$
Which confirms the stored value matches the entered value.
Create a Secret from a YAML File
Like ConfigMap's, creating a Secret declaratively (from a YAML file) is easier and faster.
Base64-encode the value.
Creating a Secret entails a slight catch because you have to Base64-encode the
value
to be stored yourself. While there are numerous websites you could use, it is also possible to do this yourself from the command line.echo -n 'P@ssw0rd' | base64
Example Output:
[oracle@ocne-control-01 ~]$ echo -n 'P@ssw0rd' | base64 UEBzc3cwcmQ=
Make a note of the value returned, because this will be used in the YAML file.
Create the Secret YAML file.
cat << 'EOF' > ~/password.yaml apiVersion: v1 kind: Secret metadata: name: password type: Opaque data: pwd: UEBzc3cwcmQ= EOF
Create the Secret.
kubectl apply -f password.yaml
Example Output:
[oracle@ocne-control-01 ~]$ kubectl apply -f password.yaml secret/password created
Confirm the Secret is created as expected.
kubectl get secret password -o yaml
Example Output:
[oracle@ocne-control-01 ~]$ kubectl get secret password -o yaml apiVersion: v1 data: pwd: UEBzc3cwcmQ= kind: Secret metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","data":{"pwd":"UEBzc3cwcmQ="},"kind":"Secret","metadata":{"annotations":{},"name":"password","namespace":"default"} "type":"Opaque"} creationTimestamp: "2023-09-22T13:01:53Z" name: password namespace: default resourceVersion: "10061" uid: b02c90a6-11e5-48c4-acd9-20c007696176 type: Opaque
Use a Secret as an Environment Variable
Using the key-value value pairs defined within a Secret is almost the same as for a ConfigMap. A difference is that instead of using the envFrom.ConfigMapRef
path, you have to use the envFrom.secretRef
instead.
Create the Pod definition.
cat << 'EOF' > ~/password-pod.yaml apiVersion: v1 kind: Pod metadata: name: password-pod spec: containers: - image: nginx:1.25.0 name: app envFrom: - secretRef: name: password EOF
Create the deployment.
kubectl apply -f password-pod.yaml
Inspect the Pod's environment variable has been injected.
kubectl exec password-pod -- env
Example Output:
[oracle@ocne-control-01 ~]$ kubectl exec password-pod -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=xterm HOSTNAME=password-pod pwd=P@ssw0rd KUBERNETES_PORT=tcp://10.96.0.1:443 KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443 KUBERNETES_PORT_443_TCP_PROTO=tcp KUBERNETES_PORT_443_TCP_PORT=443 KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1 KUBERNETES_SERVICE_HOST=10.96.0.1 KUBERNETES_SERVICE_PORT=443 KUBERNETES_SERVICE_PORT_HTTPS=443 NGINX_VERSION=1.25.0 NJS_VERSION=0.7.12 PKG_RELEASE=1~bullseye HOME=/root
You should see the Secret (
pwd=P@ssw0rd
) listed as one of the Pod's environment variables.
Mounting a Secret as a Volume
Mounting Secrets as Volumes is quite a common practice because it makes it easy to make them available to a Pod at runtime. This process is almost identical to that used to create a ConfigMap, except that in Secrets the attribute used is different. With Secrets, you need to reference the secrets.secretName
attribute.
Delete the original Pod deployment.
kubectl delete -f password-pod.yaml
Create the Pod definition.
cat << 'EOF' > ~/password-pod01.yaml apiVersion: v1 kind: Pod metadata: name: secret-pod spec: containers: - image: nginx:1.25.0 name: app volumeMounts: - name: secret-volume mountPath: /var/app readOnly: true volumes: - name: secret-volume secret: secretName: password EOF
Create the deployment.
kubectl apply -f password-pod01.yaml
Open an interactive shell to confirm the Secret lists as a file with the Pod's environment.
kubectl exec -it secret-pod -- /bin/sh
Check the expected Secret key is showing.
ls /var/app
Example Output:
# ls /var/app pwd
Confirm the file's contents correspond to the value defined in the ConfigMap.
Perform this task by confirming the
pwd
Secret key contains the expected Base64 value.cat /var/app/pwd
Example Output:
# cat /var/app/pwd P@ssw0rd#
Exit the Pod's interactive shell.
exit
Summary
This concludes the walkthrough introducing Kubernetes ConfigMaps and Secrets and how to get started using them in your application deployments.