Upgrade Oracle Linux Automation Manager

Introduction

Existing Oracle Linux Automation Manager administrators are familiar with its all-in-one single-instance deployment.

Oracle Linux Automation Manager allows administrators to upgrade their existing deployments and later migrate the upgraded instance into a clustered deployment. When upgrading, the instance converts to a single-host deployment and configures itself as a hybrid node.

After following this tutorial, you'll know how to upgrade Oracle Linux Automation Manager from a previous release.

Objectives

In this tutorial, you'll learn how to:

• Upgrade Oracle Linux Automation Manager

Prerequisites

• A system with a previous release of Oracle Linux Automation Manager installed.

  For details on installing Oracle Linux Automation Manager, see Installing Oracle Linux Automation Manager on Oracle Linux .

Deploy Oracle Linux Automation Manager

Note: If running in your own tenancy, read the linux-virt-labs GitHub project README.md and complete the prerequisites before deploying the lab environment.

1. Open a terminal on the Luna Desktop.

2. Clone the linux-virt-labs GitHub project.

   git clone https://github.com/oracle-devrel/linux-virt-labs.git

3. Change into the working directory.

   cd linux-virt-labs/olam

4. Install the required collections.

   ansible-galaxy collection install -r requirements.yml

5. Update the Oracle Linux instance configuration.

   cat << EOF | tee instances.yml > /dev/null
   compute_instances:
     1:
       instance_name: "olam-node"
       type: "control"
   passwordless_ssh: true
   olam_type: v1
   EOF

6. Deploy the lab environment.

   ansible-playbook create_instance.yml -e ansible_python_interpreter="/usr/bin/python3.6" -e "@instances.yml"

   The free lab environment requires the extra variable ansible_python_interpreter because it installs the RPM package for the Oracle Cloud Infrastructure SDK for Python. The location for this package's installation is under the python3.6 modules.

   The default deployment shape uses the AMD CPU and Oracle Linux 8. To use an Intel CPU or Oracle Linux 9, add -e instance_shape="VM.Standard3.Flex" or -e os_version="9" to the deployment command.

   Important: Wait for the playbook to run successfully and reach the pause task. The Oracle Linux Automation Manager installation is complete at this stage of the playbook, and the instances are ready. Take note of the previous play, which prints the public and private IP addresses of the nodes it deploys.

Log into the WebUI

1. Open a terminal and configure an SSH tunnel to Oracle Linux Automation Manager.

   ssh -L 8444:localhost:443 oracle@<hostname_or_ip_address>

   In the free lab environment, use the external IP address of the olam-node instance.

2. Open a web browser and enter the URL.

   https://localhost:8444

   Note: Approve the security warning based on the browser used. Click the **Advanced button for Chrome and then the Proceed to localhost (unsafe) link.

3. Login to Oracle Linux Automation Manager with the USERNAME admin and the PASSWORD admin created during the automated deployment.

   

4. After login, the WebUI displays.

   

Run the Upgrade

1. Switch to the terminal connected to the olam-node instance running the Oracle Linux Automation Manager instance.

2. Stop the Oracle Linux Automation Manager service.

   sudo systemctl stop ol-automation-manager

3. Create a backup of the secret key used by Oracle Linux Automation Manager for encrypting automation secrets in the database.

   sudo cp /etc/tower/SECRET_KEY ~/SECRET_KEY.olamv1

4. Create a backup of the database.

   This command dumps the contents of the database into a file containing all the necessary SQL commands and data to restore the database.

   sudo su - postgres -c pg_dumpall > /tmp/olamv1_db_dump

5. Stop the database service.

   sudo systemctl stop postgresql

6. Remove the existing database data files.

   sudo rm -rf /var/lib/pgsql/data

   This command removes the existing database data files. The database gets rebuilt later using the dump file.

7. Uninstall the current database package.

   sudo dnf -y remove postgresql

8. Enable the module stream for the new database version.

   Oracle Linux Automation Manager allows using PostgreSQL 12 and 13 when upgrading. This lab uses version 13.

   sudo dnf -y module reset postgresql
   sudo dnf -y module enable postgresql:13

9. Update the Oracle Linux Automation Manager repository package.

   sudo dnf -y update oraclelinux-automation-manager-release-el8

   Note: This step will report that there is nothing to do in the free lab environment as the initial deployment already installed the latest repository package for Oracle Linux Automation Manager.

10. Disable all of the Oracle Linux Automation Manager repositories.

    sudo dnf config-manager --disable ol8_automation ol8_automation2 ol8_automation2.2

11. Upgrade the Oracle Linux Automation Manager packages to Release 2.

    sudo dnf -y install ol-automation-manager --enablerepo ol8_automation2

12. Enable the current repository.

    sudo dnf config-manager --enable ol8_automation2.2

13. Update the Oracle Linux Automation Manager package.

    sudo dnf -y update ol-automation-manager

    Note: The following expected messages are displayed in the output during the upgrade.

      Upgrading        : ol-automation-manager-2.1.0-10.el8.x86_64            26/28 
    warning: /etc/tower/settings.py created as /etc/tower/settings.py.rpmnew
    
      Running scriptlet: ol-automation-manager-2.1.0-10.el8.x86_64            26/28 
    ValueError: File context for /var/run/tower(/.*)? already defined

14. Install the database server.

    sudo dnf -y install postgresql-server

15. Initialize, then start and restore the database.

    sudo postgresql-setup --initdb
    sudo systemctl start postgresql
    sudo su - postgres -c 'psql -d postgres -f /tmp/olamv1_db_dump'

16. Confirm the database is available.

    sudo su - postgres -c 'psql -l | grep awx'

    Example Output:

    [oracle@control-node ~]$ sudo su - postgres -c 'psql -l | grep awx'
     awx       | awx      | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 

    The output displays the awx account created within the database after performing the restore using the dump file.

17. Replace the default Oracle Linux Automation Manager global settings file.

    After the Oracle Linux Automation Manager package upgrade, the new rpm creates a default configuration file with an extension of .rpmnew. This process allows system administrators to check configuration differences and manually migrate necessary changes.

    sudo mv /etc/tower/settings.py /etc/tower/settings.py.save
    sudo mv /etc/tower/settings.py.rpmnew /etc/tower/settings.py
    

18. View the differences between the new and old global settings files.

    sudo diff /etc/tower/settings.py /etc/tower/settings.py.save

    Note: Any custom settings previously added to the settings.py file should be set in the WebUI or moved to a file under /etc/tower/conf.d.

19. Create a custom settings file containing the required settings.

    cat << EOF | sudo tee /etc/tower/conf.d/olamv2.py > /dev/null
    CLUSTER_HOST_ID = '$(hostname -i)'
    DEFAULT_EXECUTION_QUEUE_NAME = 'tower'
    DEFAULT_CONTROL_PLANE_QUEUE_NAME = 'tower' 
    EOF

    This command sets the CLUSTER_HOST_ID setting to the system's IP address and the default queue names to the pre-upgraded version's default name of tower. Values entered into configuration files need to follow Python syntax format.

    Note: When upgrading Oracle Linux Automation Manager, the CLUSTER_HOST_ID value needs to match the value from the previous installation to avoid Jobs getting stuck in a pending status after the upgrade. The default queue name in Release 1 is tower, while in Release 2, it is default.

20. Update the Receptor configuration file.

    The Receptor mesh is an overlay network that creates peer-to-peer connections between controllers and executors to handle the distribution of work.

    sudo sed -i "s/0.0.0.0/$(hostname -i)/" /etc/receptor/receptor.conf

21. Verify the Receptor configuration file.

    sudo cat /etc/receptor/receptor.conf

    • id: is the hostname or IP address of the Oracle Linux Automation Manager system.
    • port: is the TCP listening port for the Receptor mesh, which defaults to TCP port 27199.

22. Deploy Oracle Linux Automation Manager.

    1. Open a shell as the awx user.

       sudo su -l awx -s /bin/bash

    2. Migrate any existing containers to the latest podman version while keeping the unprivileged namespaces alive.

       podman system migrate

    3. Pull the Oracle Linux Automation Engine execution environment for Oracle Linux Automation Manager.

       podman pull container-registry.oracle.com/oracle_linux_automation_manager/olam-ee:2.2

    4. Use the awx-manage utility to set up the Oracle Linux Automation Manager instance.

       awx-manage makemigrations --merge
       awx-manage migrate
       awx-manage register_default_execution_environments
       exit
       

23. Restore the backup of the secret key file.

    sudo cp ~/SECRET_KEY.olamv1 /etc/tower/SECRET_KEY

24. Append the following parameter to the Oracle Linux Automation Manager settings file.

    cat << EOF | sudo tee -a /etc/tower/conf.d/olamv2.py > /dev/null
    
    # OLAM Reaper Job Status Tracking
    REAPER_TIMEOUT_SEC = 60
    EOF

25. Replace the default configuration for NGINX.

    cat << EOF | sudo tee /etc/nginx/nginx.conf > /dev/null
    user nginx;
    worker_processes auto;
    error_log /var/log/nginx/error.log;
    pid /run/nginx.pid;
    
    # Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
    include /usr/share/nginx/modules/*.conf;
    
    events {
    worker_connections 1024;
    }
    
    http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    
    access_log  /var/log/nginx/access.log  main;
    
    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;
    
    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;
    
    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;
    }
    EOF   

26. Update database tuning parameters.

    sudo sed -i 's/max_connections = [0-9]\+/max_connections = 1024/' /var/lib/pgsql/data/postgresql.conf

    sudo sed -i 's/shared_buffers = [0-9]\+MB/shared_buffers = 5400MB/' /var/lib/pgsql/data/postgresql.conf

    sudo sed -i 's/#\(work_mem = \)[0-9]\+MB/\1'540MB/ /var/lib/pgsql/data/postgresql.conf

    sudo sed -i 's/#\(maintenance_work_mem = \)[0-9]\+MB/\1'720MB/ /var/lib/pgsql/data/postgresql.conf

27. Restart the database.

    sudo systemctl restart postgresql

28. Restart NGINX.

    sudo systemctl restart nginx

29. Start Oracle Linux Automation Manager.

    sudo systemctl start ol-automation-manager

Verify the Upgrade

1. Refresh the web browser window used to display the previous WebUI, or open a new web browser window and enter the URL.

   https://localhost:8444

   The port used in the URL needs to match the local port of the SSH tunnel.

2. Log into Oracle Linux Automation Manager again with the Username admin and the Password admin.

   

3. After login, the WebUI displays.

   

Next Steps

The upgrade is complete, and the jobs are ready to run. To learn more about Oracle Linux Automation Manager, check out the documentation and continue to build your skills by checking out our other Oracle Linux Automation Manager training on Oracle Linux Training Station.

Related Links

Oracle Linux Automation Manager Documentation
Oracle Linux Automation Manager Training
Oracle Linux Training Station