Setup HAProxy to Load Balance an Oracle Linux Automation Manager Cluster

0
0
Send lab feedback

Setup HAProxy to Load Balance an Oracle Linux Automation Manager Cluster

Introduction

Oracle Linux Automation Manager supports a cluster deployment of multiple control and execution nodes. With the control nodes acting as the entry point into the cluster, how does an administrator manage the traffic across these nodes? That is where a load balancer becomes beneficial. A load balancer efficiently distributes incoming network traffic across a group of backend servers or the control nodes in this solution. A load balancer ensures the Oracle Linux Automation Manager infrastructure is highly available and reliable and performance does not degrade.

In the free lab environment, the below inventory defines the cluster deployment. To help automate the cluster installation on your hardware, check out the playbook in the Oracle Linux Automation Manager section of the ansible-collections project.

This inventory creates a cluster of three control plane nodes, two execution nodes, and a remote database.

playbook-inv

This image shows the topology for this cluster.

topology

Although there are many load balancer options, this lab will leverage HAProxy. HAProxy, or High Availability Proxy, is an application layer (Layer 7) load balancer and high-availability solution that you can use to implement a reverse proxy for HTTP and TCP-based Internet services. An application layer load balancer often includes many features because it can inspect the traffic content it is routing and either modify content within each packet or make decisions about handling each packet based on its content. These features simplify implementing session persistence, TLS, ACLs, and HTTP rewrites and redirection.

Objectives

In this lab, you'll learn how to:

  • Install HAProxy
  • Configure HAProxy
  • Configure Oracle Linux Automation Manager to work behind a load balancer or proxy

Prerequisites

  • A cluster with Oracle Linux Automation Manager installed and cluster configured
  • An Oracle Cloud Infrastructure (OCI) account
  • A user in the OCI account with permission to work with resources in a compartment
  • Access to that accounts credentials and OCID information



For details on installing Oracle Linux Automation Manager, see the Oracle Linux Automation Manager Installation Guide .

Install and Configure HAProxy

Note: When using the free lab environment, see Oracle Linux Lab Basics for connection and other usage instructions.

Information: The free lab environment deploys a running Oracle Linux Automation Manager multi-node cluster. The deployment takes approximately 30 minutes to finish after launch. Therefore, you might want to step away while this runs and promptly return to complete the lab.

  1. Open a terminal from the Luna Desktop and ssh into the deployed haproxy instance.

    ssh oracle@<hostname or IP address>

    In the free lab environment, use the IP address of the haproxy VM.

  2. Install the HAProxy package

    sudo dnf install -y haproxy

  3. Create an HAProxy configuration file.

    The free lab environment provides a custom haproxy configuration file that enables haproxy statistics and end-to-end SSL encryption with the three control node backends.

    cat ~/haproxy.cfg

    The following output shows the differences between the package's default configuration and the values the free lab environment uses.

    sudo diff /etc/haproxy/haproxy.cfg ~/haproxy.cfg

    See the HAProxy configuration documentation for more details on using these options.

  4. Copy the custom HAProxy configuration file to the package's default configuration file location.

    sudo cp ~/haproxy.cfg /etc/haproxy/haproxy.cfg

  5. Check if the HAProxy configuration syntax is valid.

    sudo haproxy -f /etc/haproxy/haproxy.cfg -c

    The output alerts that it's unable to load the defined SSL certificate file.

  6. Create a self-signed certificate with OpenSSL.

    Given that the Oracle Linux Automation Manager control nodes use SSL, the load balancer must also use SSL. The other option is to have SSL terminate at the load balancer, which is outside this exercise's scope.

    openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout ~/haproxy.key -out ~/haproxy.crt

    Enter the requested information or just hit the ENTER key.

    This command generates the domain certificate (crt file) and the private key.

  7. Create a single PEM file for HAProxy.

    HAProxy requires the chain hierarchy of the certificates to go upside down in a single PEM file per the following order.

    • The Certificate for your domain
    • The intermediates in ascending order to the Root CA
    • A Root CA, if any (usually none)
    • Private Key
    cat ~/haproxy.crt ~/haproxy.key | sudo tee -a /etc/haproxy/server.pem > /dev/null

  8. Recheck the HAProxy syntax.

    sudo haproxy -f /etc/haproxy/haproxy.cfg -c

    The self-signed certificate alert is gone, but a new warning regarding the default Diffie-Hellman parameters appears. Although HAProxy will automatically handle this warning, the upstream documentation recommends specifying custom Diffie-Hellman parameters since that approach is more secure.

  9. Generate the custom DH parameters.

    sudo openssl dhparam -out /etc/haproxy/dhparams.pem 2048

  10. Add the DH parameters configuration to HAProxy.

    sudo sed -i -E 's|^(\s*)ssl-default-server-ciphers(.*)$|& \n\1ssl-dh-param-file /etc/haproxy/dhparams.pem|' /etc/haproxy/haproxy.cfg

  11. Recheck the HAProxy syntax.

    sudo haproxy -f /etc/haproxy/haproxy.cfg -c

    The results show a valid configuration.

  12. Attempt to start and enable the HAProxy service.

    sudo systemctl enable --now haproxy

    The haproxy.service fails to start and exists with an error code.

  13. Use the journald logs to review the error.

    sudo journalctl -u haproxy.service -l --no-pager

    Example Output:

    [oracle@haproxy ~]$ sudo journalctl -u haproxy.service -l --no-pager
-- Logs begin at Tue 2023-05-16 15:44:36 GMT, end at Tue 2023-05-16 19:08:15 GMT. --
May 16 19:04:57 haproxy systemd[1]: Starting HAProxy Load Balancer...
May 16 19:04:57 haproxy haproxy[83949]: [ALERT] 135/190457 (83949) : Starting proxy stats: cannot bind socket [0.0.0.0:8404]
May 16 19:04:57 haproxy systemd[1]: haproxy.service: Main process exited, code=exited, status=1/FAILURE
May 16 19:04:57 haproxy systemd[1]: haproxy.service: Failed with result 'exit-code'.
May 16 19:04:57 haproxy systemd[1]: Failed to start HAProxy Load Balancer.

    The error results from SELinux restricting HAProxy binding to the defined port, as Oracle Linux enables SELinux in enforcing mode by default.

  14. Enable the SELinux boolean that allows HAProxy to bind on any ports defined within the configuration file.

    sudo setsebool -P haproxy_connect_any=1

  15. Restart the HAProxy service.

    sudo systemctl restart haproxy

  16. Open the Linux firewall to allow access to HAProxy using HTTPS.

    sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload

Add Proxy Support to Oracle Linux Automation Manager

A proxy/load balancer acts as an arbitrator for client requests seeking resources from other servers. When establishing a session, the control node associates an IP address while requesting access to Oracle Linux Automation Manager. Per policy, using that session requires matching the original associated IP address.

Whether using HAProxy, Nginx, or an OCI Load Balancer in front of Oracle Linux Automation Manager to proxy requests, the REMOTE_HOST_HEADERS list variable provides the necessary support. Administrators can manage this setting by altering the default value of ['REMOTE_ADDR', 'REMOTE_HOST'].

  1. Open a terminal from the Luna Desktop and connect to the first control node instance.

    ssh oracle@<hostname or IP address>

    In the free lab environment, use the IP address of the control-node01 VM.

  2. Open a shell as the awx user.

    sudo su -l awx -s /bin/bash

  3. Enable proxy server support.

    cat << EOF | tee /etc/tower/conf.d/remote_host_headers.py > /dev/null
REMOTE_HOST_HEADERS = ['HTTP_X_FORWARDED_FOR', 'REMOTE_ADDR', 'REMOTE_HOST']
EOF

  4. Exit out of the awx user shell.

    exit

  5. Restart the Oracle Linux Automation Manager service.

    sudo systemctl restart ol-automation-manager

  6. Disconnect from the control node instance.

    exit

  7. Repeat for each control node in the cluster.

    This requirement includes the control-node02 and control-node03 VMs in the free lab environment.

Verify Access to the Cluster Through the Load Balancer

  1. Open a web browser and enter the URL.

    https://<haproxy_ip_address>

    Note: Approve the security warning based on the browser used. For Chrome, click the Advanced button and then the Proceed to localhost (unsafe) link.

  2. Log in to the Oracle Linux Automation Manager WebUI. Use the Username admin and the Password admin created during the automated deployment.

    olam2-haproxy-login

  3. The WebUI displays after a successful login.

    olam2-haproxy-webui

(Optional) View the HAProxy Statistics WebUI

  1. Open a terminal from the Luna Desktop and configure an SSH tunnel to the deployed HAProxy instance.

    ssh -L 8404:localhost:8404 oracle@<hostname or IP address>

    In the free lab environment, use the IP address of the haproxy VM. Access to the statistics page requires an SSH tunnel as firewalld blocks its port.

  2. Open a web browser and enter the URL.

    http://localhost:8404

    The statistics show that the initial login through HAProxy routes to the olam01 backend. As the login sessions receive a csrftoken as part of the session, any additional traffic also routes to olam01 until the session ends.

Summary

The successful login using the HAProxy URL shows a running Oracle Linux Automation Manager cluster behind a load balancer.

For More Information

Oracle Linux Automation Manager Documentation
Oracle Linux Automation Manager Training
Oracle Linux Training Station

SSR