Set Up Gluster Storage on Oracle Linux

Introduction

The following tutorial describes how to install and set up Gluster Storage for Oracle Linux across multiple nodes that are running Oracle Linux 8 or later.

Objectives

Upon completion of this Lab you will be able to:

Enable access to Gluster repos and packages and install Gluster server packages
Prepare each node to work as a Gluster storage node
Create a storage pool
Create volumes
Test and mount a Gluster volume

Prerequisites

At least 4 Oracle 8 systems, 3 to constitute the nodes of a Gluster pool and 1 to function as a Gluster client.
Block storage assigned to the systems to be used with Gluster. For information about adding block storage to Oracle Cloud Infrastructure instances, see https://docs.oracle.com/en-us/iaas/Content/Block/Tasks/attachingavolume.htm .

Setting Up Gluster Storage

Note: When using the free lab environment, see Oracle Linux Lab Basics for connection and other usage instructions.

Install the Gluster packages

Perform this procedure on all of the designated nodes of the cluster.

Enable access to the Gluster packages.

sudo dnf install oracle-gluster-release-el8 -y
sudo dnf config-manager --enable ol8_gluster_appstream ol8_baseos_latest ol8_appstream

Install the Gluster server packages.
```
sudo dnf install @glusterfs/server -y
```
Enable the Gluster server service.
```
sudo systemctl enable --now glusterd
```
Configure the firewall to allow traffic on the ports that are specifically used by Gluster.
```
sudo firewall-cmd --permanent --add-service=glusterfs
sudo firewall-cmd --reload
```

Prepare the nodes for the pool configuration

Perform this procedure on all of the designated nodes of the cluster.

Create an XFS file system for the cluster data.
The following commands create an XFS file system on the /dev/sdb device, which is then mounted on the directory /data/gluster/fs/myvolume/mybrick. An entry is added to /etc/fstab to make the mount permanent.
```
sudo mkfs.xfs -f -i size=512 -L glusterfs /dev/sdb
sudo mkdir -p /data/glusterfs/myvolume/mybrick
echo 'LABEL=glusterfs /data/glusterfs/myvolume/mybrick xfs defaults 0 0'|sudo tee -a /etc/fstab
sudo mount -a
```
Ensure that each node has a resolvable name across all the nodes in the pool.
If your site is not set up with DNS resolution for your host names, you can use the /etc/hosts file instead. You would edit the file of each node to contain entries for all of your Gluster nodes.
In the lab, however, name resolution is already configured.

Create the trusted storage pool

You only need to perform these steps from one of the designated hosts. To create the pool from ol-node01, for example, you would type the following:

sudo gluster peer probe ol-node02
sudo gluster peer probe ol-node03

To list the peer members of the pool, type the following:

sudo gluster peer status

The command lists the peers of the node from which you issued the command. In this case, the peers are ol-node02 and ol-node03.

To list the nodes that comprise the pool, type the following:

sudo gluster pool list

The generated list includes both the node that you are using and its peers.

Create a volume

This example shows how to create a replicated volume.

Note: Replicated volumes are one way of configuring volumes in the cluster. For other volume configurations and the instructions for their creation, refer to the Gluster Storage for Oracle Linux documentation.

You only need to perform these steps from one of the designated hosts.

Create the volume out of the nodes in the pool.

sudo gluster volume create myvolume replica 3 ol-node{01,02,03}:/data/glusterfs/myvolume/mybrick/brick

Start the volume.
```
sudo gluster volume start myvolume
```
To display information about the volume, type:
```
sudo gluster volume info
```

Volume information would be similar to the following:

Volume Name: myvolume Type: Replicate Volume ID: 7cec07cb-7001-41d7-9a6f-e1aa05f030fc Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: ol-node01:/data/glusterfs/myvolume/mybrick/brick Brick2: ol-node02:/data/glusterfs/myvolume/mybrick/brick Brick3: ol-node03:/data/glusterfs/myvolume/mybrick/brick Options Reconfigured: transport.address-family: inet nfs.disable: on performance.client-io-threads: off

Test the Gluster setup from a client system

This section shows you how to test access to the Gluster volume from a client.

Note: The mechanism that is described in this section for mounting the file system and accessing the Gluster volume is based on a Gluster FUSE client. Other services, such as NFS or CIFS, can also be used for the same purpose and are described in the Gluster Storage for Oracle Linux documentation.

Perform the following steps on ol-client.

Enable access to the Gluster packages.

sudo dnf install oracle-gluster-release-el8 -y
sudo dnf config-manager --enable ol8_gluster_appstream ol8_baseos_latest ol8_appstream
sudo dnf module enable glusterfs -y

Install the Gluster client packages.
```
sudo dnf install @glusterfs/client -y
```
Create the directory for the client's data.
```
sudo mkdir /mydata
```
Mount the Gluster volume on the directory.
```
sudo mount -t glusterfs ol-node01:myvolume /mydata
```
This step mounts the volume in ol-node01. However, you can mount the volume from any node in the cluster. Likewise, if the node of a previous mount becomes unavailable, you can remount against any available cluster node.
Set appropriate permissions to the new mount, for example:
```
sudo chmod 777 /mydata
```
To make the mount permanent, add it to the /etc/fstab, for example, by typing the following command:
```
echo 'LABEL=ol-node01:/myvolume /mydata glusterfs defaults,_netdev 0 0'|sudo tee -a /etc/fstab
```
Create a test file in the data directory.
```
touch /mydata/test
```
On each of the Gluster server node, run the following command:
```
ls -l /data/glusterfs/myvolume/mybrick/brick
```
The three nodes ol-node01, ol-node02, and ol-node03 should list the test file as existing in their respective directories.

Video Demonstrations

The video tutorial Introduction to GlusterFS on Oracle Linux 8 provides an overview of Gluster Storage for Oracle Linux and how it is set up on your Oracle Linux system.

For more information

See the documentation in https://docs.oracle.com/en/operating-systems/oracle-linux/gluster-storage/ .