Audit Oracle Linux with Auditd

Introduction

Auditd is a userspace system daemon running in the background, generating logs about activities performed on Oracle Linux.

Objectives

In this tutorial, you'll learn to:

Install the audit packages
Manage the audit service
Create audit rules
Search the audit logs

Prerequisites

Minimum of two Oracle Linux systems
Each system should have Oracle Linux installed and configured with:
- A non-root user account with sudo access

Deploy Oracle Linux

Note: If running in your own tenancy, read the linux-virt-labs GitHub project README.md and complete the prerequisites before deploying the lab environment.

Open a terminal on the Luna Desktop.

Clone the linux-virt-labs GitHub project.

git clone https://github.com/oracle-devrel/linux-virt-labs.git

Change into the working directory.
```
cd linux-virt-labs/ol
```

Install the required collections.

ansible-galaxy collection install -r requirements.yml

Deploy the lab environment.
```
ansible-playbook create_instance.yml -e localhost_python_interpreter="/usr/bin/python3.6"
```
The free lab environment requires the extra variable local_python_interpreter, which sets ansible_python_interpreter for plays running on localhost. This variable is needed because the environment installs the RPM package for the Oracle Cloud Infrastructure SDK for Python, located under the python3.6 modules.
The default deployment shape uses the AMD CPU and Oracle Linux 8. To use an Intel CPU or Oracle Linux 9, add -e instance_shape="VM.Standard3.Flex" or -e os_version="9" to the deployment command.
Important: Wait for the playbook to run successfully and reach the pause task. At this stage of the playbook, the installation of Oracle Linux is complete, and the instances are ready. Take note of the previous play, which prints the public and private IP addresses of the nodes it deploys and any other deployment information needed while running the lab.

Install the Audit Package

Oracle Linux installs the audit package by default.

Open a terminal and connect via SSH to the ol-node-01 instance.
```
ssh oracle@<ip_address_of_instance>
```
Check if the system preinstalled the audit package.
```
sudo dnf list installed "audit"
```
If it's not on the system, install it.
```
sudo dnf install -y audit
```
You can find the default configuration file for the auditd daemon at /etc/audit/auditd.conf.

Manage the Audit Service

Once you install the audit package, start the auditd service.

Check the current status.
```
sudo systemctl status auditd
```
The output shows the service is (running) and enabled. The enabled status indicates we configured the service to start at boot time. If this is not the case, start the service as the root user to collect audit information and write it in the logs.
Start the service.
```
sudo service auditd start
```
Configure auditd to start at boot time.
```
sudo systemctl enable auditd
```
Note: The only way to appropriately interact with the auditd daemon uses the service command.
The service command ensures recording the auid value. Use the systemctl command only with the enable and status actions.

Temporarily Enable and Disable Auditing

The Audit control utility, auditctl, interacts with the kernel Audit component to manage rules and control many settings and parameters of the event generation process.

Check the status of the kernel Audit subsystem.

sudo auditctl -s

Example Output:

[oracle@ol8-server ~]$ sudo auditctl -s
enabled 1
failure 1
pid 1399
rate_limit 0
backlog_limit 8192
lost 0
backlog 0
backlog_wait_time 60000
loginuid_immutable 0 unlocked

Temporarily disable auditd.
```
sudo auditctl -e 0
```
Re-enable auditd.
```
sudo auditctl -e 1
```
Note: The pid shows the auditd service process id. A pid value of 0 indicates the service is not running.
The auditctl -e enable flag also accepts a value of 2, which locks the audit configuration. If the audit configuration is locked in this manner, a reboot is required to unlock it. More details are available in man auditctl.

Locate Audit Rules and Logs

By default, Oracle Linux stores the audit logs in /var/log/audit/audit.log. You can locate the audit rules in /ect/audit/audit.rules. Oracle Linux generates the default ruleset from the file /etc/audit/rules.d/audit.rules.

View the audit rules and default ruleset.

sudo cat /etc/audit/audit.rules

sudo cat /etc/audit/rules.d/audit.rules

Example Output:

## First rule - delete all
-D

## Increase the buffers to survive stress events.
## Make this bigger for busy systems
-b 8192

## This determine how long to wait in burst of events
--backlog_wait_time 60000

## Set failure mode to syslog
-f 1

Rules with Audit Control Utility

Use the auditctl program to control the behavior, get status, and add or delete rules.

Add an audit rule that logs any attempt to read or modify the /etc/ssh/sshd_config file.
```
sudo auditctl -w /etc/ssh/sshd_config -p rwxa -k sshd_config
```
Where:
- -w: Creates a watch at the given path.
- -p: Sets permissions [read,write,execute,attribute] that trigger the watch.
- -k: Sets a key filter that uniquely identifies the audit records produced by a rule.
Show the rule.
```
sudo auditctl -l
```
New rules get added to the bottom of the list, but it's also possible to add them to the top.
Check if the new rule got added to the /etc/audit/audit.rules file.
```
sudo cat /etc/audit/audit.rules
```
Example Output:
```
[oracle@ol8-server ~]$ sudo cat /etc/audit/audit.rules
## This file is automatically generated from /etc/audit/rules.d
-D
-b 8192
-f 1
--backlog_wait_time 60000
```
The rule does not appear in the file. Why not?
Rules created by auditctl don't add to the audit.rules file. Therefore, these changes are transient and don't survive a system reboot.
Make the rule permanent by adding it to a custom ruleset file in /etc/audit/rules.d/my.rules.
The format of the added rule matches the syntax of the auditctl command without using auditctl. You should write the rules one per line and combine them to optimize performance.
```
sudo tee /etc/audit/rules.d/my.rules > /dev/null <<'EOF'
-w /etc/ssh/sshd_config -p rwxa -k sshd_config
EOF
```
Show the rule.
```
sudo cat /etc/audit/rules.d/my.rules
```
Test the rule.
```
cat /etc/ssh/sshd_config
```
The command returns, cat: /etc/ssh/sshd_config: Permission denied, and generates the below event in the audit.log.

Show the event.

sudo cat /var/log/audit/audit.log | grep sshd_config

Example Output:

type=CONFIG_CHANGE msg=audit(1648918923.746:266810): auid=1001 ses=15792 
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 op=add_rule key="sshd_config" list=4 res=1AUID="oracle"
type=SYSCALL msg=audit(1648923583.793:268315): arch=c000003e syscall=257 success=no exit=-13 a0=ffffff9c a1=7ffd08b6c575 > a2=0 a3=0 items=1 ppid=3406680 pid=3428336 auid=1001 uid=1001 gid=1001 euid=1001 suid=1001 fsuid=1001 egid=1001 
sgid=1001 fsgid=1001 tty=pts0 ses=15792 comm="cat" exe="/usr/bin/cat" 
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="sshd_config"ARCH=x86_64 SYSCALL=openat AUID="oracle" 
UID="oracle" GID="oracle" EUID="oracle" SUID="oracle" FSUID="oracle" EGID="oracle" SGID="oracle" FSGID="oracle"
type=PATH msg=audit(1648923583.793:268315): item=0 name="/etc/ssh/sshd_config" inode=67688941 dev=fc:00 mode=0100600 
ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:etc_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 
cap_frootid=0OUID="root" OGID="root"

Search Audit Logs

Another way to search the logs is to use the ausearch command.

Search for the sshd_config event.

sudo ausearch --key sshd_config

Example Output:

Email option is specified but /usr/lib/sendmail doesn't seem executable.
----
time->Sat Apr  2 17:02:03 2022
type=CONFIG_CHANGE msg=audit(1648918923.746:266810): auid=1001 ses=15792 
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 op=add_rule key="sshd_config" list=4 res=1
----
time->Sat Apr  2 18:19:43 2022
type=PROCTITLE msg=audit(1648923583.793:268315): proctitle=636174002F6574632F7373682F737368645F636F6E666967
type=PATH msg=audit(1648923583.793:268315): item=0 name="/etc/ssh/sshd_config" inode=67688941 dev=fc:00 mode=0100600 
ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:etc_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 
cap_frootid=0
type=CWD msg=audit(1648923583.793:268315): cwd="/home/oracle"
type=SYSCALL msg=audit(1648923583.793:268315): arch=c000003e syscall=257 success=no exit=-13 a0=ffffff9c a1=7ffd08b6c575 > a2=0 a3=0 items=1 ppid=3406680 pid=3428336 auid=1001 uid=1001 gid=1001 euid=1001 suid=1001 fsuid=1001 egid=1001 
sgid=1001 fsgid=1001 tty=pts0 ses=15792 comm="cat" exe="/usr/bin/cat" 
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="sshd_config"

The output from ausearch is easier to use and read based on the output format. We can add to the ease of reading the log by adding the -i or --interpret option. This option interprets numeric entities into text, such as converting a uid to an account name.

Repeat the search using the interpret option.

sudo ausearch -i -k sshd_config

Example Output:

Email option is specified but /usr/lib/sendmail doesn't seem executable.
----
type=CONFIG_CHANGE msg=audit(04/02/2022 17:02:03.746:266810) : auid=oracle ses=15792 
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 op=add_rule key=sshd_config list=exit res=yes 
----
type=PROCTITLE msg=audit(04/02/2022 18:19:43.793:268315) : proctitle=cat /etc/ssh/sshd_config 
type=PATH msg=audit(04/02/2022 18:19:43.793:268315) : item=0 name=/etc/ssh/sshd_config inode=67688941 dev=fc:00 
mode=file,600 ouid=root ogid=root rdev=00:00 obj=system_u:object_r:etc_t:s0 nametype=NORMAL cap_fp=none cap_fi=none 
cap_fe=0 cap_fver=0 cap_frootid=0 
type=CWD msg=audit(04/02/2022 18:19:43.793:268315) : cwd=/home/oracle 
type=SYSCALL msg=audit(04/02/2022 18:19:43.793:268315) : arch=x86_64 syscall=openat success=no exit=EACCES(Permission 
denied) a0=0xffffff9c a1=0x7ffd08b6c575 a2=O_RDONLY a3=0x0 items=1 ppid=3406680 pid=3428336 auid=oracle uid=oracle 
gid=oracle euid=oracle suid=oracle fsuid=oracle egid=oracle sgid=oracle fsgid=oracle tty=pts0 ses=15792 comm=cat exe=/
usr/bin/cat subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=sshd_config

For more formatting options and ways to search the audit.log, see man ausearch.

Load Pre-Configured Rules

Oracle Linux provides a script that merges all component audit rules files found in /etc/audit/rules.d. After merging, the new file replaces the existing /etc/audit/audit.rules. This script is built into the auditd service file and runs when the service starts.

Check if there are any existing rule changes to load.
```
sudo augenrules --check
```
The output states, Rules have changed and should be updated. This change is due to our previously created my.rules file located in/etc/audit/rules.d.
Delete the previously added sshd_config custom rule to easily track the new rule additions.
```
sudo auditctl -D -k sshd_config
```

Merge the my.rules custom rule file.

sudo augenrules --load

Example Output:

[oracle@ol8-server ~]$ sudo augenrules --load
No rules
enabled 1
failure 1
pid 1395
rate_limit 0
backlog_limit 8192
lost 0
backlog 0
backlog_wait_time 60000
enabled 1
failure 1
pid 1395
rate_limit 0
backlog_limit 8192
lost 0
backlog 0
backlog_wait_time 60000
enabled 1
failure 1
pid 1395
rate_limit 0
backlog_limit 8192
lost 0
backlog 0
backlog_wait_time 60000

Check the active audit rules.

sudo auditctl -l

Example Output:

[oracle@ol-lab-2022-03-23-182415-0 ~]$ sudo auditctl -l
-w /etc/ssh/sshd_config -p rwxa -k sshd_config

Add additional rules to a new file, new.rules.

Note: Only files ending in .rules are read by augenrules and loaded.

sudo tee /etc/audit/rules.d/new.rules > /dev/null <<'EOF'
-w /etc/passwd -p wa -k passwd_changes
-w /etc/selinux/ -p wa -k selinux_changes
EOF

Load the new rules.
```
sudo augenrules --load
```
Re-check the active rules.
```
sudo auditctl -l
```
The new rules are loaded and merged into the audit.rules file.
Show updated rules.
```
sudo cat /etc/audit/audit.rules
```
View the backup file.
The system made a backup file /etc/audit/audit.rules.prev as part of the merge.
```
sudo ls -l /etc/audit
```
Information provided by the Oracle Linux Auditing System aids with intrusion detection.
Check out the man pages for the utilities shown. Then, use what you learned to add custom audit rules to your system for particular logging events.

Next Steps

You should now be able to use the Oracle Linux audit daemon to track and monitor various activities on your system. Check out our other content on the Oracle Linux Training Station to learn more about Oracle Linux.