Malware & Rootkit Scanning: rkhunter and ClamAV Guide

Although Linux servers are targeted less frequently than Windows systems, they are not immune to malware and rootkit attacks. Regular security scanning is critical, especially for internet-facing servers. This guide walks you through building a comprehensive security scanning infrastructure using rkhunter and ClamAV.

What is rkhunter?

rkhunter (Rootkit Hunter) is an open-source security tool designed to detect rootkits, backdoors, and local exploits on Linux systems. It checks hash values, permissions, and known rootkit signatures of system files.

Installing rkhunter

hljs bash

# Ubuntu/Debian
sudo apt update
sudo apt install rkhunter -y

# CentOS/AlmaLinux/Rocky Linux
sudo dnf install rkhunter -y
# or from EPEL repo:
sudo dnf install epel-release -y
sudo dnf install rkhunter -y

Updating the rkhunter Database

hljs bash

# Update signature database
sudo rkhunter --update

# Record initial hash values of system files
# Run this on a clean system!
sudo rkhunter --propupd

Only run --propupd when you are sure your system is clean. This command saves current file hashes as reference values.

Running rkhunter Scans

hljs bash

# Full system scan
sudo rkhunter --check

# Non-interactive mode (for cron)
sudo rkhunter --check --skip-keypress

# Show only warnings
sudo rkhunter --check --rwo

# Verbose output
sudo rkhunter --check --verbose-logging

# View log file
sudo cat /var/log/rkhunter.log

rkhunter Configuration

hljs bash

sudo nano /etc/rkhunter.conf

hljs ini

# /etc/rkhunter.conf important settings

# Email notification
MAIL-ON-WARNING=admin@example.com
MAIL_CMD=mail -s "[rkhunter] Warning - $(hostname)"

# Daily update
AUTO_X_DETECT=1
UPDATE_MIRRORS=1
MIRRORS_MODE=0

# Whitelist - known good files
SCRIPTWHITELIST=/usr/bin/egrep
SCRIPTWHITELIST=/usr/bin/fgrep
SCRIPTWHITELIST=/usr/bin/which

# Hidden directory check
ALLOW_HIDDEN_SYS_FILELIST=0

# SSH root login check
ALLOW_SSH_ROOT_USER=no
ALLOW_SSH_PROT_V1=0

Interpreting rkhunter Results

rkhunter output shows three states:

OK (Green): File or setting is safe
Warning (Yellow): Suspicious condition, manual review required
Not found (White): Tool or file not present on system

hljs bash

# Filter only warnings
sudo grep -i warning /var/log/rkhunter.log

# View summary report
sudo tail -50 /var/log/rkhunter.log

Some warnings may be false positives. For example, custom-compiled tools or modified system files can trigger warnings. Manually verify each warning.

What is ClamAV?

ClamAV is an open-source antivirus engine. It is widely used on Linux servers for file scanning, email attachment checking, and web content scanning.

Installing ClamAV

hljs bash

# Ubuntu/Debian
sudo apt update
sudo apt install clamav clamav-daemon -y

# CentOS/AlmaLinux
sudo dnf install clamav clamd clamav-update -y

Updating the Virus Database

hljs bash

# Stop ClamAV service (for update)
sudo systemctl stop clamav-freshclam

# Manually update database
sudo freshclam

# Restart service
sudo systemctl start clamav-freshclam
sudo systemctl enable clamav-freshclam

# Check update status
sudo freshclam --verbose

Scanning with ClamAV

hljs bash

# Scan a single directory
sudo clamscan -r /home

# Full system scan (may be slow)
sudo clamscan -r / --exclude-dir="^/sys" --exclude-dir="^/proc" --exclude-dir="^/dev"

# Show only infected files
sudo clamscan -r /var/www --infected

# Remove infected files (use with caution!)
sudo clamscan -r /home --infected --remove

# Move infected files to quarantine
sudo clamscan -r /home --infected --move=/quarantine

# Save scan results to file
sudo clamscan -r / --log=/var/log/clamav-scan.log --exclude-dir="^/sys" --exclude-dir="^/proc"

Using ClamAV Daemon (clamd)

clamd is the background ClamAV service and is much faster for repeated scans.

hljs bash

# clamd configuration
sudo nano /etc/clamav/clamd.conf

# /etc/clamav/clamd.conf
LocalSocket /var/run/clamav/clamd.ctl
User clamav
ScanMail true
ScanArchive true
ArchiveBlockEncrypted false
MaxDirectoryRecursion 20
FollowDirectorySymlinks false
FollowFileSymlinks false
ReadTimeout 180
MaxThreads 12
MaxConnectionQueueLength 15
LogSyslog false
LogFile /var/log/clamav/clamav.log
LogFileMaxSize 2M
LogTime true
LogClean false
LogVerbose false
DatabaseDirectory /var/lib/clamav

hljs bash

# Start clamd service
sudo systemctl start clamav-daemon
sudo systemctl enable clamav-daemon

# Fast scan with clamdscan
sudo clamdscan -r /home

Setting Up Scheduled Scans

Automated Scanning with Cron

hljs bash

# Create cron file
sudo nano /etc/cron.d/security-scan

hljs bash

# /etc/cron.d/security-scan

# rkhunter scan every night at 02:00
0 2 * * * root /usr/bin/rkhunter --check --skip-keypress --report-warnings-only 2>&1 | mail -s "[rkhunter] $(hostname) Daily Report" admin@example.com

# ClamAV full scan every Sunday at 03:00
0 3 * * 0 root /usr/bin/clamscan -r / --exclude-dir="^/sys" --exclude-dir="^/proc" --exclude-dir="^/dev" --infected --log=/var/log/clamav-weekly.log

# ClamAV database update every day
0 1 * * * root /usr/bin/freshclam --quiet

Creating a Scan Script

hljs bash

sudo nano /usr/local/bin/security-scan.sh

hljs bash

#!/bin/bash
# Comprehensive security scan script

DATE=$(date +%Y%m%d_%H%M%S)
LOG_DIR="/var/log/security-scans"
REPORT="$LOG_DIR/scan_$DATE.txt"

mkdir -p $LOG_DIR

echo "=== Security Scan Report: $DATE ===" > $REPORT
echo "Server: $(hostname)" >> $REPORT
echo "" >> $REPORT

# rkhunter scan
echo "--- rkhunter Scan ---" >> $REPORT
rkhunter --check --skip-keypress --report-warnings-only >> $REPORT 2>&1

# ClamAV scan
echo "" >> $REPORT
echo "--- ClamAV Scan ---" >> $REPORT
clamscan -r /home /var/www /tmp --infected >> $REPORT 2>&1

# Send results by email
if grep -q "Warning\|FOUND" $REPORT; then
    mail -s "[WARNING] Security Scan Report - $(hostname)" admin@example.com < $REPORT
fi

echo "Scan completed: $REPORT"

hljs bash

sudo chmod +x /usr/local/bin/security-scan.sh

Incident Response

If a threat is detected during scanning:

1. Isolate the System

hljs bash

# Check network connections
sudo ss -tlnp
sudo netstat -tlnp

# Find suspicious connections
sudo ss -anp | grep ESTABLISHED

# Restrict network access if needed
sudo ufw default deny incoming
sudo ufw default deny outgoing
sudo ufw allow ssh  # Keep SSH access

2. Collect Evidence

hljs bash

# Save running processes
ps aux > /tmp/processes_$(date +%Y%m%d).txt

# Save network connections
ss -anp > /tmp/connections_$(date +%Y%m%d).txt

# Find recently modified files
find / -mtime -1 -type f 2>/dev/null | grep -v proc | grep -v sys

# Check cron jobs
crontab -l
ls -la /etc/cron*

3. Cleanup and Recovery

hljs bash

# Quarantine suspicious files
mkdir -p /quarantine
mv /path/to/suspicious/file /quarantine/

# Verify system files
sudo debsums -c  # Debian/Ubuntu
sudo rpm -Va     # CentOS/RHEL

# Rootkit cleanup (last resort: reinstall)
# For serious infections, reinstall the server

When a serious rootkit infection is detected, the safest solution is to reinstall the server from scratch. Since rootkits can modify system tools, cleanup may not always be reliable.

Conclusion

Regular malware and rootkit scanning is a fundamental component of server security. Using rkhunter and ClamAV together creates a comprehensive protection layer. Scheduled scans and email notifications allow you to detect security threats early.

Malware & Rootkit Scanning: rkhunter and ClamAV Guide

Table of Contents