Slow Disk I/O Detection and Resolution
Detect and resolve slow disk I/O issues on Linux servers using iotop, iostat, and performance optimization techniques.
Table of Contents
Introduction
Slow disk I/O is a common issue that severely impacts server performance. Database queries slow down, web applications become unresponsive, and the system feels sluggish overall. This guide covers how to detect and resolve slow disk I/O.
Symptoms
Typical signs of slow disk I/O:
- Commands hang for a long time (especially
ls,find, file read/write) - Database queries much slower than normal
- High
wa(I/O wait) value intoporhtop - Timeout errors in application logs
dforducommands take very long
Step 1: Check I/O Wait
# Check I/O wait with top
top
# Look at 'wa' column — above 5% indicates a problem
# Instant I/O status with vmstat
vmstat 1 10
# 'wa' column shows I/O wait percentage
# Disk statistics with iostat
iostat -x 1 5
# %util above 80% means the disk is saturated
If I/O wait is consistently above 10%, you have a disk I/O bottleneck.
Step 2: Which Process Is Using the Disk?
# Real-time disk I/O monitoring with iotop
sudo apt install iotop -y
sudo iotop -o # Show only processes with active I/O
# iotop snapshot
sudo iotop -b -n 5 -o
# Per-process I/O with pidstat
sudo apt install sysstat -y
pidstat -d 1 5
# Which files are open with lsof
sudo lsof | grep -E '(REG|DIR)' | sort -k7 -rn | head -20
Step 3: Disk Performance Test
# Simple write test
dd if=/dev/zero of=/tmp/test_write bs=1M count=1024 oflag=direct
# Result: X MB/s — 200+ MB/s normal for SSD, 80+ MB/s for HDD
# Simple read test
dd if=/tmp/test_write of=/dev/null bs=1M
# Comprehensive test with fio
sudo apt install fio -y
# Random read test
fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread \
--bs=4k --direct=1 --size=1G --numjobs=4 --runtime=60 \
--group_reporting --filename=/tmp/fio_test
# Clean up
rm /tmp/fio_test /tmp/test_write
Step 4: Disk Health Check
# SMART status
sudo apt install smartmontools -y
sudo smartctl -a /dev/sda
# Short SMART test
sudo smartctl -t short /dev/sda
# Disk errors
dmesg | grep -i -E '(error|fail|bad|ata|scsi|disk)'
If Reallocated_Sector_Ct or Pending_Sector_Count in SMART output is greater than 0, the disk is failing. Back up immediately.
Step 5: File System Optimization
# Disk usage
df -h
# Largest directories
du -sh /* 2>/dev/null | sort -rh | head -20
# Clean logs
sudo journalctl --vacuum-size=500M
# Add noatime to /etc/fstab for better read performance
# UUID=xxx / ext4 defaults,noatime 0 1
# I/O scheduler for SSD
echo 'none' | sudo tee /sys/block/sda/queue/scheduler
Step 6: Use tmpfs to Reduce I/O
# tmpfs for /tmp
echo 'tmpfs /tmp tmpfs defaults,noatime,mode=1777,size=2G 0 0' | sudo tee -a /etc/fstab
sudo mount -a
Keep tmpfs size below 25% of your total RAM.
Conclusion
To resolve slow disk I/O, first identify which process is using the disk with iotop and iostat, then check disk health with SMART. File system optimizations (noatime, I/O scheduler) and tmpfs usage can significantly improve performance. If the disk is physically failing, back up immediately and contact REXE support for disk replacement.
Related Articles
SSH Connection Problem: Can't Connect to Server
SSH connection troubleshooting guide: Path.net firewall rules, Default Block, creating a filter rule for SSH port, and step-by-step troubleshooting.
RDP Connection Problem: Can't Connect to Windows Server
RDP connection troubleshooting guide: Path.net firewall rules, Default Block, creating a filter rule for RDP port, and step-by-step troubleshooting.
How to Run an MTR Test and Submit Results to Support
Guide to running MTR network tests with WinMTR and Linux MTR, creating ICMP filter rules, and submitting results to support. Diagnose packet loss and latency.