Skip to main content
Back to Category

Diagnosing High CPU and RAM Usage

Guide to diagnosing high CPU and RAM usage on servers using top, htop, ps commands, OOM killer mechanism, process analysis, and resource optimization.

Read time: 15 min Troubleshooting
cpuramtophtoppsoom-killermemoryperformance
Author
REXE Teknoloji Network & Security Team
Editor
REXE Teknoloji Technical Editorial
First published
Last updated

Introduction

High CPU or RAM usage on your server can cause performance degradation, service interruptions, and even make the server unresponsive. In this guide, we'll examine the tools and methods you can use to identify resource-consuming processes and resolve the issue.

Quick Status Check

top Command

top is the most fundamental tool for real-time system resource monitoring:

hljs bash
top

Key fields:

  • %CPU: Processor usage percentage
  • %MEM: Memory usage percentage
  • RES: Physical RAM usage
  • VIRT: Virtual memory usage
  • TIME+: Total CPU time

Useful shortcuts:

  • P — Sort by CPU usage
  • M — Sort by memory usage
  • k — Kill process (enter PID)
  • 1 — Show each CPU core separately
  • c — Show full command line

htop Command

htop is a more advanced and user-friendly version of top:

hljs bash
# Installation
sudo apt install htop -y

# Run
htop

htop advantages:

  • Colorful visual interface
  • Mouse support
  • Horizontal and vertical scrolling
  • Tree view (F5)
  • Filtering (F4) and search (F3)

One-Time Status View

hljs bash
# Top 10 processes by CPU usage
ps aux --sort=-%cpu | head -11

# Top 10 processes by RAM usage
ps aux --sort=-%mem | head -11

# Processes of a specific user
ps -u www-data --sort=-%cpu

# Process tree view
ps auxf

CPU Usage Analysis

Identifying High CPU Processes

hljs bash
# Real-time CPU usage
mpstat -P ALL 1 5

# Per-process CPU usage (5 second intervals)
pidstat -u 5 3

# CPU usage of a specific process
pidstat -p PID_NUMBER 1

CPU Usage History

hljs bash
# Historical CPU data with sar
sar -u 1 10

# Last 24 hours CPU usage
sar -u -f /var/log/sysstat/sa$(date +%d)

# Load average check
uptime
cat /proc/loadavg

Load average should be compared with your CPU core count. On a 4-core server, a load average of 4.0 means 100% CPU utilization. 8.0 means the system is overloaded.

Analyzing CPU-Intensive Processes

hljs bash
# Detailed process information
ps -p PID_NUMBER -o pid,ppid,user,%cpu,%mem,etime,cmd

# Open files of a process
lsof -p PID_NUMBER

# System calls of a process
strace -p PID_NUMBER -c

# Process threads
ps -T -p PID_NUMBER

RAM Usage Analysis

Memory Status Check

hljs bash
# General memory status
free -h

# Detailed memory information
cat /proc/meminfo

# Memory usage summary
vmstat 1 5

Reading free -h output:

              total        used        free      shared  buff/cache   available
Mem:           16Gi       8.2Gi       1.1Gi       256Mi       6.7Gi       7.2Gi
Swap:          4.0Gi       512Mi       3.5Gi

On Linux, "free" memory may appear low because the system uses free memory as disk cache. The actual available memory is in the "available" column.

Identifying RAM-Intensive Processes

hljs bash
# Sort by RSS (Resident Set Size)
ps aux --sort=-rss | head -11

# Detailed memory analysis with smem
sudo apt install smem -y
smem -t -k -s rss

# Per-process memory usage
pmap -x PID_NUMBER

Swap Usage Analysis

hljs bash
# Processes using swap
for file in /proc/*/status; do
  awk '/VmSwap|Name/{printf $2 " " $3}END{print ""}' $file
done | sort -k 2 -n -r | head -10

# Monitor swap activity
vmstat 1 10

# Swappiness value
cat /proc/sys/vm/swappiness

# Set swappiness (temporary)
sudo sysctl vm.swappiness=10

# Set swappiness (permanent)
echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.conf

OOM Killer Mechanism

The Linux kernel activates the OOM (Out of Memory) Killer when memory is exhausted, terminating the process using the most memory.

Checking OOM Killer Logs

hljs bash
# Search for OOM killer events
dmesg | grep -i "oom\|out of memory"

# OOM in system logs
journalctl -k | grep -i oom

# Recent OOM events
grep -i 'oom' /var/log/syslog

OOM Score Check

hljs bash
# OOM score of a process
cat /proc/PID_NUMBER/oom_score

# OOM scores of all processes
for proc in $(find /proc -maxdepth 1 -regex '/proc/[0-9]+'); do
  printf "%s\t%s\t%s\n" "$(cat $proc/oom_score 2>/dev/null)" "$(cat $proc/oom_adj 2>/dev/null)" "$(cat $proc/comm 2>/dev/null)"
done | sort -rn | head -10

Protecting from OOM Killer

hljs bash
# Protect critical process from OOM killer
echo -1000 > /proc/PID_NUMBER/oom_score_adj

# OOM protection in systemd service
# /etc/systemd/system/critical-service.service
[Service]
OOMScoreAdjust=-1000

OOM Killer terminating a process indicates a serious memory issue on your server. Adding OOM protection without finding the root cause won't solve the problem.

Process Management

Process Priority Adjustment

hljs bash
# Start with nice value (-20 highest, 19 lowest priority)
nice -n 10 ./long-task.sh

# Change priority of running process
renice -n 15 -p PID_NUMBER

# Lower priority of all user processes
renice -n 10 -u username

Terminating Processes

hljs bash
# Graceful termination (SIGTERM)
kill PID_NUMBER

# Force termination (SIGKILL)
kill -9 PID_NUMBER

# Terminate by name
killall process_name
pkill -f "process_pattern"

# Kill processes exceeding CPU threshold
ps aux | awk '$3 > 90 {print $2}' | xargs kill -9

Resource Monitoring Script

hljs bash
#!/bin/bash
# Resource usage monitoring and alert script

CPU_THRESHOLD=90
MEM_THRESHOLD=90
LOG_FILE="/var/log/resource-monitor.log"

while true; do
  TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
  
  # CPU check
  CPU_USAGE=$(top -bn1 | grep 'Cpu(s)' | awk '{print $2}' | cut -d'.' -f1)
  if [ "$CPU_USAGE" -gt "$CPU_THRESHOLD" ]; then
    echo "[$TIMESTAMP] WARNING: CPU usage $CPU_USAGE%" >> $LOG_FILE
    echo "Top CPU consuming processes:" >> $LOG_FILE
    ps aux --sort=-%cpu | head -6 >> $LOG_FILE
  fi
  
  # RAM check
  MEM_USAGE=$(free | grep Mem | awk '{printf("%.0f", $3/$2 * 100)}')
  if [ "$MEM_USAGE" -gt "$MEM_THRESHOLD" ]; then
    echo "[$TIMESTAMP] WARNING: RAM usage $MEM_USAGE%" >> $LOG_FILE
    echo "Top RAM consuming processes:" >> $LOG_FILE
    ps aux --sort=-%mem | head -6 >> $LOG_FILE
  fi
  
  sleep 60
done

Conclusion

High CPU and RAM usage are critical issues that directly affect server performance. With tools like top, htop, ps, and vmstat, you can identify problematic processes, adjust priorities with nice/renice, and terminate processes when necessary. Regularly checking OOM Killer logs is important for early detection of memory issues.

Frequently Asked Questions

What is load average and how should it be interpreted?

Load average shows the average system load over the last 1, 5, and 15 minutes. It should be compared with your CPU core count. On a 4-core server, a load average of 4.0 means 100% utilization. Consistently above the core count indicates overload.

The free command shows very little free memory, is there a problem?

No, Linux uses free memory as disk cache. The actual available memory is in the 'available' column of free -h output. If this value is low, then there's a memory issue.

Which process does OOM Killer terminate?

OOM Killer terminates the process with the highest oom_score, which is typically the one using the most memory. You can protect critical services by setting oom_score_adj to -1000.

CPU usage is 100% but I can't find which process, what should I do?

Press 'c' in top to display the full command line. Also press 'H' to see threads. If kernel threads (in square brackets) are using high CPU, it could be a hardware or driver issue.

Swap usage is very high, what should I do?

First identify which processes are using swap. If RAM is insufficient, upgrade your server plan. You can reduce swappiness (vm.swappiness=10) to make the system use swap less frequently. However, the real solution is providing adequate RAM.

Related Articles

SSH Connection Problem: Can't Connect to Server

SSH connection troubleshooting guide: Path.net firewall rules, Default Block, creating a filter rule for SSH port, and step-by-step troubleshooting.

8 min
sshconnection problemtroubleshooting
Network TrafficInbound— GbpsOutbound— Gbps