Service Auto-Restart Configuration with systemd
Configure automatic service restart with systemd for crash recovery, health checks, and monitoring in production environments.
Table of Contents
Introduction
In production environments, services crashing unexpectedly is inevitable. Properly configured auto-restart mechanisms minimize downtime and improve system reliability. This guide covers configuring service auto-restart with systemd.
Why Auto-Restart?
- Service crashes due to memory leak or unexpected error
- Temporary network issues affect the service
- Dependent service starts late and main service fails at startup
- Kernel OOM killer terminates the service
Step 1: Check Current Service Status
# Service status
systemctl status service_name
# Service logs
journalctl -u service_name -n 50
# Recent crashes
journalctl -u service_name --since "24 hours ago" | grep -i -E '(fail|error|crash|killed|exit)'
# Restart count
systemctl show service_name | grep -E '(NRestarts|ActiveState|SubState)'
Step 2: systemd Auto-Restart Configuration
[Unit]
Description=My Service
After=network.target
[Service]
Type=simple
User=www-data
ExecStart=/usr/bin/myapp
Restart=always
RestartSec=5
StartLimitIntervalSec=60
StartLimitBurst=5
[Install]
WantedBy=multi-user.target
Restart Options
| Option | Description |
|---|---|
Restart=no | No auto-restart (default) |
Restart=always | Restart in all cases |
Restart=on-failure | Restart only on failure |
Restart=on-abnormal | Restart on abnormal exit |
Restart=on-abort | Restart when killed by signal |
# Reload and start
sudo systemctl daemon-reload
sudo systemctl enable service_name
sudo systemctl start service_name
Step 3: Advanced Restart Configuration
Restart Limit
[Service]
Restart=on-failure
RestartSec=10
# Stop if fails more than 5 times in 5 minutes
StartLimitIntervalSec=300
StartLimitBurst=5
Memory Limit with Auto-Restart
[Service]
Restart=always
RestartSec=5
MemoryMax=512M
MemoryHigh=400M
Watchdog Health Check
[Service]
Restart=on-failure
RestartSec=5
WatchdogSec=30
NotifyAccess=main
Step 4: Application-Specific Configurations
Node.js Application
[Unit]
Description=My Node.js App
After=network.target
[Service]
Type=simple
User=nodeuser
WorkingDirectory=/var/www/app
ExecStart=/usr/bin/node /var/www/app/index.js
Restart=always
RestartSec=5
Environment=NODE_ENV=production
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
Docker Container
# docker-compose.yml
services:
app:
image: myapp
restart: unless-stopped
# Docker restart policy
docker update --restart=unless-stopped container_name
Step 5: Service Dependencies
[Unit]
Description=My Web App
After=network.target mysql.service
Requires=mysql.service
[Service]
Restart=on-failure
RestartSec=10
ExecStartPre=/bin/sh -c 'until mysqladmin ping -h localhost; do sleep 2; done'
Step 6: Troubleshooting
# Why isn't the service starting?
journalctl -u service_name -n 100 --no-pager
# Check restart loop
systemctl show service_name | grep NRestarts
# List all failed services
systemctl --failed
# Reset failed counter
systemctl reset-failed service_name
When StartLimitBurst is reached, systemd stops auto-restarting. Use systemctl reset-failed service_name to reset the counter.
Conclusion
Systemd auto-restart configuration significantly improves service reliability in production. Use Restart=always or Restart=on-failure for basic configuration, set the restart interval with RestartSec, and prevent infinite loops with StartLimitBurst. Monitor service status continuously with tools like Uptime Kuma.
Related Articles
SSH Connection Problem: Can't Connect to Server
SSH connection troubleshooting guide: Path.net firewall rules, Default Block, creating a filter rule for SSH port, and step-by-step troubleshooting.
RDP Connection Problem: Can't Connect to Windows Server
RDP connection troubleshooting guide: Path.net firewall rules, Default Block, creating a filter rule for RDP port, and step-by-step troubleshooting.
How to Run an MTR Test and Submit Results to Support
Guide to running MTR network tests with WinMTR and Linux MTR, creating ICMP filter rules, and submitting results to support. Diagnose packet loss and latency.