High CPU usage on a Linux server: find the cause and fix it

Your server's CPU is pinned or its load average is climbing. How to find the process responsible, read us, sy, wa and st correctly, tell a busy server from a stuck one, and when to add cores.

· 6 min read · By the Spot Downtime team

A CPU alert, a slow site and a load average in double digits. High CPU is either honest work (more traffic than the server can serve) or waste: a runaway process, a busy loop, a cron job that overlaps itself, or a miner someone installed for you. Five minutes with the right tools tells you which.

1. What's using it?

bash
top -o %CPU          # press 1 to see each core, c for full commands
ps -eo pid,user,%cpu,%mem,etime,cmd --sort=-%cpu | head -15

htop is easier to read if it's installed. A process at 100% uses one full core; on a 4-core server, 400% is everything. Note the user and the full command line: it tells you whether this is your app, a database, a backup or something you don't recognize.

2. Read the CPU line

The summary line in top splits CPU time. Which part is high matters more than the total:

FieldMeaningWhen it's high
usYour programsApp code, a database query, compression: find the process
syThe kernel on their behalfLots of small I/O, network packets or process creation
waWaiting for the diskNot a CPU problem: the disk is the bottleneck
stStolen by the hypervisorYour VM's neighbours, or a burstable instance out of CPU credits
si / hiInterruptsHeavy network traffic, sometimes an attack

Steal time on cloud servers

On burstable instances (AWS t3 and t4g, some small VPS plans), you get a baseline share of a core plus credits for bursts. When the credits run out, the hypervisor holds you to the baseline and st climbs. The fix is a bigger or non-burstable instance, not your code.

3. Load average vs CPU

uptime shows load over 1, 5 and 15 minutes: the number of processes running or waiting to run, plus those waiting for the disk. Compare it with the number of cores (nproc): load 4 on 4 cores is fully used; 12 on 4 cores means work is queueing. A high load with low CPU use means processes are stuck waiting for I/O, often a slow or failing disk or a network filesystem that stopped answering.

Spot Downtime's CPU figure counts time waiting for the disk (wa) as idle, so a server can show low CPU and a high load at the same time. That combination points at storage.

Common causes

Traffic, really

If CPU rises with requests and falls with them, the server is doing its job and needs help: caching (a page cache or CDN in front), fewer queries per request, more workers on more cores, or another server behind a load balancer.

A runaway process

A process stuck at 100% of one core for hours, while the app is otherwise idle, is a busy loop. Capture what it's doing before you restart it, or it'll come back: sudo py-spy dump --pid PID for Python, jstack PID for Java, sudo perf top -p PID for anything.

Overlapping scheduled jobs

A cron job that takes longer than its interval starts a second copy, then a third. Look for many copies of the same command in ps. Overlapping cron runs covers locking with flock.

The database

postgres or mysqld at the top means queries. pg_stat_activity or SHOW PROCESSLIST shows what's running now; a missing index on a hot query is the usual answer.

Something that shouldn't be there

An unfamiliar process name, a binary in /tmp or /dev/shm, or a process using every core under a web server's user is a sign of compromise, often a crypto miner. Isolate the server, find how it got in (an exposed admin panel, a weak SSH password, an unpatched app), and rebuild rather than clean.

Inside containers

docker stats shows CPU per container; 200% means two full cores. A container limited with --cpus is throttled at its limit, which looks like slowness rather than 100% on the host.

Alerts that don't cry wolf

  • A CPU spike for a few seconds is normal. Spot Downtime's server alerts use the average over a window (5 minutes by default), so only sustained load alerts you.
  • The limit is 90% by default. Lower it for servers that should be quiet, raise or turn it off for batch servers that are meant to be busy.
  • Each running container's CPU is on the server's Containers table, which finds the culprit on a Docker host at a glance.

Set it up with the server agent. Memory alerts are a different story: see out of memory on Linux.

Get new guides by email

Practical posts on monitoring, outages, error fixes and on-call, when they're published. No spam.

Unsubscribe from any email with one click. See our privacy policy.

Keep reading