Reading a server load graph: memory, CPU and disk

The most common misreading in hosting is a memory graph near the top of its range. For a Java server that is not a warning, it is the design: the runtime takes the heap it was given and uses it. A flat line at ninety per cent can be perfectly healthy, and a gentle sawtooth that never quite returns to its floor can be a server that will be dead by Thursday. The graphs on a panel are three or four coloured lines with no labels beyond a percentage, and almost everything useful in them is in the shape rather than the number. This is how to read that shape, what each line is actually counting, and the specific cases where the graphs are not where your problem is at all.

What the panel is actually measuring#

Before reading anything, know what is being sampled and how often, because both change what you are allowed to conclude.

Memory is read from the container's own kernel accounting, not from the game. It is the whole process group: the JVM heap plus everything outside it, or the Node process plus every buffer, plus anything else running in the container. It is not what -Xmx says, and it is not what the game reports in /mem or a spark summary. A Minecraft server told to use a 5 GB heap in a 6 GB container will show roughly 5.5 GB in the panel and 5 GB in the game, and neither number is wrong.

There is a subtlety worth knowing about: on cgroup v2, the raw memory counter includes page cache the container caused - file data the kernel is holding because it might be read again. Most panels report the figure that subtracts inactive file cache, which is the same one docker stats shows, but not all of them do. If your memory line jumps during a backup or a large file copy and drifts back down afterwards without anything crashing, you were watching cache, not the application.

CPU is a percentage of one core, and the plan's figure is the ceiling. 100 is one core, 250 is two and a half. A graph reading 210% is not broken; it means two cores' worth of work is happening. On RE:NODE that ceiling is a hard throttle to the share bought, so a line pinned to it is the limit doing its job, and a server sitting at 100% is slow rather than in trouble - it is never suspended for it.

Sampling interval is the limit on what any graph can tell you. A point on the chart is an average or a sample over some number of seconds. A spike that lasts 400 milliseconds does not appear at all. This matters enormously, because the thing players complain about - a stutter, a rubber-band, a freeze - happens on a millisecond scale, and resource graphs are a second-scale instrument at best. If the complaint is "it lagged for a moment", the graph is the wrong tool and the in-game tick metrics are the right one. What tick rate actually means covers those.

The memory line#

Memory is the line people act on most and understand least. Five shapes cover nearly everything.

Rises after start, then flat, well below the limit. Healthy. This is what a correctly sized server looks like and there is nothing to do.

Flat near the ceiling from the first minute. Almost always a pre-claimed heap. A JVM started with -Xms equal to -Xmx and -XX:+AlwaysPreTouch reserves and touches the whole heap immediately, on purpose, so the graph is pinned before a single player joins. It looks alarming and it is correct. Check the tick time: pinned memory with good tick times is just the flag set.

A regular sawtooth returning to the same floor. Garbage collection working normally. The floor is your live data; the teeth are the garbage between collections. Nothing to fix. Deep, frequent teeth with visible pauses mean the heap is a little small for the working set, which costs CPU.

A sawtooth with a rising floor. Each collection recovers less than the last. That is a leak, and it is the single most actionable shape on any graph. It will end in a crash at a time you cannot choose. For a Node application the equivalent measurement is in Node.js memory limits; for Minecraft it is usually a plugin, and the spark profiler will name it.

A vertical climb into the limit followed by a cliff to zero. An out-of-memory kill. The container reached its limit, the kernel stopped it, and the line restarts from the bottom as the server comes back. On RE:NODE that is deliberate behaviour - the container is stopped and restarted clean rather than allowed to swap and drag the machine down - and it means the console ends abruptly with no crash report. If you see this shape repeating, you have a restart loop rather than a crash: why your game server keeps restarting is the full diagnosis.

The CPU line#

CPU is the line people ignore and should not.

Spikes to the ceiling for a few seconds. A world save, a chunk generation burst, a backup starting, a plugin's scheduled task. Nearly always fine. A spike that coincides with a freeze players noticed is worth timing - if it happens every five minutes, it is your autosave interval and it can be moved or shortened.

Sustained at the ceiling for hours. The simulation cannot keep up, and no amount of memory will change it. This is throttling: your container is being stopped at its quota, repeatedly, mid-tick. With shell access you can prove it:

$ cat /sys/fs/cgroup/cpu.statnr_periods 940213nr_throttled 71204throttled_usec 902133945

nr_throttled divided by nr_periods is the fraction of scheduling windows in which you hit the cap. Above a few per cent, with the server feeling bad, you have a definite answer rather than a theory.

Pinned at exactly 100% on a plan that allows 200 or 250. The most misread state on any panel. One thread is saturated and the rest of the allocation is idle, which is the normal condition of a busy game server, because almost every engine runs its simulation on a single thread. Buying more cores does nothing here; a faster core does. CPU or RAM: which one is actually holding your server back works through the distinction properly.

Near the floor while everything feels slow. The server is waiting rather than working - on disk, on a network call, or on a lock. A plugin making a database query on the main thread produces exactly this: low CPU, terrible tick times. So does a failing DNS lookup with a timeout.

A regular pulse at a fixed interval. Something scheduled. Match the period against your autosave interval, your backup schedule, and the host's. If it is on the hour, so is everyone else's - moving your own schedules off :00 is free and occasionally noticeable. Cron expressions explained has the syntax.

The disk line#

Two different things get called disk, and only one of them is usually graphed.

Disk usage is gigabytes used against the plan's allocation. It moves slowly and it only matters at the top, where it matters a great deal: a server that cannot write cannot save, and a save interrupted by a full disk is how worlds get corrupted rather than merely lost. Treat 85% as the line at which you investigate, not 99%.

What fills a disk, in order of how often it is the answer:

• Log files that nothing rotates. Years of logs/*.log.gz is the single most common cause, and it is the easiest to fix.
• Crash reports and heap dumps, which are written at exactly the moment nobody is looking.
• In-place world backups - the ones the game itself writes next to the world, which are not backups in any useful sense because they share the disk with the thing they protect.
• Downloaded workshop or mod content that accumulates across versions.
• For applications: dependency trees duplicated per deploy, and build caches.

$ du -sh /home/container/* | sort -h | tail -20$ find /home/container -type f -size +100M -exec ls -lh {} \;

On RE:NODE the panel's own backups are stored off the machine they protect, so they do not consume the server's disk allocation - but anything the game writes into its own folder does. Backups that actually restore is the difference between the two.

Disk throughput - reads and writes per second - is rarely graphed on a panel and shows up as a symptom instead: a brief, total freeze on a schedule, with CPU and memory unremarkable. People report it as lag and buy CPU. What NVMe actually changes covers what storage speed does and does not fix.

A catalogue of shapes#

Reading three graphs together#

One line in isolation is a guess. The technique is always the same:

1. Get a timestamp. Ask when it was bad, to the nearest ten minutes. Without this you are looking at a whole day and everything is suspicious.
2. Find which line moved first. Not which line looks worst - which one changed before the others. Memory that climbs and then drags CPU up with it is a memory problem. CPU that saturates and then lets a queue build is a CPU problem.
3. Read the console at the same timestamp. The graph says something changed; the log usually says what. On RE:NODE the console and the graphs are on the same page, which means the same clock. Reading the console covers what to look for.
4. Ask what changed that day. A mod, a plugin, a config edit, a player count record, a game update. In that order, because that is the order of likelihood.

The fourth branch is the one that saves the most money. Three calm graphs and unhappy players means the problem is not in the resources you are being sold more of. It is the network path, one client, a neighbour on the machine, or a single blocking operation too short to appear in any sample. Latency, jitter and packet loss and shared CPU and noisy neighbours cover the two external cases.

Establishing your own normal#

There is no correct number, only a normal one for your server. A 4 GB Minecraft server at 3.6 GB is either perfectly healthy or two hours from a crash, and the graph alone cannot tell you which. What tells you is the comparison against the same server on a good evening.

So take the baseline deliberately, while things are fine:

• On a good evening, at peak, note memory, CPU and player count together. A screenshot with a date beats a memory of what it looked like.
• Note the same three numbers at 04:00, when nobody is playing. That is your idle floor, and the distance between idle and peak is the number that actually describes your headroom.
• Write them in a text file with the date. Panel history is finite, and the comparison you want in three months is with a week that has scrolled off.
• Repeat after any change worth naming: a new plugin, a version upgrade, a plan change, a world import.

Two data points a week apart with the same method beat any amount of staring at one graph. They are also the only evidence that makes an upgrade decision defensible rather than superstitious - when to upgrade your plan sets out the thresholds, and monitoring that tells you something covers turning this into something automatic.

A worked example of the whole method. A 6 GB, 2 vCPU Paper server, twenty regulars, complaints about evening lag.

memory  5.9 GB of 6.0 GB, flat from the first minute, no cliffscpu     40% overnight, 185% of an allowed 200% from 19:00 to 23:00disk    41 GB of 50 GB, up from 33 GB nine days ago

Read in order: the memory line is pinned but never falls off a cliff and starts pinned, so it is -Xms5G -Xmx5G with pre-touch and it is not the problem, however much it looks like one. The CPU line saturates precisely during the hours people complain, which is the answer to the question that was asked. And the disk line, which nobody asked about, is gaining roughly 900 MB a day and will hit the limit in about nine days, at which point the server stops being able to save. The lag complaint is a CPU problem; the thing that was about to cause an outage was 14 GB of unrotated logs.

When the graph is not where the problem is#

Four cases where the lines are honest and useless:

• One player is having a bad time. A routing problem between that player and the machine. Graphs cannot show it; a traceroute can.
• The stutter is sub-second. Below the sampling interval, so it is invisible by construction. Use in-game tick timings, which measure at the right resolution. For Minecraft that is /mspt and a profiler; why TPS drops and what to do is the long version.
• The server is waiting on something external. A database, a web API, a licence check. CPU low, memory flat, everything slow.
• The client is the problem. Especially with modded clients, where the player's own frame rate is the experience and the server is blameless.

The general rule: resource graphs tell you when a limit is being approached. They do not tell you why the simulation is slow. Those are different questions and they have different instruments, and buying a bigger plan to answer the second one is the most expensive mistake available on any hosting panel.

FAQ#

My memory is at 95%. Should I upgrade?

Not on that number alone. If it has been flat at 95% since the server started, it is a pre-claimed heap and it is fine. If it climbs to 95% over hours and then the server restarts itself, that is an out-of-memory kill and either the heap is set wrong or you genuinely need more. Check the heap setting against the container limit before spending anything.

Why does CPU show 200% when I only have 2 vCPU?

Because the figure is a percentage of one core, not of your whole allocation. Two vCPU is 200%, so a graph at 200% is a fully used allocation rather than a doubly overloaded one. It is the same convention top uses on Linux.

Is a spiky CPU graph bad?

Usually not. Spikes are saves, chunk loads, backups and scheduled tasks doing their job. What matters is whether the spikes coincide with something players notice, and whether the line ever comes back down. A plateau is a problem; a picket fence is a server working.

The graph looked fine while the server was lagging. What happened?

Either the lag was shorter than one sample, or it was not a resource problem at all. Tick lag happens in milliseconds; graphs sample in seconds. Measure with the game's own tick timings and compare the two clocks before concluding the graph is broken.

How long should I watch a graph before deciding anything?

Two peaks. One good evening to learn what normal looks like, one bad one to see what changed. A single afternoon of a single day will convince you of whatever you already suspected.

Does a restart fix a rising memory floor?

It resets the graph, which is not the same thing. A weekly restart is a reasonable way to live with a slow leak while you find it, and restart schedules that help covers doing it without annoying anyone. It is not a fix, and the interval between restarts will keep shrinking until you do fix it.