RE:NODE

Operations13 min read

How to read a server console and find the real cause

Startup, ready, warning and fatal look alike at a glance. How to find the ready line, read a stack trace upward, and spot the lines that only look fatal.

Updated

1 reader

The console is the server telling you exactly what happened, and it is the first thing any support team asks for because it is almost always sufficient. The problem is that a console is a thousand lines of narration with the important four buried in the middle, and every one of them is formatted the same. Reading it properly is a skill worth about twenty minutes to learn and then worth an hour every time something breaks. There are four techniques: find the line that says the server started, work out the severity of what you are looking at, read upward from the failure rather than at it, and recognise the dozen lines that look catastrophic and mean nothing.

The anatomy of a log line#

Most server logs are the same five fields in a different order.

code
[21:14:07] [Server thread/INFO]: Done (11.402s)! For help, type "help"[21:16:44] [Server thread/WARN]: Can't keep up! Is the server overloaded?[21:19:02] [Craft Scheduler Thread #3/ERROR]: Could not pass event to ShopPlus v3.2

Timestamp, thread, level, message. The thread is more useful than it looks: in Minecraft, Server thread is the main tick loop and anything that fails there affects everybody, while a Craft Scheduler Thread failing is a plugin's background task and usually affects one feature. Newer Paper builds print a shorter form with the level folded into the timestamp bracket, so do not be surprised when the shape differs slightly between versions.

Two things about the console specifically, as opposed to the log file behind it:

  • It is stdout and stderr merged. Ordering between the two streams can be slightly wrong around a crash, because they are buffered separately. If an error appears to arrive just after the line that should have caused it, that is why.
  • The timestamps are the server's clock, not yours. On a host that is very often UTC. When a player says "it broke at nine", establish whose nine before searching.

On a panel, the console is the server's own unfiltered output plus whatever the panel prints about power actions. On RE:NODE the noise that web servers generate - every bot on the internet probing /wp-login.php - is folded behind a count you can switch off, which is the difference between a readable log and a scrolling wall.

Find the ready line first#

Every server prints something when it has finished starting. Finding it is the single highest-value action in the whole process, because it splits the log into two halves and tells you which one to read.

ServerThe line that means it started
Minecraft (vanilla, Paper, Spigot)Done (11.402s)! For help, type "help"
Source engine (CS2, TF2, Garry's Mod)Connection to Steam servers successful. after the map loads
ValheimSession "Name" with join code 483920 and IP ... is active
Project Zomboid*** SERVER STARTED ***
PostgreSQLdatabase system is ready to accept connections
A Node or Python appWhatever you print after binding. If you print nothing, add it
NginxNothing. Silence on start is success

If that line exists, the server started, and your problem is everything after it. If it does not exist, everything printed is startup and the last few lines before the end are the reason it stopped. That one distinction resolves a large share of "my server will not work" tickets before any other thinking happens.

read upwardstill unclearit never got thereThe last linewhere it gave up10-40 lines upthe first thing you addedThe ready linedid it ever startThe startup blockconfig, ports, versions
Where to look in a console, in order

What a healthy startup looks like#

A startup block runs through the same stages in the same order every time, and knowing them means the point where your log stops is the diagnosis. A Minecraft server, lightly trimmed:

code
[21:13:52] [ServerMain/INFO]: Starting minecraft server version 1.21.1[21:13:52] [ServerMain/INFO]: Loading properties[21:13:52] [ServerMain/INFO]: Default game type: SURVIVAL[21:13:53] [ServerMain/INFO]: Generating keypair[21:13:53] [Server thread/INFO]: Starting Minecraft server on *:25565[21:13:53] [Server thread/INFO]: Using epoll channel type[21:13:54] [Server thread/INFO]: This server is running Paper version 1.21.1[21:13:55] [Server thread/INFO]: [LuckPerms] Enabling LuckPerms v5.4[21:13:56] [Server thread/INFO]: Preparing level "world"[21:14:01] [Server thread/INFO]: Preparing start region for dimension minecraft:overworld[21:14:04] [Server thread/INFO]: Preparing spawn area: 62%[21:14:07] [Server thread/INFO]: Time elapsed: 11123 ms[21:14:07] [Server thread/INFO]: Done (11.402s)! For help, type "help"

Five stages: identify and read config, bind the port, load plugins, load the world, finish. Where it stops tells you which one failed.

Where the log stopsWhat failed
Before Loading propertiesThe start command, the jar, or the Java version
At Starting Minecraft server onThe port is taken, or the bind address is wrong
Among the plugin enable linesThe plugin named immediately before the gap
At Preparing levelThe world files, or the disk they are on
At Preparing spawn area: N%Usually nothing. Generation is slow, not stuck - wait
After DoneNot a startup problem at all. Read forward instead

The Preparing spawn area row catches people on a first start with a new world, especially with a large view distance or a generation mod. It can legitimately sit at a percentage for minutes. Give it ten before declaring it hung, and if the percentage never moves at all, check disk.

Severity, in order#

LevelWhat it isHow often it is your problem
TRACE / DEBUGOnly present because somebody enabled itAlmost never
INFONarration. The server describing itselfAlmost never, however alarming the wording
WARNSomething is wrong but survivableWorth reading, rarely worth panic
ERRORAn operation failed. The server may still be runningOften
FATALThe reason it is not running any moreAlways

One caveat that matters: the level is chosen by whoever wrote the line, not by an authority. Plugin authors log routine startup chatter at ERROR and genuine catastrophes at INFO with dispiriting regularity. Treat the level as a hint about where to look first, not as a verdict. The same applies to the stream: plenty of well-behaved programs write banners, version notices and garbage collection logs to stderr, so a line appearing there is not by itself evidence of anything.

The other useful rule: the first ERROR of a session is worth more than the hundredth. Errors cascade. One plugin failing to load produces forty subsequent failures from everything that depended on it, and those forty are noise. Scroll to the earliest one.

Read upward from the failure#

The last line is where the server gave up, not where it went wrong. The cause is usually ten to forty lines above it, and it is the first line that names something you installed rather than something the game ships with.

Paste the fifty lines before the failure, not the last one. The last line is the least informative part of any crash.

Search for these, in this order, when you do not already know what you are looking for:

bash
$ grep -n -iE "fatal|exception|caused by|error" logs/latest.log | head -40$ sed -n '/Done (/,$p' logs/latest.log | head -100     # everything after startup$ tail -n 200 logs/latest.log

The second one is the most underused. Printing only what came after the ready line removes every startup warning from view at once, and startup warnings are the bulk of what misleads people.

Reading a Java stack trace#

Most game servers are Java, and a Java trace has a structure that rewards knowing it.

code
[21:19:02] [Server thread/ERROR]: Could not pass event PlayerInteractEvent to ShopPlus v3.2org.bukkit.event.EventException: null    at org.bukkit.plugin.java.JavaPluginLoader$1.execute(JavaPluginLoader.java:306)    at org.bukkit.plugin.EventExecutor.execute(EventExecutor.java:70)    at org.bukkit.plugin.RegisteredListener.callEvent(RegisteredListener.java:70)    ... 24 moreCaused by: java.lang.NullPointerException: Cannot read field "price" because "shop" is null    at net.example.shopplus.ShopListener.onInteract(ShopListener.java:128)    ... 27 more

Five rules read that in about ten seconds:

  1. The first line is a summary, and it usually names the culprit already. ShopPlus v3.2 is in it. Half of all traces are solved here.
  2. Frames are most recent first. The top at line is where execution was when it failed.
  3. Skip frames in the game's own packages. org.bukkit, net.minecraft, java.util - these are the messenger. Look for the first package that is somebody's plugin or mod.
  4. Follow `Caused by:` to the bottom. The chain runs from symptom to root, so the last Caused by is the actual cause. People read the first one and fix the wrong thing.
  5. `... 27 more` means identical frames were omitted. It is not truncation you need to recover.

The exception type alone is often the whole diagnosis:

ExceptionUsually means
NullPointerExceptionA bug, or a config value that is missing or misspelled
NoSuchMethodError, NoClassDefFoundErrorCompiled against a different version. The plugin and the server disagree
ClassNotFoundExceptionA dependency plugin is not installed at all
UnsupportedClassVersionErrorThe Java version is too old for this jar
java.net.BindExceptionThe port is already in use
java.lang.OutOfMemoryErrorThe heap is too small, or something is leaking into it
ConcurrentModificationExceptionUsually a plugin touching the world from the wrong thread

UnsupportedClassVersionError names both numbers, which decodes cleanly: class file version 52 is Java 8, 61 is Java 17, and 65 is Java 21. A message saying the jar is version 65 and the runtime understands up to 61 means you need Java 21. Minecraft JVM flags and Java versions has the mapping for every Minecraft release.

Other runtimes, other shapes#

The read direction is not the same everywhere, and getting it backwards wastes real time.

Node.js puts the error first and the frames below it, most recent first, and there is no Caused by chain unless a library builds one. The useful line is the top one.

code
Error: listen EADDRINUSE: address already in use :::3000    at Server.setupListenHandle [as _listen2] (node:net:1872:16)    at listenInCluster (node:net:1920:12)

The four you will actually meet: EADDRINUSE (the port is taken, often by the previous instance), MODULE_NOT_FOUND with Cannot find module (dependencies were not installed, or a case-sensitive path works on your Mac and not on Linux), ECONNREFUSED (something you depend on is not up), and an unhandled promise rejection, which terminates the process by default on current Node versions. Node.js memory limits covers the memory-shaped ones.

Python is the reverse of Java: the traceback frames run oldest first and the exception is the last line. In Python, the last line genuinely is the answer.

Source engine servers have no log levels at all - every line is a print. Startup is mostly noise about Steam and breakpad, and a crash typically ends with a signal and an invitation to add -debug to the run command to produce a debug.log. That log is where the useful part is; the console will only tell you it died.

Unreal-based servers (Satisfactory, Squad, The Isle) prefix lines with the subsystem, like LogNet: Warning:, and write a separate crash folder with its own log when they die. Read the crash folder, not the console.

Lines that look fatal and are not#

This list saves more time than anything else in the post. Every one of these is routine:

  • Can't keep up! Is the server overloaded? Running 2500ms or 50 ticks behind - a report that the server was slow for a moment, not a crash. It appears after every restart, every large save and every chunk-generation burst. It matters only when it is constant, at which point why TPS drops and what to do applies.
  • --- DO NOT REPORT THIS TO PAPER - THIS IS NOT A BUG OR A CRASH --- - Paper's watchdog early warning, printed because a tick is taking a long time. It is telling you about lag, and the banner exists precisely because it looks like a crash report.
  • WARNING: An illegal reflective access operation has occurred - a JDK warning about a library using an old mechanism. Harmless on the Java versions that print it.
  • Picked up JAVA_TOOL_OPTIONS: - the JVM confirming an environment variable. Informational.
  • Setting breakpad minidump AppID - a Source engine server configuring its own crash handler at startup. It means nothing has crashed.
  • Missing optional dependency notices from plugins, such as a permissions or economy API a plugin can work without.
  • Single connection errors in a web application log: someone's browser closed a connection mid-request, or a bot probed a path that does not exist. One is nothing. A thousand a minute is rate limits and abuse.

And the inverse - lines that look mild and are not: a single WARN about a config value being reset to a default (something in your file is invalid and is being silently ignored), a WARN about a world upgrade or data fix running (your save is being converted and there is no way back), and anything mentioning a backup being skipped.

The console as an input, and where the logs live#

The console is a two-way device. On RE:NODE it has a command line with history and tab completion, and anything you type goes to the server's standard input exactly as if you had typed it at the machine. stop in Minecraft, quit on a Source server, and the game's own admin commands all work. It is also the correct way to shut a game server down: a clean stop saves the world, a Kill does not.

Two cautions. First, whatever you type is in the log, so do not paste passwords, tokens or RCON credentials into it - use the Startup tab for variables and see environment variables and secrets and using RCON safely. Second, console scrollback is finite and is cleared on restart. The file on disk is neither.

ServerWhere the log is written
Minecraftlogs/latest.log, rotated to logs/YYYY-MM-DD-N.log.gz
Project ZomboidZomboid/Logs/, one file per subsystem per session
Source engine<gamedir>/logs/ only when logging is turned on in server.cfg
ValheimStandard output only, unless you pass -logFile
A Node or Python appNowhere, unless you redirect it or write one
bash
$ zgrep -i "outofmemory\|fatal" logs/*.log.gz$ ls -lht logs/ | head

Download the log before you restart, not after. The most common way to lose a diagnosis is to press Restart to "see if it happens again", which it does, having removed the evidence of the first time. Getting into the habit of keeping them is logs worth keeping.

When you open a ticket, send four things: the fifty lines before the failure as text rather than a screenshot, the exit code if you have it, the timestamp with a timezone, and what changed that day. On RE:NODE a ticket from the panel reaches all staff and takes private attachments, which is where a log with a token in it belongs. If the console ends abruptly with no error at all, that is its own diagnosis and why your game server keeps restarting covers it, usually alongside the memory graph.

FAQ#

The console is empty. What does that mean?

That the process produced no output, which normally means it never started. Check the Startup tab or start command, check that the main file exists and is not a half-finished upload, and check disk space. A container that cannot execute its entry point exits before anything is written.

What is the difference between ERROR and FATAL?

An ERROR is one operation that failed while the server carried on; a FATAL is the server stopping. A log can contain hundreds of errors and still be a healthy server, which is why "I see errors" is not on its own a diagnosis. Look for whether the ready line came after them.

How far back should I read before a crash?

Fifty lines as a default, and further if those fifty are all from the same cascade. What you are looking for is the first line that names something you installed. If the whole fifty are the game's own packages, keep going up.

My server prints warnings every few seconds but works fine. Ignore them?

Read each one once, then decide. Recurring warnings are usually a plugin complaining about a config value or an optional dependency. The ones not to ignore are those saying a setting was reset to a default, because that means something you configured is not in effect.

Why does the console stop scrolling when I am not watching?

Most panels cap the live buffer to keep the browser responsive, and the buffer is cleared on restart. The file on disk keeps everything. If you need history, take it from the file manager or over SFTP rather than from the scrollback.

Can I get console output into Discord?

Yes, though not from the panel itself - it is done with a plugin or mod on the server side that posts to a webhook, filtered to the lines you actually want. Send joins, leaves, deaths and errors; sending everything recreates the wall of text you were trying to escape.


Comments

Completely anonymous: no account, no email, no cookie. We store the name you type, the text and the time - nothing else. Links are limited and markup is not rendered.

0/2000