RE:NODE

Operations11 min read

Scheduled tasks worth having on every game server

The cron expression, the task order and the delays that make a schedule safe: warnings, saves, backups, dumps and restarts, with the commands per game.

Updated

0 readers

Most servers run with no schedules set at all, and most of the problems people report would have been prevented by four of them: a warning, a save, a backup, and a restart, in that order, every night, at an hour nobody is playing. That is roughly ten minutes of setup and it removes the "it gets slow after a few days" report, the "we lost an afternoon of building" report, and the "the server went down without warning" report in one go.

The tab that does this takes a cron expression and a list of ordered tasks with delays between them. Each task is one of three things: a console command, a backup, or a power action. Everything below is about which tasks, in which order, with what gap, and the exact commands for the games where the command is not obvious.

How a schedule is built: cron, tasks and delays#

A cron expression is five fields separated by spaces. It is not a time; it is a pattern that a clock is compared against every minute.

FieldRangeNotes
Minute0-59
Hour0-2324-hour clock, no am/pm
Day of month1-31
Month1-12 or JAN-DEC
Day of week0-6 or SUN-SAT0 is Sunday; 7 is usually Sunday too
code
0 5 * * *      05:00 every day15 4 * * *     04:15 every day0 */6 * * *    every six hours, on the hour*/15 * * * *   every fifteen minutes0 5 * * 1      05:00 on Mondays30 3 * * 0     03:30 on Sundays0 5 1 * *      05:00 on the first of the month

Two traps. First, */5 in the minute field means every five minutes; a bare 5 means at five minutes past the hour, once. Getting those the wrong way round produces either a job that never seems to run or one that runs 288 times a day. Second, when both the day-of-month and the day-of-week fields are restricted, cron runs the job when either matches, not both. 0 5 13 * 5 is not "Friday the 13th", it is "the 13th of every month, and every Friday". Leave one of them as * unless you mean that.

Time zone matters more than people expect. The schedule runs on the panel's clock, not your laptop's, so check what the interface shows before assuming 05:00 is your 05:00. And avoid scheduling anything between 02:00 and 03:00: in a zone that observes summer time, jobs in that window run twice on one night of the year and not at all on another. 04:00 and 05:00 are safe. Cron expressions explained takes the syntax apart in more detail, including the shorthands, which are not supported everywhere - write the five fields and you are never surprised.

The delays are the other half. A schedule is a list of tasks in order, each with a wait before it runs. That wait is what turns a set of commands into a safe sequence, because a save that has not finished when the backup starts is the exact failure the backup was meant to prevent.

The nightly sequence, minute by minute#

Here is the full arrangement for a Minecraft server as one schedule on 0 4 * * *, with the delays doing the work.

TaskDelay beforeWhat it does
Command: say Restart in 15 minutes0People can get somewhere safe
Command: say Restart in 5 minutes600sThe one that actually gets read
Command: save-off240sStops automatic writes while the archive is made
Command: save-all flush10sPuts everything currently in memory on disk
Backup60sArchives a world that is not being written to
Command: save-on600sRe-enables saving whatever happens next
Power: restart60sClean stop, which saves again on the way down

The principle is simple: announce, quiesce, copy, release, restart. The numbers are yours to tune - a 3 GB world needs longer between the backup task and save-on than a 300 MB one - but the order does not change.

Two mistakes worth naming. Taking the backup at the same minute as the restart gives you a backup of a server that is shutting down. And running save-off without a matching save-on later leaves the server unable to save until it restarts, which is fine on this schedule and catastrophic on one that does not end in a restart. If you are unsure, leave save-off out entirely: save-all flush alone is a large improvement over nothing.

The four schedules most servers should have#

Strip everything else away and these are the ones that pay for themselves.

  1. A nightly restart. Long-running game servers accumulate: entities, chunk data, memory fragmentation, plugins that leak slowly. A restart at 05:00 costs nobody anything and resets all of it. This single schedule prevents more "it gets slow after a few days" reports than any hardware change. What it hides, and when it is the wrong answer, is the subject of restart schedules that help.
  2. A backup before it. Not at the same moment - before, with a save in between, as above. On a two-slot plan this rotates: last night overwrites the night before, which is why the second slot should hold a locked known-good copy rather than another nightly one. Backups that actually restore covers the retention maths.
  3. A warning before that. A restart nobody was told about is indistinguishable from a crash, and players report it as one. Two messages, fifteen minutes and five minutes out, are enough.
  4. Something specific to your game. The save command, the wipe, the database dump, the map rotation. The table below is the starting point.

A fifth is worth adding once the first four are running: a weekly job that does something you would otherwise forget. Pruning logs, rotating a world backup into a locked slot, or running the restore drill reminder. Logs worth keeping has the retention side of that.

Commands worth scheduling, by game#

The panel console sends what you type to the server's standard input or its RCON, depending on the game. Where a game has no console interface, the only tasks available are backup and power action - and that is not a limitation of the panel, it is the game.

GameCommandNotes
Minecraftsave-all flush, save-off, save-on, say <text>flush forces the write before returning
PalworldSave, Broadcast <text>, Shutdown <seconds> <text>RCON; Broadcast does not accept spaces in some builds
Project Zomboidsave, servermsg "<text>", quitquit saves and exits cleanly
7 Days to Diesaveworld, say "<text>", shutdownAlso reachable over telnet
Terrariasave, say <text>, exitVanilla server console; TShock adds more
Factorio/server-save, /playersSlash commands on the headless console
ValheimnoneNo console commands and no RCON: backup and restart only
DayZ, Arma 3none as standardRestart on a schedule; mission or mod tools may add more
Source gamesno save to takeRestart plus log handling is the whole schedule

Valheim is the one that surprises people. The dedicated server has no command interface at all, so the flush step is impossible and the schedule is backup then restart, relying on the fact that a clean stop saves the world on the way down. It also means the backup itself can be up to a full -saveinterval behind the live game, which is the argument for lowering that value - see the Valheim dedicated server guide.

For a Rust server, the wipe goes on its own cadence rather than the nightly one, and it is as much a community decision as a technical one: Rust wipes without losing your players covers the timing. For anything with a database behind it - a FiveM framework, a Minecraft economy, a web app - the dump belongs on its own schedule, a few minutes before the backup so the archive picks it up.

Schedules for app, web and database servers#

Game servers get most of the attention, but the same three task types cover a lot of application work.

  • The database dump. A console command that runs pg_dump or mongodump into the server's folder at 04:05, so that the 04:15 backup archives it and takes it off the machine. The commands and flags are in database backups and restores.
  • The framework's own scheduler. Laravel's schedule:run expects to be called every minute, which is a legitimate use of */1 * * * * and one of the few jobs that should run that often. Django and Rails equivalents are the same shape.
  • Log pruning. Anything that writes a log file forever will eventually fill the disk, and a full disk is a much worse incident than a slow server. A weekly job that deletes files older than fourteen days costs nothing.
  • A restart, only if it is earning its place. A Node or Python process that is healthy does not need a nightly restart, and adding one hides the leak you should be fixing. If memory grows every day, read Node memory limits explained before scheduling around it.

One thing not to schedule: certificate renewal. On the proxy slot the certificate is issued and renewed automatically inside a 21-day window, and a cron job poking at it does nothing useful.

Checking that a schedule actually ran#

Silent failure is the normal failure mode for scheduled work. Cron does not complain; it simply does not produce anything, and a schedule that has been failing for six weeks looks exactly like one that has been working.

Three checks, in order of effort:

  1. Look at the backup list. The timestamps are the cheapest proof you have. If the newest archive is from eleven days ago, you already know.
  2. Watch the size. An archive half the size of last week's means something stopped being included. Growth is normal; sudden shrinkage never is. This is the earliest warning any of this gives you, and it only works if somebody looks - monitoring that tells you something is about making that automatic rather than heroic.
  3. Read the console at the scheduled hour, once. Set the schedule to five minutes from now, watch the commands land, then set it back. It takes a few minutes and it catches the wrong command name, the wrong quoting and the wrong assumption about which console the game listens on. Reading the console explains what the output should look like.

Watch out for % if you are writing cron on a VDS rather than in a panel: in a crontab, an unescaped % terminates the command and everything after it becomes standard input. A dated filename like $(date +%F) has to be written $(date +\%F) there. This single character accounts for a remarkable share of backup jobs that produce nothing.

If you run several servers, stagger them. Ten backups all starting at 04:00 on the same node compete for the same disk and each takes longer than it should. Spread them across the hour: 5 4, 20 4, 35 4, 50 4. The same applies to restarts, and it has the pleasant side effect that not every one of your communities goes dark at the same instant.

When a schedule is the wrong tool#

Automation removes work; it does not remove the reason for the work. Three cases where adding a schedule makes things worse:

  • Restarting to mask a leak. Legitimate as a stopgap, dishonest as a strategy. Write down what it is hiding; that note is the bug report. Why your game server keeps restarting separates the causes.
  • Backing up more often instead of backing up properly. Six backups a day into two slots means your oldest recoverable state is eight hours ago. A problem introduced yesterday is now unrecoverable. Depth beats frequency once you are past one a day.
  • Scheduling around a crash. A restart every four hours because it falls over every five is not maintenance, it is a countdown. It also runs into the crash watcher: three restarts in an hour that you did not ask for raise a warning on the server page and open a ticket automatically, and scheduled restarts are not counted, so the ones being counted are real crashes.

FAQ#

What time should the nightly restart run?

The hour with the fewest players, which for a European community is usually between 04:00 and 06:00 local. Check your own numbers rather than copying a default: a server whose players are mostly in another time zone has its quiet hour somewhere else entirely. Avoid 02:00 to 03:00 because of summer time changes.

How many schedules can one server have?

As many as you need; the practical limit is your own clarity. One schedule holding an ordered sequence of tasks is easier to reason about than five schedules that happen to run near each other, so prefer delays within a single schedule to separate cron lines a few minutes apart.

Does a scheduled restart count against crash detection?

No. The watcher counts restarts you did not ask for - a process that exited, or uptime that went backwards. Restarts you scheduled or pressed are not counted, which is why a nightly schedule does not trip the three-in-an-hour warning.

Can a schedule take a database dump?

Yes, as a console command task, as long as the tools are available in the server's environment. Write the dump into the server's own folder and put a backup task a few minutes later so the archive carries it off the machine. Dumping to a location the backup does not cover is the most common way this ends up useless.

What happens if a task is still running when the next one starts?

Nothing good, which is what the delays are for. A backup of a 20 GB server does not finish in sixty seconds, and a restart landing on top of it produces a partial archive. Time your own tasks once and set the delays from the measured numbers rather than from hope.

Should I schedule a backup before every update?

Schedule the nightly one and take the pre-update one by hand. Updates do not happen on a timetable, and the backup you want before a risky change is one you take deliberately and then lock, so that the nightly rotation does not push it out of your slots. That locked copy is the one you will actually use - see testing a restore before you need it for proving it works.


Comments

Completely anonymous: no account, no email, no cookie. We store the name you type, the text and the time - nothing else. Links are limited and markup is not rendered.

0/2000