Redis: when you need it, and how to run it yourself

Redis is fast, which is why it gets added to projects that did not need it. It is a second service to install, secure, monitor, size and lose data from, and it should earn its place by doing a specific job that nothing you already run does well. "My database is slow" is not that job; that is usually a missing index.

There are four jobs it is genuinely the right tool for, and they all have the same shape: small pieces of state, shared between processes, that you would not mind losing on a bad day. Sessions, caches, queues and counters. If your reason is on that list, add it. If it is not, this post will save you a daemon. And if you do add it, you will be running it yourself - there is no Redis product here, so the second half of this post is how to install, configure and secure one on a machine you have root on.

What Redis actually is#

Not a cache. A cache is one of the things you can build with it. Redis is an in-memory data structure server: it keeps your data in RAM, optionally writes a copy to disk, and speaks a small text protocol on TCP port 6379. Commands operate on typed structures rather than on opaque blobs, and that is where the usefulness comes from.

Command execution is single-threaded. Every command is therefore atomic without you doing anything, which is exactly why it is so good at counters and locks - and also why one slow command blocks everything else on the server. KEYS * on a million keys is not a slow query, it is a short outage. Use SCAN instead, always.

The other property that defines it: everything lives in memory. Disk is a copy, not the store. That is the source of both the speed and the risk, and it is the reason "can I lose this?" is the first question to ask about anything you put in it.

The four jobs worth adding it for#

Sessions shared between processes. The moment you run more than one application process, in-process session state stops working: a user logs in on worker 1 and is anonymous on worker 2. A shared session store fixes it in one line of configuration. This is the most common honest reason to add Redis.

A cache for something genuinely expensive. Expensive means a query that cannot be indexed away, a report that aggregates a million rows, or an API call to someone else that you pay for. The pattern is cache-aside: look in the cache, and on a miss compute the value and store it with a TTL you have thought about.

const key = `v1:report:${teamId}:${day}`;let report = await redis.get(key);if (!report) {  report = JSON.stringify(await buildReport(teamId, day));  await redis.set(key, report, "EX", 900);   // 15 minutes}

Two details make the difference between a cache and a liability. Put a version prefix in the key, so that changing the shape of the value is a prefix bump rather than a flush. And give everything a TTL, even things you think never change, because a cache with no expiry is a second source of truth that nobody is maintaining.

A queue for work that should not happen during a request. Sending email, resizing an image, calling a slow third party. The request writes a job and returns; a worker process picks it up. BullMQ on Node and Celery or RQ on Python all sit on Redis and all do the retry, delay and dead-letter handling you would otherwise write badly.

Counters that must be shared. Rate limiting is the canonical case: an in-process counter multiplies your limit by the number of workers and resets on every deploy. INCR with an expiry is two commands and correct across every process. Rate limiting: where to put limits covers what to count and what to key it on.

Notice what is not in that diagram: anything you would mind losing. The database still owns the truth.

The reasons that are not reasons#

"The database is slow." Find out why first. Run the query with EXPLAIN ANALYZE, look for a sequential scan over a big table, add the index, and measure again. Caching an unindexed query hides it until the cache is cold - which happens at a deploy, at a restart, or at exactly the moment traffic spikes and every miss arrives at once. Reading EXPLAIN ANALYZE is a cheaper afternoon than operating a cache.

"Everybody uses it." Everybody also has more than one server, an ops rota and a staging environment.

"It will be faster." A local Redis round trip is a few tenths of a millisecond. So is a primary-key lookup in PostgreSQL against a warm cache. If your page takes 800 ms, the database round trip is not the 800 ms.

"We will need it eventually." Adding it later takes an afternoon. Operating it for a year you did not need it takes a year.

And one genuine danger rather than a weak reason: storing anything you cannot afford to lose. Redis can be configured to persist, but it can also be configured not to, a cache with an eviction policy will throw your data away by design when memory is tight, and the default persistence settings lose up to a second of writes on a hard stop. If the answer to "what happens if this key vanishes" is "a customer complains", it belongs in PostgreSQL or MongoDB, not here.

What you already have without it#

Before adding a service, check whether the ones you run already do the job. Usually they do, at a scale far past where most projects ever get.

• Sessions on one process. In-process memory is correct and fastest. On several processes, a signed cookie carrying the session itself needs no server state at all, and a database-backed session table handles thousands of users without noticing.
• Caching in front of the page. Cache-Control, ETag and a properly configured web server keep requests away from your application entirely, which is faster than any cache your code can consult. HTTP caching headers explained covers the ones that matter.
• An in-process cache. A bounded LRU inside the application - lru-cache on Node, functools.lru_cache or cachetools on Python - costs nothing, needs no network hop, and is right for anything small, hot and identical across users, such as configuration or a lookup table. Its limit is that each process has its own copy.
• A queue in PostgreSQL. SELECT ... FOR UPDATE SKIP LOCKED gives you a correct, transactional work queue in a table, and it has the property Redis does not: enqueueing the job and committing the data that caused it happen in one transaction. At a few hundred jobs a minute it is entirely adequate, and background jobs on a small server walks through it.
• Rate limits in the web server. limit_req in nginx enforces a limit before your application is involved and needs no store at all.

-- A queue that needs no second serviceUPDATE jobs SET status = 'running', started_at = now()WHERE id = (  SELECT id FROM jobs  WHERE status = 'queued' AND run_after <= now()  ORDER BY run_after  FOR UPDATE SKIP LOCKED  LIMIT 1)RETURNING id, payload;

Installing Redis on a VDS#

RE:NODE does not sell Redis: the database line is PostgreSQL and MongoDB, and the application plans run the start command from your repository rather than a second daemon. So Redis is something you run on a machine you have root on, which means a VDS or dedicated server. That is not a downgrade - a single Redis instance is one of the easier things to operate, and on your own box it can listen on the loopback interface only, which removes most of the ways it goes wrong. What you take on in exchange is the whole machine: patches, firewall, monitoring and backups, weighed up in choosing between a VDS and a game panel.

On Debian or Ubuntu:

$ sudo apt update && sudo apt install redis-server$ sudo systemctl enable --now redis-server$ redis-cli pingPONG

The distribution package puts the configuration at /etc/redis/redis.conf, runs it under systemd as the redis user, and already binds it to localhost. If you need a newer version than your distribution ships, the Redis project publishes its own apt and rpm repositories; on a box that already runs containers, docker run with a named volume is just as reasonable - see Docker on a VDS.

The settings worth changing on day one, all in /etc/redis/redis.conf:

bind 127.0.0.1 -::1protected-mode yesport 6379requirepass a-long-random-string-from-a-password-managermaxmemory 512mbmaxmemory-policy allkeys-lruappendonly yesappendfsync everysecsave 900 1save 300 10save 60 10000

Then two kernel settings Redis will complain about in its own log if you skip them. Memory overcommit must be enabled, or a background save can fail on a box with little free memory; and transparent huge pages cause latency spikes.

$ echo 'vm.overcommit_memory = 1' | sudo tee /etc/sysctl.d/99-redis.conf$ sudo sysctl -p /etc/sysctl.d/99-redis.conf$ cat /sys/kernel/mm/transparent_hugepage/enabled

Check the log after a restart with journalctl -u redis-server and fix anything it warns about there rather than waiting to meet it under load. The first hour on any new machine has a wider checklist - updates, a non-root user, SSH keys, a firewall - in first hour on a new VDS, and keeping your own application alive beside it is systemd services for your apps.

One note on the name. Redis changed its licence in 2024, which is why Valkey exists as a fork under the Linux Foundation, and the licensing moved again in 2025. For a single self-hosted instance none of it changes anything practical: Valkey speaks the same protocol on the same port with the same commands and the same configuration file, and your client library will not notice which one it connected to.

Persistence: RDB, AOF, and what you can lose#

Redis offers two mechanisms, and understanding the difference is the difference between an informed risk and a surprise.

RDB snapshots write the whole dataset to a single file at intervals, controlled by the save lines above - "after 900 seconds if at least 1 key changed", and so on. The snapshot is taken by forking the process, so it is cheap in CPU and potentially expensive in memory: the copy-on-write child can, in the worst case, need almost as much memory again. Restoring is fast and the file is easy to copy elsewhere.

AOF, the append-only file, logs every write command as it happens and replays the log at start-up. appendfsync everysec is the default and the sensible setting: you can lose up to one second of writes on a hard stop. always fsyncs on every write and is much slower. no leaves it to the kernel and can lose tens of seconds.

Run both. RDB gives you a compact file to take away as a backup; AOF gives you a much smaller loss window. Neither makes Redis a system of record: a clean shutdown saves, and a kill, a power cut or the kernel stopping the process out of memory does not. That is the same arithmetic as any other write-behind store - the data that must survive belongs where a transaction committed it, and the copies that must survive belong off the machine, as database backups and restores argues at more length.

Memory, maxmemory and eviction#

maxmemory is the most important setting in the file, and its default is unlimited, which means Redis will grow until the kernel intervenes. Set it explicitly to something well below the machine's memory - half is a reasonable starting point when persistence is on - and choose a policy that matches the job.

The trap is in the last two rows. A volatile-* policy only considers keys that have an expiry set. If nothing in your database has a TTL, it behaves exactly like noeviction and your writes start failing with an out-of-memory error while the server looks half idle. Either set TTLs on everything cacheable, or use allkeys-lru and keep the things you must not lose somewhere else.

Sizing is measurement, not arithmetic. Each key carries roughly a hundred bytes of overhead before its value, so a million tiny keys is not a small dataset. The tools:

$ redis-cli info memory | grep -E 'used_memory_human|maxmemory_human'$ redis-cli info stats | grep -E 'evicted_keys|keyspace_hits|keyspace_misses'$ redis-cli --bigkeys$ redis-cli --latency

The hit ratio - hits divided by hits plus misses - is the number that says whether the cache is worth its existence. Below about 80 per cent, something is wrong with your TTLs or your keys. A rising evicted_keys means maxmemory is too low for the working set, and that is a real signal, unlike CPU graphs. Monitoring that tells you something is about choosing exactly these kinds of numbers.

Securing it, and the mistake that gets servers mined#

An unauthenticated Redis reachable from the internet is not a leak, it is a shell. The command set includes changing the working directory and the dump filename, so an attacker can write an arbitrary file as the Redis user - the classic version writes an SSH key into authorized_keys and logs in. Automated scanners find open instances within minutes of them appearing, and what they install is usually a miner.

The defence is short and it is not optional:

1. Bind to localhost unless something on another machine genuinely needs it. bind 127.0.0.1 -::1 is the default in most packages; leave it alone. If another host must connect, use a private interface or an SSH tunnel, never the public address.
2. Set `requirepass` to something long and random, and keep it in your application's environment rather than in the repository - environment variables and secrets covers where it should live.
3. Close the port at the firewall. Default deny inbound, allow the handful of ports you meant to publish, and 6379 is not one of them. A UFW firewall guide has the rules.
4. Use ACL users for anything beyond a single application. Redis 6 and later can give each client a username with a restricted command set, so your web app cannot call CONFIG or FLUSHALL even if it is compromised.
5. Never expose it through a reverse proxy. Redis speaks its own protocol, not HTTP, and putting it behind a web server does not make it safe.

Leaving protected-mode yes in place is a useful backstop: with no password and no explicit bind, Redis refuses connections from anywhere but the loopback interface. It is a seatbelt, not a strategy.

FAQ#

Do I need Redis for a small website?

Almost certainly not. One application process with an in-process cache, HTTP caching headers in front, and a properly indexed database covers a site with thousands of daily visitors. Add Redis when you have more than one process that must share state, or a job you want off the request path.

Can I use Redis as my main database?

You can, and most people should not. It has persistence, but the default settings lose up to a second of writes on a hard stop, an eviction policy can delete keys by design, and everything must fit in memory. Use it for state you can rebuild, and keep the records that matter in a database that commits transactions to disk.

Where do I run Redis if my host does not sell it?

On a machine where you have root: a VDS or a dedicated server. Install the distribution package, bind it to localhost, set a password and a maxmemory limit, and let your application connect over the loopback interface. That is also the most secure configuration available, because the port is never exposed at all.

Redis or Valkey?

For a self-hosted single instance, either. Valkey is the Linux Foundation fork created after the 2024 licence change; it speaks the same protocol, uses the same configuration file and works with the same clients. Pick whichever your distribution packages and move on.

How much memory should I give it?

Enough for your working set plus overhead, and no more than about half the machine, because a background save forks the process and the copy can briefly need a lot of extra memory. Set maxmemory explicitly, watch evicted_keys, and raise it when that number starts climbing.

Why are my writes failing with an out-of-memory error?

Because maxmemory has been reached and the policy will not evict anything. Either the policy is noeviction, or it is a volatile-* policy and none of your keys have a TTL, which amounts to the same thing. Set TTLs, switch to allkeys-lru, or raise the limit.