kelvin@cluxnei:~/blog$ cat 2gb/04-the-deploy.md

2026-09-20 EN ~20 min read ler em português

What you ship is not your working directory

The machine is legible, the process supervised, the front door handing out traffic. What is left is the question you have been putting off: how the code gets there. The answer everyone reaches for first is git pull in the directory that is live, and it hides the most treacherous class of bug there is, because it never reproduces on your machine.

The artifact is not your working directory

There is a difference between the code and what runs in production, and it disappears the moment you copy your working directory to the server. What travels along, uninvited:

• Compiled caches holding paths and package lists from your machine. A dependency injection container cache, a plugin discovery cache, a route cache. It lists a development package, production installed without dev dependencies, and the application comes up with Class not found on top of a file that should not have been there.
• The whole node_modules, compiled for your architecture and your Node version, which may not be the server's.
• Your local .env, pointing at your database, with your keys.
• .git, with the full history and, in plenty of repositories, secrets that were committed and later removed but still sit in the objects.
• Yesterday's build output, which nobody cleaned and which the server will happily serve again.

The pattern is always the same, and it is what makes this bug so expensive: it depends on something existing on your machine and not on the server. Locally everything works, because locally the missing piece is installed. The error only shows up in production and cannot be reproduced, because reproducing it would mean not having what you have.

The rule that fixes it is one sentence: what ships is a built artifact, and you declare what goes into it. In practice:

rsync -az --delete \ --exclude '.git' \ --exclude 'node_modules' \ --exclude '.env*' \ --exclude 'tests/' \ --exclude 'storage/framework/cache/' \ --exclude 'bootstrap/cache/' \ ./ deploy@server:/srv/myapp/releases/2026-09-20-1432/

One observation about that list: it is a denylist, and every denylist is wrong about some item you have not discovered yet. If your project allows it, prefer the opposite: build a clean directory, copy into it only what is needed, and ship that directory. Whatever was not invited does not go.

Numbered releases and the atomic swap

The layout that solves deploys, rollbacks and persistent files at once:

/srv/myapp/ ├── releases/ │ ├── 2026-09-18-2210/ │ ├── 2026-09-19-0930/ │ └── 2026-09-20-1432/ the new one ├── shared/ │ ├── .env secrets, outside every release │ ├── uploads/ what users sent │ └── storage/ └── current -> releases/2026-09-20-1432

current is a symlink, and it is what nginx and the systemd unit point at. Publishing means changing that link. And here is a detail almost every homegrown deploy script gets wrong.

The intuitive way is to delete the link and create it again:

rm -f current ln -s releases/2026-09-20-1432 current

Between those two commands, current does not exist. Any request arriving in that window finds a broken path. It sounds theoretical. I set up a reader opening current/index.html in a tight loop, and a swapper alternating between two releases by each method, six seconds each:

# rm + ln -s reads ok: 129,749 FileNotFoundError: 667,736 # ln -sfn + mv -T reads ok: 305,276 FileNotFoundError: 0

The first method failed on 84% of reads. To be fair to that number: the swapper was alternating non stop, which is not a real deploy, where the swap happens once. What the test shows is not the rate, it is that the window exists and it is wide. The second method did not fail once across three hundred thousand reads, and that is not luck:

ln -sfn releases/2026-09-20-1432 current.new mv -T current.new current

mv -T becomes a rename() call, and rename() is atomic by kernel guarantee. At any instant, current points at either the old release or the new one. There is no instant where it points at nothing.

One detail that cancels all of this: if the nginx config uses root /srv/myapp/current/public, it resolves the link per request and the swap works. But nginx keeps a cache of open descriptors once you enable open_file_cache, and then it goes on serving the old release for a few seconds. A systemctl reload nginx after the swap settles it.

Rollback in one command

This is why the old releases are kept rather than overwritten. Going back is the same atomic operation pointing the other way:

# back to the previous release previous=$(ls -1t /srv/myapp/releases | sed -n 2p) ln -sfn /srv/myapp/releases/$previous /srv/myapp/current.new mv -T /srv/myapp/current.new /srv/myapp/current systemctl restart myapp

Seconds, and no network. Keep five releases and delete the rest, because each one occupies disk and disk is part 1's other wall.

Rolling back code is cheap. Rolling back a database does not exist, which is why the migrations section is the most delicate part of this text.

Where to build, when the build does not fit

On a 2 GB VPS this is a real decision, not a preference. I measured two steps every JavaScript project has, with systemd counting the peak:

Two results worth pointing at. The first is that installing costs almost as much as compiling, which almost nobody plans for. The second is the disproportion of that second step: 438 MB of peak and 17 seconds of CPU to produce 48 KB.

Now put that next to part 1's budget. If your stack already uses 600 MB at rest, a 438 MB spike leaves you a few hundred megabytes from the wall. And if the build runs during a traffic spike, the build is not what dies: by part 1, the OOM killer picks the largest process, which is probably your database.

I ran the same build under part 2's ceiling, roughly what would be left on a busy box:

systemd-run --property=MemoryMax=256M ... npx webpack b-wp4.service: Failed with result 'oom-kill' b-wp4.service: Consumed 8.119s CPU time, 256M memory peak. bundle: NOT produced

It died, and it died alone. Nothing else on the machine was touched. That sorts the three options into order of preference:

1. Build elsewhere and ship the finished artifact. On your machine, on a CI runner, anywhere that is not the server currently serving traffic. The server receives files and compiles nothing. It is the best option and it is almost always available.
2. Build on the server, with a ceiling. If you really must, wrap the build in a systemd-run with MemoryMax. The build either finishes or dies alone and the deploy fails cleanly. A failed deploy is infinitely better than a site that goes down.
3. Build on the server, with no ceiling. The option almost everyone picks by default, and the one that trades a deploy error for a database outage.

Secrets that do not travel

The .env lives in shared/ and is linked into each release. It is never in the repository and never copied by rsync:

ln -sfn /srv/myapp/shared/.env /srv/myapp/releases/NEW/.env chown deploy:deploy /srv/myapp/shared/.env chmod 600 /srv/myapp/shared/.env

The 600 matters more than it looks on a machine shared with other services: the default 644 lets any system user read your database passwords. And in part 2 the systemd unit was already reading that file through EnvironmentFile, so the application receives the variables without them passing through the command line, where they would show up in ps for anyone.

The same goes for uploads/. If it lives inside the release, the first deploy erases everything users sent. It lives in shared/, and the release gets a link.

Migrations and the ordering nobody thinks about

Here is the part of a deploy that has no undo, and the mistake is always the same: treating code and schema as though they change at the same instant. They do not. Between running the migration and swapping the symlink there is a window where the old code is live against the new schema.

Renaming a column is the shortest example. You rename name to full_name, and the old code, still serving requests, looks for name and breaks on every request until the swap completes.

The way out is to make every change in two deploys, usually called expand and contract:

deploy 1, expand migration: add full_name, copy the data, keep name code: write to both, read full_name with a fallback deploy 2, contract (days later, once deploy 1 is stable) code: use full_name only migration: drop name

At every instant, the code that is live works with the schema that is in the database. It is more work, and it is the difference between a deploy that can be reverted and one that cannot.

Before any destructive migration, a dump. pg_dump into a dated file, before running it. It is the only thing between you and a column that does not come back. Part 5 covers backups that have been tested, which is a different story.

The database: the sum part 1 left open

In part 1 I measured that an idle Postgres costs about 20 MB, and that each idle connection adds around 2.5 MB. The other number was missing, the one that shows up when the connections are working.

work_mem is how much memory one sort or hash operation may use before spilling to disk. It is not a global budget. It is per operation, per connection. I measured with 20 concurrent connections each sorting 1.5 million rows, changing only that parameter:

Same connections, same query, seven times the memory. The 64 MB run reached the 3 GB ceiling I had given the container, so the real figure is at least that.

On a 2 GB VPS, work_mem = 64MB with twenty concurrent sorts does not make the site slow. It kills the machine. And it is exactly the tuning that shows up in every "optimise your Postgres" guide, written for dedicated servers.

The numbers that fit in 2 GB

shared_buffers = 256MB # ~12% of RAM. The 128MB default is fine too work_mem = 4MB # per operation, per connection. Raise it afraid maintenance_work_mem = 64MB # only for VACUUM and CREATE INDEX, one at a time max_connections = 30 # with a pool in front effective_cache_size = 1GB # allocates nothing, just guides the planner

The max_connections line is the one that frightens people coming from managed platforms, because 30 looks small. It is not, because a connection is not a user. One connection serves one query at a time, in milliseconds. What you need is a pool on the application side with a low limit, and every request borrows and returns.

If your stack is process per request, like part 2's PHP-FPM, every child opens its own connection, and then the connection limit and pm.max_children are the same number. Eight children, eight connections, and the sum closes itself.

The whole script

Put together, a complete deploy fits in twenty lines. It runs on your machine, not on the server:

#!/usr/bin/env bash set -euo pipefail SRV=deploy@server APP=/srv/myapp REL=$(date +%Y-%m-%d-%H%M%S) npm ci && npm run build # build here, not there rsync -az --delete \ --exclude '.git' --exclude 'node_modules' --exclude '.env*' \ ./ "$SRV:$APP/releases/$REL/" ssh "$SRV" bash -euo pipefail < $APP/shared/dumps/pre-$REL.sql npm run migrate --prefix releases/$REL ln -sfn $APP/releases/$REL $APP/current.new mv -T $APP/current.new $APP/current # atomic sudo systemctl restart myapp sudo systemctl reload nginx ls -1t releases | tail -n +6 | xargs -r -I{} rm -rf releases/{} REMOTE echo "published: \$REL"

No tool, no agent, no YAML. The set -euo pipefail on the first line is what makes this safe: on the first command that fails the script stops, and it stops before the mv -T. A deploy that breaks halfway leaves an incomplete release on disk and current pointing at the old version, which carries on working.

And the only sudo needed is the two service lines at the end, which you grant narrowly in sudoers instead of giving the deploy user blanket sudo.

The checklist

What comes in part 5

Everything is up and you can publish. What is left is the part that only hurts later, and it is the difference between a project that survives a year and one that does not.

Part 5 closes the series: backups you have restored at least once, because an untested backup is not a backup; log rotation, which is the number one way a small box dies; cheap monitoring, what is worth it and what is decoration at this scale; cron and the run that starts before the previous one finished; the precedence order of the four caches when you published and still see the old thing; and the final question, which is when 2 GB stops being enough and what the real next step is.

kelvin@cluxnei:~/blog$