kelvin@cluxnei:~/blog$ cat 2gb/03-the-front-door.md

2026-09-20 EN ~20 min read ler em português

One IP, one port 443, several applications

At the end of part 2 your application was running, coming back on its own and carrying a memory ceiling. And it was still invisible, because it listens on 127.0.0.1:3000 and nobody types port 3000 into a browser. The missing piece receives the whole world on a single port and hands it out. This part is about that piece, with four measured experiments, one of which explains why the 2 GB sum works at all.

Why a proxy, even with one app

It is a fair question. If you have one application, why not let it listen on port 443 directly and be done?

Because port 443 is a single resource on the machine and everything that needs it will fight. The moment a second application shows up, or a second domain, or a folder of static files, you will have to put something in front anyway. But even with one app, the proxy solves five things you would otherwise write yourself:

• TLS. Certificate, renewal, accepted protocols, all in one place instead of inside every application.
• Privileged ports. Binding 443 needs root. With a proxy the root is its, and your application keeps running as deploy on a high port.
• Static files. Measured below: the proxy serves nearly twice the requests per second and frees memory.
• Slow clients. nginx takes the whole request before handing it over. A bad 3G connection occupies nginx, not one of your application's workers.
• Limits. Upload size, timeouts, rate per IP, security headers. All outside the code.

Several sites, one IP

One IP, and the browser says which site it wants in the Host header. nginx picks the server block by server_name. One file per site in /etc/nginx/sites-available, symlinked into sites-enabled:

server { listen 443 ssl; server_name app.mysite.dev; location / { proxy_pass http://127.0.0.1:3000; } } server { listen 443 ssl; server_name site.mysite.dev; root /var/www/site; }

Two things that save an afternoon:

The first block listening on a port is that port's default server. Anyone arriving with an unknown Host, or straight by IP, lands there. If that bothers you, declare an explicit block with default_server returning 444, which closes the connection without answering anything.

And nginx -t before every reload. It validates the whole configuration without applying it. A reload with a broken config does not take nginx down, it refuses and carries on with the old version, but you walk away thinking you applied something.

The matching order is not top to bottom

This section does the most damage, because the symptom does not point at the cause.

Everyone's intuition is that nginx reads location blocks in file order and uses the first one that matches. That is wrong. The real order is by kind of location, and the file has nothing to do with it:

1. location = /exact exact match, beats everything 2. location ^~ /prefix prefix that blocks regexes 3. location ~ regex regex, in file order location ~* regex same, case insensitive 4. location /prefix plain prefix, the loser

Read items 3 and 4 together. A regex beats a plain prefix, wherever it sits in the file. Now consider the most common configuration in the world, long cache for assets plus a proxy for the application:

location ~ \.js$ { expires 1y; } location /app/ { proxy_pass http://127.0.0.1:8100; }

I set up exactly that on a real nginx, with a backend that answers any path, and asked for /app/client.js. I put a header in each block to see which one took it:

# with location /app/ HTTP/1.1 404 Not Found X-Matched-By: regex .js # with location ^~ /app/ HTTP/1.1 200 OK X-Matched-By: prefix /app/ body: from the backend: /app/client.js

In the first version the file 404s. The \.js$ regex won, nginx went looking for client.js on disk, did not find it, and returned 404. The proxy was never consulted. The application loads, the HTML arrives, and only the scripts vanish. Nothing shows up in the application log, because the request never reached it.

Two characters fix it. ^~ means if this prefix matches, stop here and do not consider any regex.

The practical rule: every prefix location that does proxy_pass gets ^~. No exceptions. It costs nothing, and the day somebody adds an asset regex to the file, nothing breaks.

What your application stops knowing

Crossing the proxy costs information. The application now talks to nginx rather than to the browser, and by default it loses almost everything about who called.

I brought up a backend that prints the headers it receives and went through a proxy_pass with nothing else configured, asking for the site mysite.dev:

# proxy_pass on its own Host: 127.0.0.1:8101 client TCP address seen by the backend: 127.0.0.1

That is all that arrives. Look at the Host: the application believes it is 127.0.0.1:8101. Every absolute link it builds, every redirect, every signup confirmation email will point at http://127.0.0.1:8101. And the visitor's IP is always 127.0.0.1, so your per IP rate limit, your access log and your fraud checks see the entire world as one person.

With the four lines that were missing:

proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme;

# same request, now Host: mysite.dev X-Real-IP: 127.0.0.1 X-Forwarded-For: 127.0.0.1 X-Forwarded-Proto: http

X-Forwarded-Proto deserves a paragraph of its own, because it is the source of a bug that eats whole afternoons. TLS terminates at nginx. From nginx to your application the traffic is plain HTTP, and that is exactly as intended. Except the application, looking at its own connection, correctly concludes it is on HTTP, and generates an http:// link on an https:// site. The browser then blocks mixed content, or HSTS forces HTTPS, the server redirects, and you have a redirect loop with no visible cause.

Sending the header is half the fix. The other half is the application believing it, and that is explicit in every framework:

Careful with that trust. Only trust X-Forwarded-* if the proxy is the only route to your application. If it also listens on 0.0.0.0, anyone can send whatever header they like and your per IP rate limit becomes decoration. That is the second reason, on top of part 2's, to listen only on 127.0.0.1.

WebSocket and the connection that cannot be closed

A detail worth explaining turned up in the tests above: the backend received Connection: close. By default nginx talks to the application in HTTP/1.0 and closes the connection on every request, even while the browser is on keep-alive on the outside.

For ordinary traffic that costs one TCP handshake per request. For WebSocket it is fatal: the connection has to be upgraded, and a connection the proxy closes cannot be upgraded into anything.

location ^~ /ws/ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_read_timeout 3600s; }

The first three lines make the upgrade happen. The fourth is the one that usually goes missing: proxy_read_timeout defaults to 60 seconds, and a WebSocket idle for over a minute gets dropped by the proxy. The symptom is a chat that disconnects itself when nobody is talking, and the blame lands on the client, which had nothing to do with it.

Who serves the static file

Almost every framework can serve a public folder. The question is whether it should.

I measured with a 120 KB file, a realistic bundle size, 4000 requests over 20 concurrent connections, two rounds each:

About 1.7 times faster, and without the 28.6 MB. The test client ran on the same machine, so the absolute figures say little. The ratio is what matters, and it comes from a structural difference: nginx hands the file over with sendfile, copying from page cache to socket inside the kernel, without the data passing through userspace at all. Node reads the file into its own memory, assembles the response and writes it. From the second request on both read from cache, and the shorter path still wins.

In practice this is one location:

location ^~ /assets/ { alias /srv/myapp/current/public/assets/; expires 1y; add_header Cache-Control "public, immutable"; access_log off; }

immutable is for files with a hash in the name, and tells the browser not to revalidate even when the user hits reload. If your build does not put hashes in the names, do not use a one year cache, because you will not be able to ship a fix.

2000 connections, 0.8 MB

Here is the measurement the whole series rests on.

nginx does not use a thread per connection. Each worker is a single process with an event loop on top of epoll, and an idle connection is a file descriptor plus a few hundred bytes of state. Saying that is easy. I opened 2000 simultaneous keep-alive connections against a real nginx and measured its own memory before and during:

before: 1 connection nginx at 5.9 MB during: 2001 connections nginx at 6.7 MB 2000 connections cost 0.8 MB, roughly 0.4 KB each

Compare that with the opposite model, PHP-FPM from part 1: one process per concurrent request, at 25 to 40 MB each. Serving 2000 requests at once under that model would ask for tens of gigabytes. This is not a tuning question, it is the architecture.

That is why the 2 GB sum works out. The layer that talks to the world, the one absorbing every spike and every slow connection, is the cheapest in the stack. The expensive processes sit behind it, in small and controlled numbers.

There is a ceiling, and it is worth knowing where:

worker_processes auto; # one per vCPU. With 4 vCPUs, 4 workers events { worker_connections 768; # Ubuntu default, per worker } # ceiling = 4 x 768 = 3072 connections # each proxied request uses two: browser and backend

On a 2 GB VPS you can raise worker_connections to 4096 without pain, because the cost per connection is that 0.4 KB. What runs out first is the process's file descriptor limit, which you raise with LimitNOFILE in the systemd unit, exactly as in part 2.

Where TLS terminates

It terminates at nginx. The certificate is its, the negotiation is its, and from there inwards the traffic is plain HTTP on 127.0.0.1, which is why the whole headers section exists.

Free certificate with automatic renewal:

apt install certbot python3-certbot-nginx certbot --nginx -d mysite.dev -d www.mysite.dev # renewal already ships as a systemd timer: systemctl list-timers | grep certbot certbot renew --dry-run

certbot edits your nginx file and installs a timer. Running the --dry-run once is worth it, because finding out that renewal never worked ninety days later, with the site down, is an avoidable experience.

The HTTP to HTTPS redirect goes in its own block, and port 80 has to stay open for renewal validation to work:

server { listen 80; server_name mysite.dev www.mysite.dev; location ^~ /.well-known/acme-challenge/ { root /var/www/certbot; } location / { return 301 https://$host$request_uri; } }

One word of warning about HSTS. The Strict-Transport-Security header tells the browser never to try HTTP on that domain again, and it is remembered for the length of max-age. It is a good idea, and it is a door that locks from the outside: until that period expires, no visitor who has been to the site can reach it without valid TLS, however much you want them to. Start with a short max-age.

The limits nobody configures

Four lines that avoid four incidents:

client_max_body_size 10m; # default is 1m. Bigger upload gets a 413 client_body_timeout 15s; send_timeout 15s; proxy_read_timeout 60s; # how long the proxy waits for your app gzip on; gzip_types text/css application/javascript application/json image/svg+xml; gzip_min_length 1024; # below this, compressing costs more than it saves

The 1 MB default for client_max_body_size is the champion of support tickets. A user tries to upload a phone photo, gets a 413, and there is no record of it anywhere in the application log, because nginx refused before handing it over.

And a 60 second proxy_read_timeout combined with a slow route produces 504s that go missing from the application log for the opposite reason: it did answer, just too late, and nobody was listening any more.

The checklist

What comes in part 4

The machine is legible, the process is supervised and the front door hands out the traffic. What is left is the question you have been putting off since the start: how the code gets there.

Part 4 is the deploy. What you ship is not your working directory, and the difference between those two things is the most treacherous class of bug there is, because it never reproduces locally. Numbered releases with an atomic swap, rollback in one command, where to build when the build does not fit on the machine, secrets with permissions, and the Postgres max_connections sum that part 1 left open.

kelvin@cluxnei:~/blog$