Configure Network Load Balancing on Linux with HAProxy: A Step‑by‑Step Guide

When it comes to distributing traffic across multiple servers, HAProxy has earned a reputation as a lightweight, reliable, and highly configurable load balancer for Linux environments. Whether you’re running a small web farm or a massive micro‑services architecture, mastering HAProxy gives you the control you need to keep users happy and services resilient. In this guide we’ll walk through everything you need to configure network load balancing on Linux with HAProxy, from installation to fine‑tuning, while highlighting common mistakes and sharing practical tips.

What You’ll Need

• A Linux server (Ubuntu 22.04, Debian 12, or CentOS 9 are all fine) with root or sudo privileges.
• At least two backend servers that will serve the same application (e.g., two Apache or Nginx instances).
• Basic knowledge of Linux command line and networking concepts.
• Optional: A domain name and SSL certificate if you plan to terminate TLS at HAProxy.

Step 1: Install HAProxy

First, bring HAProxy onto your front‑end machine. On Ubuntu or Debian run:

sudo apt update
sudo apt install -y haproxy

On CentOS/RHEL you can use the EPEL repository:

sudo dnf install -y epel-release
sudo dnf install -y haproxy

After installation, verify the version to ensure you have a recent release (2.6+ is recommended):

haproxy -v

If you need a newer version, consider building from source – the official HAProxy site provides clear instructions.

Step 2: Prepare Backend Servers

Each backend server must be reachable from the HAProxy host on the service port (usually 80 for HTTP or 443 for HTTPS). For this example we’ll assume two web servers with IPs 10.0.1.101 and 10.0.1.102 listening on port 80.

Test connectivity:

curl http://10.0.1.101
curl http://10.0.1.102

If you receive the expected HTML output, you’re ready to move on. Remember to open the necessary firewall ports (e.g., ufw allow 80/tcp or firewall-cmd --add-port=80/tcp --permanent).

Step 3: Create a Basic HAProxy Configuration

HAProxy’s main config lives at /etc/haproxy/haproxy.cfg. Back up the original file before editing:

sudo cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bak

Replace its contents with a minimal, functional configuration:

global
    log /dev/log    local0
    log /dev/log    local1 notice
    daemon
    maxconn 2000
    tune.ssl.default-dh-param 2048

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

frontend http_front
    bind *:80
    default_backend web_back

backend web_back
    balance roundrobin
    server web1 10.0.1.101:80 check
    server web2 10.0.1.102:80 check

This configuration does three things:

• Listens on port 80 for incoming HTTP traffic.
• Uses the roundrobin algorithm to distribute requests evenly.
• Performs a basic health check (the check flag) on each backend server.

Save the file and test the syntax:

sudo haproxy -c -f /etc/haproxy/haproxy.cfg

If the output says “Configuration file is valid”, you can start HAProxy:

sudo systemctl enable haproxy
sudo systemctl start haproxy

Visit the HAProxy server’s IP address in a browser – you should see the same content as the backend servers, confirming that traffic is being balanced.

Step 4: Enable Advanced Health Checks

Basic TCP checks are fine for simple services, but HTTP‑level health checks give you more confidence. Modify the backend section:

backend web_back
    balance roundrobin
    option httpchk GET /health
    http-check expect status 200
    server web1 10.0.1.101:80 check check-ssl verify none
    server web2 10.0.1.102:80 check check-ssl verify none

Here we ask each server to respond to /health with a 200 status. Adjust the path to match whatever health endpoint your application exposes (e.g., /status or /ping).

Reload HAProxy without dropping connections:

sudo systemctl reload haproxy

Check the HAProxy stats page (if you enable it later) or run echo "show stat" | sudo socat /var/run/haproxy.sock stdio to see health status.

Step 5: Fine‑Tune the Load‑Balancing Algorithm

Round‑robin works for most cases, but HAProxy offers many algorithms:

• leastconn – Sends new connections to the server with the fewest active sessions.
• source – Hashes the client IP, providing session persistence without cookies.
• uri – Hashes the request URI, useful for cache‑friendly workloads.

To switch to leastconn, edit the backend block:

backend web_back
    balance leastconn
    ...

After reloading, you can monitor which algorithm is in effect via the stats socket or the web UI.

Step 6: Secure HAProxy with SSL Termination

Terminating TLS at HAProxy offloads encryption work from your backends and lets you use a single certificate for all traffic. First, obtain a certificate (Let’s Encrypt, self‑signed, or commercial). Place the .crt and .key files in /etc/haproxy/certs/ and combine them:

sudo mkdir -p /etc/haproxy/certs
sudo cat fullchain.pem privkey.pem > /etc/haproxy/certs/haproxy.pem
sudo chmod 600 /etc/haproxy/certs/haproxy.pem

Update the frontend to listen on 443 and use the certificate:

frontend https_front
    bind *:443 ssl crt /etc/haproxy/certs/haproxy.pem alpn h2,http/1.1
    redirect scheme https if !{ ssl_fc }
    default_backend web_back_secure

backend web_back_secure
    balance roundrobin
    option httpchk GET /health
    http-check expect status 200
    server web1 10.0.1.101:80 check
    server web2 10.0.1.102:80 check

Note the use of alpn to enable HTTP/2 when the client supports it. Reload HAProxy and test with curl -k https://your‑haproxy‑ip or a browser.

Step 7: Test, Monitor, and Automate

Before you declare victory, run a few real‑world tests:

• Use ab -n 1000 -c 50 http://haproxy_ip/ or wrk to generate load and watch the distribution.
• Inspect HAProxy logs at /var/log/haproxy.log (or journalctl -u haproxy on systemd).
• Enable the built‑in stats page for a live dashboard. Add to frontend http_front:

      bind *:8404
      stats enable
      stats uri /stats
      stats refresh 10s
      stats auth admin:strongpassword
  

Finally, make HAProxy start on boot (already done with systemctl enable) and consider adding a watchdog script or using systemd’s Restart=on-failure to keep it running.

Common Mistakes to Avoid

Even experienced admins trip over a few pitfalls when configuring HAProxy:

• Forgetting to open firewall ports. If iptables or ufw blocks 80/443, clients will see connection timeouts.
• Mismatched health‑check paths. A typo in option httpchk will mark healthy servers as down.
• Using “maxconn” too low. The global maxconn limits total concurrent connections; set it high enough for your traffic spikes.
• Not reloading after config changes. Editing haproxy.cfg without systemctl reload haproxy leaves the old config in effect.
• Storing certificates with insecure permissions. HAProxy will refuse to start if the PEM file is world‑readable.

Tips and Tricks

Here are a few pro‑level adjustments that can squeeze extra performance and reliability out of HAProxy:

• Enable TCP fast open. Add tune.tcpka.accept 1 and tune.tcpka.queue 1000 in the global section.
• Use stick tables for rate limiting. Example:

  backend web_back
      stick-table type ip size 1m expire 10s store http_req_rate(10s)
      tcp-request connection track-sc0 src
      acl too_fast sc_http_req_rate(0) gt 100
      tcp-request connection reject if too_fast
  

• Log in JSON format. Modern log aggregators love JSON. Add log-format {"time":"%t","src":"%ci","dst":"%si","status":%ST}
under global.
• Deploy HAProxy in Docker or Kubernetes. The same haproxy.cfg works inside a container; just mount the config and certificates as volumes.

Frequently Asked Questions

Can HAProxy handle UDP traffic?

Yes, HAProxy 2.0+ includes native UDP support. You define a frontend with proto udp and a matching backend. However, for DNS or VoIP workloads you may prefer specialized load balancers like CoreDNS or Kamailio.

Do I need to run HAProxy on a separate machine?

Not necessarily. Small setups can run HAProxy on the same host as one of the backends, but for true high availability you should deploy at least two HAProxy instances behind a virtual IP (using keepalived or Corosync) to avoid a single point of failure.

How do I enable HTTP/2 to the backend servers?

HAProxy can speak HTTP/2 on the client side but only forwards HTTP/1.1 to backends unless you enable mode tcp and let the backends handle TLS termination themselves. For full end‑to‑end HTTP/2 you need backend servers that support it and configure HAProxy with option http-use-htx and http-request set-var(req.backend_proto) str(HTTP/2) – a more advanced topic beyond this guide.

Conclusion

Setting up HAProxy for network load balancing on Linux is a straightforward process once you understand the core concepts: install the binary, define front‑ends and back‑ends, enable health checks, and optionally add SSL termination. By following the steps above, avoiding common pitfalls, and applying the performance tricks, you’ll have a robust, scalable load‑balancing layer that can grow with your infrastructure. Keep the HAProxy stats page handy, monitor logs, and iterate on the configuration as traffic patterns evolve. Happy balancing!

Photo by Kevin Ache on Unsplash