How to Secure Your Linux Server with a Firewall – A Step‑by‑Step Guide

Running a Linux server without a firewall is like leaving the front door wide open – you’re inviting trouble before you even notice it. Whether you’re hosting a web app, a database, or a simple file server, a properly configured firewall is your first line of defense against unwanted traffic, brute‑force attacks, and accidental exposure of services. In this guide we’ll walk through the entire process of securing a Linux server with a firewall, from installing the right tools to testing and fine‑tuning rules. The steps are written for users with a basic command‑line background, but we’ll also dive into some intermediate concepts so you come away with a robust, production‑ready setup.

What You’ll Need

• A fresh or already running Linux server (Ubuntu, Debian, CentOS, or Fedora are covered)
• Root or sudo privileges
• SSH access (or direct console) to run commands
• Basic knowledge of the services you intend to expose (e.g., SSH, HTTP, MySQL)
• A text editor such as nano or vim

Step 1: Choose and Install a Firewall Tool

Linux ships with several firewall frameworks. The most common choices are UFW (Uncomplicated Firewall) for Ubuntu/Debian‑based systems and firewalld for RHEL/CentOS/Fedora. Advanced users may prefer raw iptables or nftables. For this guide we’ll use UFW because it balances simplicity with power, but we’ll also show the equivalent iptables commands for those who need them.

On Ubuntu/Debian, install UFW with:

sudo apt update && sudo apt install ufw -y

On CentOS 8+ or Fedora, enable firewalld (the default) and install iptables-services if you prefer the classic approach:

sudo dnf install firewalld -y
sudo systemctl enable --now firewalld

If you decide to work directly with iptables, make sure the package is installed:

sudo apt install iptables -y   # Debian/Ubuntu
sudo dnf install iptables-services -y   # RHEL/Fedora

Step 2: Set Default Policies

Before opening any ports, define a default stance. The safest default is to deny all inbound traffic while allowing all outbound traffic. This ensures your server can still reach the internet (for updates, apt, etc.) but strangers cannot initiate connections.

With UFW:

sudo ufw default deny incoming
sudo ufw default allow outgoing

With iptables (equivalent):

sudo iptables -P INPUT DROP
sudo iptables -P FORWARD DROP
sudo iptables -P OUTPUT ACCEPT

These commands set the policy for the three main chains. Remember to save iptables rules later (see Step 7).

Step 3: Allow Essential Services

Now whitelist the services you actually need. Most servers require SSH for remote management, and if you’re running a web site you’ll need HTTP (port 80) and HTTPS (port 443). Add each rule one at a time and verify the syntax.

UFW example:

# Allow SSH (port 22) – limit to mitigate brute‑force
sudo ufw limit ssh
# Allow HTTP and HTTPS
sudo ufw allow http
sudo ufw allow https

If you use a non‑standard SSH port (e.g., 2222), replace ssh with the port number:

sudo ufw allow 2222/tcp

iptables equivalent (using iptables‑save format later):

sudo iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT
# Allow related/established traffic back out
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

Step 4: Create Custom Rules for Application‑Specific Ports

Many servers host databases, caching layers, or internal APIs that listen on non‑standard ports. Expose only what is required and restrict access to trusted IP ranges whenever possible.

Example: Allow MySQL (port 3306) **only** from the internal network 10.0.0.0/24.

# UFW syntax – source address restriction
sudo ufw allow from 10.0.0.0/24 to any port 3306 proto tcp

# iptables equivalent
sudo iptables -A INPUT -p tcp -s 10.0.0.0/24 --dport 3306 -m conntrack --ctstate NEW -j ACCEPT

For a Redis cache (port 6379) that should never be reachable from the internet, you might block it entirely:

# UFW – deny all inbound to 6379 (default policy already drops, but be explicit)
sudo ufw deny 6379
# iptables – explicit drop (helps readability)
sudo iptables -A INPUT -p tcp --dport 6379 -j DROP

Step 5: Enable Logging and Real‑Time Monitoring

Seeing what the firewall blocks is crucial for troubleshooting and for spotting attacks. UFW ships with a simple logging toggle; iptables requires a bit more setup.

UFW logging:

sudo ufw logging on   # default level is low; use "high" for more detail
sudo ufw status verbose   # shows current rules and logging state

iptables logging (prepend a LOG rule before the DROP policy):

sudo iptables -N LOGGING
sudo iptables -A INPUT -j LOGGING
sudo iptables -A LOGGING -m limit --limit 5/min -j LOG --log-prefix "[IPTABLES DROP] " --log-level 4
sudo iptables -A LOGGING -j DROP

Check logs via journalctl -k or /var/log/kern.log depending on your distro.

Step 6: Test the Firewall Thoroughly

Never assume the rules work; always verify from both inside and outside the network.

• From the server itself, run nc -zv localhost 22 to confirm SSH is reachable.
• From a remote machine, try telnet your_server_ip 22 (or nc -zv your_server_ip 22) to ensure the port is open.
• Attempt to connect to a blocked port (e.g., telnet your_server_ip 3306 from an unauthorized IP) and confirm the connection is refused.

If a rule is too restrictive, you’ll see “Connection timed out” or “Connection refused”. Adjust the offending rule and reload the firewall (UFW: sudo ufw reload).

Step 7: Persist Rules Across Reboots

UFW automatically saves changes, but with raw iptables you must store the rule set.

For iptables on Debian/Ubuntu:

sudo sh -c "iptables-save > /etc/iptables/rules.v4"
# Ensure the iptables‑restore service loads at boot
sudo apt install iptables-persistent -y

On RHEL/CentOS, use the service:

sudo service iptables save   # writes to /etc/sysconfig/iptables
sudo systemctl enable iptables

Firewalld rules are persistent by default, but you can make a backup with:

sudo firewall-cmd --runtime-to-permanent

Step 8: Harden Further with Rate Limiting and Fail2Ban (Optional)

Even a well‑configured firewall can be overwhelmed by brute‑force attempts. Adding rate limits on SSH and deploying Fail2Ban gives an extra layer of protection.

UFW rate‑limit (already used in Step 3) automatically blocks an IP after 6 connection attempts within 30 seconds. For custom services you can add:

sudo ufw limit 8080/tcp   # Example for a custom web app

Install Fail2Ban and enable the default SSH jail:

sudo apt install fail2ban -y   # Debian/Ubuntu
sudo systemctl enable --now fail2ban
# Verify status
sudo fail2ban-client status sshd

Adjust /etc/fail2ban/jail.local to protect other services like nginx‑http-auth or mysqld‑auth. The combination of firewall rate limits and Fail2Ban dramatically reduces the chance of a successful credential‑guessing attack.

Common Mistakes to Avoid

1 Locking yourself out – Adding a deny rule before allowing SSH is a classic error. Always add allow rules first, then set the default deny policy, and test SSH access immediately after changes.

2 Forgetting to allow established connections – Without a rule that permits ESTABLISHED,RELATED traffic, outbound connections (e.g., apt updates) will be dropped.

3 Over‑opening ports – It’s tempting to “just allow everything” for convenience. Each open port is a potential attack surface; only expose what you truly need.

4 Neglecting to save iptables rules – On systems that don’t auto‑save, a reboot will wipe your configuration, leaving the server unprotected.

5 Ignoring logs – Logs reveal scanning attempts and mis‑configured rules. Regularly review /var/log/kern.log or journalctl -u ufw.

Tips and Tricks

• Use ufw status numbered to see rule order; lower numbers are evaluated first.

• Group related rules into a single UFW application profile (e.g., create /etc/ufw/applications.d/webserver).

• Combine firewalls with SELinux or AppArmor for defense‑in‑depth.

• When managing many servers, consider a configuration management tool (Ansible, Chef) to push identical firewall policies.

Frequently Asked Questions

Do I need both UFW and iptables?

No. UFW is a front‑end for iptables, translating its rules into the underlying netfilter tables. Choose one tool to avoid rule conflicts.

Can I use UFW on CentOS?

UFW is primarily packaged for Debian‑based distributions, but it can be installed from EPEL on CentOS 7+ if you prefer its syntax. However, firewalld is the native solution on RHEL‑derived systems.

What if I need to open a port temporarily?

UFW lets you add a rule with a time limit using ufw allow 1234/tcp && sleep 3600 && ufw delete allow 1234/tcp, or you can schedule removal with at. For iptables, add the rule and later delete it with iptables -D.

Conclusion

Securing a Linux server with a firewall is a straightforward yet powerful way to reduce attack surface and protect critical services. By installing a reliable firewall tool, defining strict default policies, whitelisting only the ports you truly need, and adding logging, rate limiting, and persistence, you create a robust defensive perimeter. Remember to test your configuration, keep backups of your rule set, and revisit the rules whenever you add new services. With the steps outlined above, your server will be much harder for attackers to breach, giving you peace of mind and a solid foundation for further hardening measures.

Photo by Albert Stoynov on Unsplash