Running a Linux server in production means you’re responsible for keeping it locked down against the endless stream of attacks that roam the internet. While cloud providers give you a baseline of security, the real protection comes from the tools you configure yourself. In this guide we’ll walk you through setting up a robust firewall (using UFW or iptables) and pairing it with Fail2Ban to automatically block malicious login attempts. By the end you’ll have a hardened server that can still serve web traffic, SSH, and any other services you need—without exposing unnecessary ports.
What You’ll Need
- A fresh or existing Linux server (Ubuntu 20.04+, Debian 11+, or CentOS 8/RHEL 8)
- Root or sudo privileges
- Basic familiarity with the command line
- Access to the server via SSH
- An internet connection for package installation
Step 1: Update the System and Install Required Packages
Before adding any security layers, make sure the OS is up to date. This prevents known vulnerabilities from being exploited.
sudo apt update && sudo apt upgrade -y # Debian/Ubuntusudo dnf update -y # CentOS/RHEL
Next, install the firewall utility (UFW is the default on Ubuntu, firewalld on CentOS) and Fail2Ban:
# Ubuntu/Debiansudo apt install ufw fail2ban -y# CentOS/RHELsudo dnf install firewalld fail2ban -y
Enable the services so they start on boot:
sudo systemctl enable ufw && sudo systemctl start ufw # Ubuntu/Debiansudo systemctl enable firewalld && sudo systemctl start firewalld # CentOS/RHEL
Step 2: Define Your Trusted Services
Identify which ports your server truly needs. Common services include:
- SSH – 22/tcp (or a custom port)
- HTTP – 80/tcp
- HTTPS – 443/tcp
- MySQL – 3306/tcp (if the database is exposed)
Write them down; you’ll reference this list when configuring the firewall.
Step 3: Configure the Firewall
Option A – Using UFW (Ubuntu/Debian)
First, set the default policies to deny everything inbound and allow outbound:
sudo ufw default deny incomingsudo ufw default allow outgoing
Now open only the ports you noted:
sudo ufw allow 22/tcp # SSHsudo ufw allow 80/tcp # HTTPsudo ufw allow 443/tcp # HTTPS
If you changed the SSH port, replace 22 with your custom port.
Enable the firewall:
sudo ufw enable
Verify the rules:
sudo ufw status verbose
Option B – Using firewalld (CentOS/RHEL)
Firewalld works with zones; we’ll use the default public zone.
sudo firewall-cmd --set-default-zone=public
Set default policies (firewalld defaults to reject inbound, but we’ll be explicit):
sudo firewall-cmd --permanent --set-target=DROP
Add the required services:
sudo firewall-cmd --permanent --add-service=sshsudo firewall-cmd --permanent --add-service=httpsudo firewall-cmd --permanent --add-service=https
If you use a custom SSH port, add it manually:
sudo firewall-cmd --permanent --add-port=2222/tcp # example custom port
Reload to apply:
sudo firewall-cmd --reload
Check the active rules:
sudo firewall-cmd --list-all
Step 4: Harden SSH Access
Even with a firewall, a publicly reachable SSH daemon is a prime target. Edit /etc/ssh/sshd_config and apply the following tweaks:
# Disable root login
PermitRootLogin no
# Use only protocol 2
Protocol 2
# Limit authentication to keys
PasswordAuthentication no
# Optional: Change default port (e.g., 2222)
Port 2222
# Optional: Restrict which users can SSH
AllowUsers alice bob
After saving, restart SSH:
sudo systemctl restart sshd
Test the new settings in a separate terminal before closing your current session—otherwise you could lock yourself out.
Step 5: Install and Configure Fail2Ban
Fail2Ban monitors log files for repeated failed authentication attempts and automatically adds temporary firewall bans. The default configuration is safe, but we’ll tailor it for SSH and any web services you run.
Copy the default jail file to a local version so updates won’t overwrite your changes:
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
Edit /etc/fail2ban/jail.local and enable the sshd jail:
[sshd]
enabled = true
port = ssh,2222 # match your SSH port(s)
filter = sshd
logpath = /var/log/auth.log # Ubuntu/Debian
# logpath = /var/log/secure # CentOS/RHEL
maxretry = 5
bantime = 1h
findtime = 10m
If you run a web server, enable the nginx‑http-auth or apache‑auth jails similarly.
Start and enable Fail2Ban:
sudo systemctl enable fail2ban && sudo systemctl start fail2ban
Check status:
sudo fail2ban-client status sshd
Step 6: Verify Everything Works
Now that the firewall and Fail2Ban are live, run a quick sanity check.
- Port scan: From another machine, use
nmapto ensure only the allowed ports are open. - SSH test: Attempt a login with a wrong password three times; Fail2Ban should ban your IP after the threshold.
- Service continuity: Confirm your web site (if any) is still reachable over HTTPS.
Example nmap command:
nmap -p 22,80,443,3306 your.server.ip
You should see closed or filtered for any port you didn’t explicitly allow.
Common Mistakes to Avoid
1. Locking yourself out: Always keep a second SSH session open while tweaking sshd_config. If the new config fails, you can revert without losing access.
2. Forgetting to reload the firewall: Adding rules without ufw reload or firewall-cmd --reload leaves the old rule set in place.
3. Using the default jail.conf for changes: Updates will overwrite it. Always edit jail.local or create a separate .conf in /etc/fail2ban/jail.d/.
4. Neglecting log file paths: Fail2Ban won’t work if you point it at the wrong log (e.g., using /var/log/auth.log on CentOS where the file is /var/log/secure).
5. Setting bantime too low: A short ban (e.g., 30 seconds) gives attackers a chance to keep trying. Start with 1 hour and adjust based on your risk profile.
Tips and Tricks
• Whitelist your own IP: Add sudo ufw allow from YOUR_IP to any port 22 so you never get blocked by a mis‑configured rule.
• Use rate limiting in UFW: sudo ufw limit ssh/tcp automatically drops connections that exceed 6 attempts per minute.
• Log all dropped packets: On Ubuntu, edit /etc/ufw/ufw.conf and set LOGLEVEL=high. On firewalld, enable the audit zone.
• Combine Fail2Ban with recidive jail: The recidive jail tracks repeat offenders across all jails and can apply longer bans.
• Automate backups of config files: A simple cron job that copies /etc/ufw, /etc/fail2ban, and /etc/ssh to a remote location can save hours after a mis‑step.
Frequently Asked Questions
Can I use iptables directly instead of UFW or firewalld?
Yes. UFW and firewalld are front‑ends that generate iptables rules for you. If you prefer raw iptables, you can write iptables commands manually, but you’ll need to persist them across reboots (e.g., using iptables‑save or a systemd service).
What if I need to open a temporary port for a one‑off task?
UFW lets you add a rule that expires after a set time: sudo ufw allow 8080/tcp && sudo ufw delete allow 8080/tcp after you’re done. With firewalld you can use --add-port=8080/tcp --timeout=3600 to auto‑remove after an hour.
How does Fail2Ban interact with the firewall?
Fail2Ban creates temporary iptables (or nftables) rules that drop traffic from offending IPs. When the ban expires, Fail2Ban removes those rules automatically.
Conclusion
Securing a Linux server doesn’t have to be a daunting, one‑off checklist. By layering a strict firewall with automated intrusion‑prevention via Fail2Ban, you create a dynamic defense that adapts to real‑time threats while keeping legitimate traffic flowing. Follow the steps above, test each change, and you’ll have a server that’s both resilient and easy to manage—giving you more time to focus on the services that matter most.
Photo by Lewis Kang’ethe Ngugi on Unsplash





