Secure remote access is the backbone of modern system administration, but an improperly configured SSH service is a gold mine for attackers. In this guide we’ll walk you through every essential measure to lock down SSH on a Linux server, from generating key pairs to hardening the firewall. By the end you’ll have a robust, key‑only SSH setup that resists brute‑force attacks and limits exposure.
What You’ll Need
- A fresh or existing Linux server with root or sudo access
- A workstation with an SSH client (Linux, macOS, or PuTTY on Windows)
- Basic knowledge of Linux command line
- An internet connection for package installation
- Optional: a mobile device for two‑factor authentication
Step 1: Update the System and Install Essential Packages
Before tightening security, make sure the server is up to date. Run the following commands (use apt for Debian/Ubuntu or yum/dnf for RHEL/CentOS):
sudo apt update && sudo apt upgrade -y # Debian/Ubuntu # or sudo dnf update -y # Fedora/CentOS 8+
Next, install fail2ban to automatically block repeated failed login attempts:
sudo apt install fail2ban -y # Debian/Ubuntu # or sudo dnf install fail2ban -y # Fedora/CentOS
Enable and start the service:
sudo systemctl enable fail2ban sudo systemctl start fail2ban
Fail2ban will monitor the SSH log and ban IPs that exceed a configurable threshold.
Step 2: Create a Dedicated Non‑Root User
Logging in as root is risky because a compromised key gives an attacker full control instantly. Create a regular user and grant sudo privileges:
sudo adduser adminuser # Follow the prompts for password and info sudo usermod -aG sudo adminuser # Debian/Ubuntu # For RHEL/CentOS use: sudo usermod -aG wheel adminuser
Test the new account:
ssh adminuser@your_server_ip
If the login works, you can safely disable direct root SSH later.
Step 3: Harden the SSH Daemon Configuration
The main configuration file lives at /etc/ssh/sshd_config. Open it with your favorite editor:
sudo nano /etc/ssh/sshd_config
Make the following changes (add them if they’re missing):
Port 2222– move SSH off the default port 22 to reduce automated scans.Protocol 2– enforce the more secure protocol version.PermitRootLogin no– block direct root logins.PubkeyAuthentication yes– enable key‑based auth.PasswordAuthentication no– will be enforced later after keys work.AllowUsers adminuser– restrict which accounts may log in via SSH.
After editing, test the configuration syntax:
sudo sshd -t
If no output appears, the file is syntactically correct. Reload the daemon:
sudo systemctl reload sshd
If you changed the port, remember to adjust any firewall rules accordingly.
Step 4: Set Up Public‑Key Authentication
Generate a key pair on your workstation (skip this step if you already have one):
ssh-keygen -t ed25519 -C "your_email@example.com"
Accept the default location (~/.ssh/id_ed25519) and optionally add a passphrase for extra protection.
Copy the public key to the server for the new user:
ssh-copy-id -i ~/.ssh/id_ed25519.pub adminuser@your_server_ip -p 2222
If ssh-copy-id is unavailable, you can manually append the key:
ssh adminuser@your_server_ip -p 2222 "mkdir -p ~/.ssh && chmod 700 ~/.ssh" cat ~/.ssh/id_ed25519.pub | ssh adminuser@your_server_ip -p 2222 "cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
Now test the key‑only login:
ssh -i ~/.ssh/id_ed25519 adminuser@your_server_ip -p 2222
If you can connect without a password, you’re ready for the next step.
Step 5: Disable Password Authentication Globally
Open /etc/ssh/sshd_config again and ensure the line reads:
PasswordAuthentication no
Save, test the config, and reload:
sudo sshd -t && sudo systemctl reload sshd
From now on, only users with a valid private key can log in. This eliminates the most common brute‑force vector.
Step 6: Restrict Root Login and Use Sudo Sparingly
Even though we already set PermitRootLogin no, double‑check that the root account cannot be accessed via SSH. Additionally, configure sudo to require a password for privilege escalation, which adds a second factor of authentication:
sudo visudo
Make sure the line for your user looks like:
adminuser ALL=(ALL) ALL
Avoid adding NOPASSWD unless you have a very specific automation need.
Step 7: Enable Two‑Factor Authentication (2FA)
For an extra layer of security, install Google Authenticator or a compatible PAM module. On Debian/Ubuntu:
sudo apt install libpam-google-authenticator -y
Run the setup for the user you’ll log in as:
google-authenticator
Answer the prompts (recommend “time‑based tokens” and “rate‑limit login attempts”).
Edit /etc/pam.d/sshd and add:
auth required pam_google_authenticator.so nullok
Then modify /etc/ssh/sshd_config to include:
ChallengeResponseAuthentication yes
Reload SSH:
sudo systemctl reload sshd
Now each login will require the one‑time code from your authenticator app after the key verification.
Step 8: Harden the Firewall (UFW/Firewalld)
Allow only the custom SSH port and any other services you need. Using UFW (Ubuntu/Debian):
sudo ufw default deny incoming sudo ufw default allow outgoing sudo ufw allow 2222/tcp # replace with your chosen port sudo ufw enable
On RHEL/CentOS with firewalld:
sudo firewall-cmd --permanent --add-port=2222/tcp sudo firewall-cmd --permanent --remove-service=ssh # optional, removes default 22 sudo firewall-cmd --reload
Verify the rule:
sudo ufw status # or sudo firewall-cmd --list-all
Your server now only accepts SSH on the non‑standard port, from any IP. For stricter security you can limit the source IP range:
sudo ufw allow from 203.0.113.0/24 to any port 2222 proto tcp
Common Mistakes to Avoid
1 Leaving PasswordAuthentication enabled: The most common oversight; always test key‑only login before disabling passwords.
2 Changing the SSH port without updating the firewall: You’ll lock yourself out if the new port is blocked.
3 Using weak or default usernames: Attackers often try root, admin, or user. Choose a unique login name.
4 Storing private keys without a passphrase: If the key file is stolen, the attacker gains immediate access.
5 Granting NOPASSWD sudo rights: This bypasses the second authentication factor and defeats the purpose of hardening.
6 Forgetting to restart or reload sshd after changes: The old configuration stays active until you reload.
7 Not backing up sshd_config before editing: A syntax error can lock you out; keep a copy you can restore.
Tips and Tricks
• Use Fail2Ban custom jail: Create /etc/fail2ban/jail.d/ssh.conf with maxretry = 3 and bantime = 86400 for aggressive blocking.
• Deploy SSH certificates: For large fleets, OpenSSH certificates signed by a CA simplify key rotation.
• Enable TCPKeepAlive and ClientAliveInterval: Prevent idle connections from lingering:
ClientAliveInterval 300 ClientAliveCountMax 2
• Log all SSH activity: Set LogLevel VERBOSE in sshd_config to capture key fingerprint usage.
• Use a jump host (bastion): Funnel all external SSH through a hardened gateway instead of exposing each server directly.
Frequently Asked Questions
Do I really need to change the default SSH port?
Changing the port is security‑by‑obscurity; it won’t stop a determined attacker but it reduces noise from automated scans and buys you time to notice suspicious activity.
Can I still use scp and rsync after hardening SSH?
Yes. All these tools rely on the SSH daemon, so once key‑based authentication and the custom port are configured, they work unchanged—just remember to specify the port with -P 2222 for scp or -e "ssh -p 2222" for rsync.
What if I lose my private key?
If you lose the key, you’ll need console or physical access to the server to add a new key for your user. Always keep a backup of the private key in a secure password‑manager or encrypted USB drive.
Conclusion
Securing SSH is a layered process: keep the software updated, enforce key‑based authentication, lock down the daemon configuration, add fail2ban and optional 2FA, and finally restrict network access with a firewall. By following the steps above you’ll dramatically reduce the attack surface of your Linux server while maintaining convenient, reliable remote access. Remember to audit your settings regularly and rotate keys periodically—security is a habit, not a one‑time task.
