SSH Hardening Guide

SSH (Secure Shell) is the primary remote access protocol for Linux servers. Because it is exposed to the internet on nearly every server, it is also one of the most attacked services. Automated bots scan the entire IPv4 address space and begin brute-force attempts within minutes of a new server coming online. This guide covers key generation, agent forwarding, client configuration, server hardening, fail2ban, port knocking, and jump hosts -- everything you need to lock down SSH access properly.

Hub: Linux Security Hardening | See also: Firewall Best Practices, Audit & Intrusion Detection

Generating SSH Keys

Ed25519 keys are the modern default. They are fast, compact (68 characters for a public key versus 544 for RSA-4096), and based on strong elliptic-curve cryptography. They are supported by OpenSSH 6.5 and later, which covers every actively maintained Linux distribution.

# Generate an Ed25519 key pair with a descriptive comment
ssh-keygen -t ed25519 -C "you@workstation-2025"

# If you need RSA for legacy compatibility (older appliances, etc.)
ssh-keygen -t rsa -b 4096 -C "you@workstation-2025"

# Copy the public key to a remote server
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server

# Verify the key fingerprint on the remote server
ssh-keygen -lf ~/.ssh/id_ed25519.pub

Always protect your private key with a strong passphrase. If an attacker obtains the key file (from a stolen laptop, a backup, or a compromised workstation), the passphrase is the last line of defense. Without a passphrase, the key file alone grants access.

Key File Permissions

SSH is strict about file permissions and will refuse to use keys or configs with overly permissive settings:

# Correct permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/config

Using ssh-agent

Typing your passphrase every time you connect is tedious. ssh-agent caches the decrypted key in memory for the duration of your session so you enter the passphrase once:

# Start the agent (most desktop environments do this automatically)
eval "$(ssh-agent -s)"

# Add your key (you will be prompted for the passphrase once)
ssh-add ~/.ssh/id_ed25519

# Add with a timeout (key is forgotten after 4 hours)
ssh-add -t 14400 ~/.ssh/id_ed25519

# List loaded keys
ssh-add -l

# Remove all keys from the agent
ssh-add -D

# Remove a specific key
ssh-add -d ~/.ssh/id_ed25519

On macOS you can add --apple-use-keychain to store the passphrase in the system keychain so it persists across reboots. On Linux, GNOME Keyring or KDE Wallet serves a similar role.

Client Configuration (~/.ssh/config)

The SSH client configuration file eliminates repetitive command-line options, reduces typing errors, and can enforce security settings globally:

# ~/.ssh/config

# Global defaults applied to every connection
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    AddKeysToAgent yes
    IdentitiesOnly yes
    HashKnownHosts yes
    VisualHostKey yes

# Jump host (bastion)
Host bastion
    HostName bastion.example.com
    User admin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

# Production web server (accessed via the bastion)
Host webprod
    HostName 10.0.1.50
    User deploy
    ProxyJump bastion
    IdentityFile ~/.ssh/id_ed25519

# Development server
Host dev
    HostName dev.example.com
    User developer
    Port 22
    ForwardAgent yes
    IdentityFile ~/.ssh/id_ed25519

With this configuration, ssh webprod automatically tunnels through the bastion host. IdentitiesOnly yes prevents the client from offering every key in your agent to every host, which is both a security improvement (a malicious server could catalogue which keys you offer) and a usability improvement (you avoid "too many authentication failures" errors).

HashKnownHosts yes stores hostnames as hashes in known_hosts, so an attacker who reads the file cannot learn which servers you connect to.

Hardening sshd_config

The server-side configuration at /etc/ssh/sshd_config is where the most impactful hardening happens. Apply these settings and restart sshd:

# /etc/ssh/sshd_config  (key hardening directives)

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
AllowUsers deploy admin
MaxAuthTries 3
LoginGraceTime 30
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no
ClientAliveInterval 300
ClientAliveCountMax 2
Port 2222

# Restrict to strong cryptographic algorithms
KexAlgorithms curve25519-sha256,[email protected]
Ciphers [email protected],[email protected],[email protected]
MACs [email protected],[email protected]
HostKeyAlgorithms ssh-ed25519,rsa-sha2-512

# Display a legal banner
Banner /etc/ssh/banner.txt

# Validate the configuration before restarting (catches syntax errors)
sudo sshd -t

# Restart the service
sudo systemctl restart sshd

Critical safety tip: always keep an existing SSH session open while testing changes. If the new configuration locks you out, the existing session lets you revert. Also ensure your firewall allows the new port before restarting sshd.

Fail2ban for Brute-Force Protection

Fail2ban monitors log files and temporarily bans IP addresses that show malicious signs such as repeated authentication failures:

# Install fail2ban
sudo apt install fail2ban    # Debian/Ubuntu
sudo dnf install fail2ban    # Fedora/RHEL

# Create a local override (never edit jail.conf directly)
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit /etc/fail2ban/jail.local:

[DEFAULT]
bantime  = 3600
findtime = 600
maxretry = 3
banaction = iptables-multiport

[sshd]
enabled  = true
port     = 2222
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 3
bantime  = 3600
findtime = 600

# Start and enable fail2ban
sudo systemctl enable --now fail2ban

# Check jail status (shows banned IPs and ban count)
sudo fail2ban-client status sshd

# Manually unban an IP that was accidentally banned
sudo fail2ban-client set sshd unbanip 203.0.113.50

# View the fail2ban log
sudo tail -f /var/log/fail2ban.log

For persistent offenders, increase bantime or use bantime.increment = true in the DEFAULT section for exponentially increasing ban durations.

Port Knocking Concept

Port knocking hides the SSH port behind a sequence of connection attempts to closed ports. Only after the correct sequence does the firewall open SSH for that source IP:

# Install knockd
sudo apt install knockd

# /etc/knockd.conf
# [openSSH]
#   sequence    = 7000,8000,9000
#   seq_timeout = 5
#   command     = /sbin/iptables -I INPUT -s %IP% -p tcp --dport 2222 -j ACCEPT
#   tcpflags    = syn
#
# [closeSSH]
#   sequence    = 9000,8000,7000
#   seq_timeout = 5
#   command     = /sbin/iptables -D INPUT -s %IP% -p tcp --dport 2222 -j ACCEPT

# Client side: knock then connect
knock server.example.com 7000 8000 9000 && ssh -p 2222 [email protected]

Port knocking is security through obscurity and should complement -- not replace -- strong authentication. It does reduce log noise dramatically because bots never discover the open port.

ProxyJump (Jump Hosts)

ProxyJump simplifies tunneling through a bastion host. Instead of two separate SSH sessions or manual port forwarding, a single command chains the connection transparently:

# Command-line jump
ssh -J bastion.example.com user@internal-server

# Multi-hop jump
ssh -J bastion1.example.com,bastion2.example.com user@deep-internal

# Equivalent entry in ~/.ssh/config (shown earlier)
# Host webprod
#     ProxyJump bastion

This architecture keeps internal servers completely off the public internet. The bastion is the only host with a public SSH port, and its attack surface can be minimised with all the techniques in this guide.

SSH hardening is one of the highest-impact security investments you can make. Combined with firewall rules and intrusion detection, it makes unauthorized remote access extremely difficult.