Any public-facing WHMCS install gets hammered with brute-force login attempts within hours of going live. Fail2ban watches your logs, spots repeated failures, and firewalls the offending IPs automatically. This guide sets it up for WHMCS (and the SSH + web stack around it) on Ubuntu.

Prerequisites

  • Ubuntu 22.04 with root access
  • WHMCS running behind nginx or Apache
  • WHMCS activity logging enabled (it is by default)

Step 1 — Install Fail2ban

apt update
apt install -y fail2ban
systemctl enable fail2ban

Step 2 — Create a local jail config

Never edit jail.conf directly (updates overwrite it). Create /etc/fail2ban/jail.local:

[DEFAULT]
# Ban for 1 hour after 5 failures within 10 minutes
bantime  = 3600
findtime = 600
maxretry = 5
# Ignore your own IPs (office, home, monitoring)
ignoreip = 127.0.0.1/8 ::1 YOUR.OFFICE.IP

[sshd]
enabled = true
port    = 2222

Adjust port to your real SSH port and add your static IPs to ignoreip so you never lock yourself out.

Step 3 — Enable escalating bans for repeat offenders

The recidive jail bans IPs that keep coming back after their first ban expires — much longer:

[recidive]
enabled  = true
logpath  = /var/log/fail2ban.log
banaction = iptables-allports
bantime  = 604800   ; 1 week
findtime = 86400    ; 1 day
maxretry = 3

Step 4 — Create a WHMCS filter

WHMCS logs failed admin logins to its database, but it also writes to the web server log on repeated hits. The cleanest approach is a filter on WHMCS's own failed-login log. First, point WHMCS at a log file — or parse the web access log for the login endpoint.

Create /etc/fail2ban/filter.d/whmcs.conf:

[Definition]
# Matches failed admin/client login POSTs returning to login page
failregex = ^<HOST> .* "POST /(dologin|admin/login)\.php.* 200
            ^<HOST> .* "POST /index\.php\?rp=/login.* 200
ignoreregex =

This is a starting pattern — tune failregex to your WHMCS URL structure and template. The key is matching the login POST from the client IP.

Step 5 — Add the WHMCS jail

Append to /etc/fail2ban/jail.local:

[whmcs]
enabled  = true
port     = http,https
filter   = whmcs
logpath  = /var/log/nginx/access.log
maxretry = 5
findtime = 600
bantime  = 7200

Point logpath at your actual nginx/Apache access log for the WHMCS vhost.

Step 6 — Watch out for the real-IP problem

If WHMCS is behind a CDN or reverse proxy, your access log shows the CDN's IP, not the visitor's — Fail2ban would ban the CDN and take your whole site down.

Fix this in nginx by restoring the real IP first. For SprintCDN or Cloudflare, add their ranges to a real-IP config:

# /etc/nginx/snippets/realip.conf
set_real_ip_from 1.2.3.0/24;   # your CDN's ranges
real_ip_header X-Forwarded-For;
real_ip_recursive on;

Include it in your server block and make sure the log format uses $remote_addr (which now reflects the real client). Only then will Fail2ban ban the right IP.

Step 7 — Start and verify

systemctl restart fail2ban
fail2ban-client status

You should see your active jails: sshd, recidive, whmcs. Check a specific jail:

fail2ban-client status sshd

This shows currently banned IPs and total counts.

Step 8 — Test it works

From a throwaway IP (a phone on mobile data works), deliberately fail the SSH login or WHMCS login 6 times. Then on the server:

fail2ban-client status sshd
# Should list the test IP under "Banned IP list"
iptables -L -n | grep <TEST_IP>

Unban it when done:

fail2ban-client set sshd unbanip <TEST_IP>

Step 9 — Get notified of bans (optional)

Fail2ban can email you on each ban. In jail.local under [DEFAULT]:

destemail = you@yourdomain.com
sender    = fail2ban@yourdomain.com
action    = %(action_mwl)s

action_mwl = ban + email with whois + log lines, so you get context on who's attacking.

Layered defense

Fail2ban handles the application layer, but volumetric attacks (someone throwing 50 Gbit/s at your login page) need network-level defense. On Hostfory, DDoS scrubbing absorbs the volumetric layer before it reaches your box, and Fail2ban cleans up the smart, low-volume brute-forcers that slip through. Use both.

Troubleshooting

Fail2ban bans nobody: your failregex doesn't match the log lines. Test it: fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/whmcs.conf. It shows how many lines matched.

Fail2ban banned my CDN / my whole userbase: the real-IP problem from Step 6. You're reading the proxy IP. Fix real-IP restoration before re-enabling.

Locked yourself out: if you have console/IPMI access (included on all Hostfory dedicated servers), get in that way and add your IP to ignoreip. This is why IPMI matters.

Where to go from here

Combine this with a WireGuard management tunnel so SSH isn't even exposed publicly, and restrict the WHMCS admin path to your office IP at the nginx level. Defense in depth beats any single control.