I am trying to deal with a persistent application level DOS attack on a web server (apache httpd 2.4, debian 10). The attacker appears to be spoofing public IP addresses given the sheer number of different ones used, but it is not close to consuming total bandwidth. I can isolate the offending requests within Apache and reject them instead of processing them but it still chews up too much of httpd's ability to handle everything. When I used fail2ban to parse the apache logs and add the offending IP addresses to a chain, it grew to over 13000 entries and performance on the server as a whole suffered (ksoftirqd pegged to 100% on one cpu core and the ability to handle network events slowed to a crawl).
Following recommendations from https://javapipe.com/blog/iptables-ddos-protection/ I tried adding
-A INPUT -p tcp -m conntrack --ctstate NEW -m limit --limit 5/s --limit-burst 5 -j ACCEPT
-A INPUT -p tcp -m conntrack --ctstate NEW -j DROP
to the iptables rules. Unfortunately this rate limited all incoming connections, regardless of IP address. I can see by the apache access log all incoming trafffice has slowed down to 5/s regardless of source IP. As soon as I added this, all my attempts to connect to the web server timed out. When querying the conntrack table, my IP address was not listed. It appears I was getting caught by the drop rule.
When I replaced these two rules with just
-A INPUT -p tcp -m connlimit --connlimit-above 5 -j REJECT --reject-with tcp-reset
my connections succeeded. My IP also appeared in the conntrack table. If I add the 2 conntrack rules from above after the connlimit rule, it still throttles all traffic (not per IP) as before.
My complete INPUT chain is as follows:
-A INPUT -p tcp -j f2b-recidive
-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
-A INPUT -i lo -j ACCEPT
-A INPUT -s 127.0.0.0/8 ! -i lo -j REJECT --reject-with icmp-port-unreachable
-A INPUT -p tcp -m connlimit --connlimit-above 5 --connlimit-mask 32 --connlimit-saddr -j REJECT --reject-with tcp-reset
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-port-unreachable
What do I need to change to have per-ip rate limit actually work?
MH
