Join Log in
Score:-1
HDR avatar Server

My server is attacking other servers

cz flag
HDR

I've been reached by OVH multiple times regarding a dedicated server which I bought from them, and they're saying that the server is attacking other hosts on their network. The first time, the server was an open proxy due to a misconfiguration (I've enabled the proxy with Apache and didn't restrict it), so I've reinstalled the server and resolved the issue (that's what I thought).

And now it's the second time I've been reached by them saying that the server is attacking other nodes again, and here are the logs that they've provided me with:

Attack detail : 1Mpps/537Mbps
dateTime                   srcIp:srcPort           dstIp:dstPort           protocol flags     packets      bytes reason               
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:34408    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:34408    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:34408    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:34408    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:34408    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:34408    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:41980    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP           
2022.10.31 02:34:14 CET    XX.XX.XX.XX:34408    XXX.XXX.XXX.XXX:80       UDP      ---         16384    1048576 ATTACK:UDP

As you can see this time, the attacks that the server is generating are UDP based, unlike the last time when they were TCP. And this time the proxy isn't even enabled, so what I'm doing wrong?

For the detail of the configuration (of the last installation), I have chosen Debian 11 in the templates that OVH provides as my OS. And I didn't install anything out of the ordinary in it besides apache2 and MariaDB.

I've followed the following tutorials:

For the UDP running services, you can find below the result of netstat command:

user@host:~$ sudo netstat -nputwl

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      1271/mariadbd
tcp        0      0 0.0.0.0:89              0.0.0.0:*               LISTEN      1197/sshd: /usr/sbi
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1683/exim4
tcp6       0      0 :::89                   :::*                    LISTEN      1197/sshd: /usr/sbi
tcp6       0      0 ::1:25                  :::*                    LISTEN      1683/exim4
udp        0      0 0.0.0.0:68              0.0.0.0:*                           643/dhclient
udp        0      0 127.0.0.1:323           0.0.0.0:*                           1174/chronyd
udp6       0      0 ::1:323                 :::*                                1174/chronyd

user@host:~$ sudo netstat -n --udp --listen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
udp        0      0 0.0.0.0:68              0.0.0.0:*
udp        0      0 127.0.0.1:323           0.0.0.0:*
udp6       0      0 ::1:323                 :::*

And when I've started nethogs, I've found plenty of unknown connections (this is just the first 10 seconds of running because the connections just kept coming):

    PID USER     PROGRAM                                                                                                                         DEV        SENT      RECEIVED
      ? root     XX.XX.XX.XX:80-AA.AA.AA.AA:44980                                                                                                         0.000       3.636 KB/sec
      ? root     XX.XX.XX.XX:443-BB.BB.BB.BB:55541                                                                                                        0.063       0.077 KB/sec
      ? root     XX.XX.XX.XX:32492-CC.CC.CC.CC:40709                                                                                                      0.000       0.012 KB/sec
      ? root     XX.XX.XX.XX:13576-DD.DD.DD.DD:41831                                                                                                      0.000       0.012 KB/sec
      ? root     XX.XX.XX.XX:31990-DD.DD.DD.DD:40709                                                                                                      0.000       0.012 KB/sec
      ? root     XX.XX.XX.XX:50403-DD.DD.DD.DD:16234                                                                                                      0.000       0.012 KB/sec
      ? root     XX.XX.XX.XX:50925-CC.CC.CC.CC:16234                                                                                                      0.000       0.012 KB/sec
      ? root     XX.XX.XX.XX:14046-CC.CC.CC.CC:41831                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:50402-DD.DD.DD.DD:16234                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:32491-CC.CC.CC.CC:40709                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:31989-DD.DD.DD.DD:40709                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:13575-DD.DD.DD.DD:41831                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:50924-CC.CC.CC.CC:16234                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:50401-DD.DD.DD.DD:16234                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:14045-CC.CC.CC.CC:41831                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:32490-CC.CC.CC.CC:40709                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:31988-DD.DD.DD.DD:40709                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:50923-CC.CC.CC.CC:16234                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:13574-DD.DD.DD.DD:41831                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:111-EE.EE.EE.EE:8088                                                                                                         0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:31987-DD.DD.DD.DD:40709                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:14044-CC.CC.CC.CC:41831                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:32489-CC.CC.CC.CC:40709                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:50400-DD.DD.DD.DD:16234                                                                                                      0.000       0.000 KB/sec
      ? root     XX.XX.XX.XX:13573-DD.DD.DD.DD:41831                                                                                                      0.000       0.000 KB/sec
142
1 + 4
cn flag
Håkan Lindqvist
If we assume that the server has not been compromised: what UDP-based services are you running? DNS? NTP? Something else?
1
Reply
diya avatar
la flag
diya
*"so what I'm doing wrong?"* becomes a guessing came when you don't explain how you configure your server, what services you've installed and enabled after you re-deployed and re-configured your server after the previous compromise.
2
Reply
diya avatar
la flag
diya
Does this answer your question? [How do I deal with a compromised server?](https://serverfault.com/questions/218005/how-do-i-deal-with-a-compromised-server)
2
Reply
HDR avatar
cz flag
HDR
@håkan-lindqvist I've updated my question to answer what you've just asked.
0
Reply
Score:1
Sabre avatar Server
cn flag
Sabre

Attack is relative, can they quantify what the "attack" is? Sending errant packets does not constitute an attack. The packet count and payload size is consistent. Since you own the server instance, you could do a packet dump of the data leaving your server with wire shark, and it would not take long to narrow down what is generating it. Using something like nethogs would dig it out very fast as well, as it would show the PID of the offending process.

sudo apt install nethogs

Then just run "nethogs" in a terminal and watch....

+ 5
HDR avatar
cz flag
HDR
I've installed nethogs and I found plenty of connections from random ports on my end to random hosts on random ports in their end, like what's mentioned here https://unix.stackexchange.com/questions/159273/tons-of-unknown-connections-in-nethogs
0
Reply
Sabre avatar
cn flag
Sabre
IT should show you the PID of that process, which should lead you to what is causing it. Then you can configure whatever it is appropriately, or thwart whatever is running awry. if you only have console access not GUI, you can use TCPdump to get a packet capture as well, and see what the packets actually contain, that would indicate a lot more if the PID is known but does not make sense why it is behaving that way. The random source ports (your end) is normal behavior and how the IP stack works. Google ephemeral ports for more info on that.
0
Reply
Sabre avatar
cn flag
Sabre
Curious, what services do you have exposed to the internet? IT could be you are the passive victim of an amplification attack. In which case your system is not the true origin of the attack, bots or people spoof IP addresses to things like time servers, to have the reply go to another address other than the source. That could produce behavior like this, and being that it is targeting port 80 it may be someone leveraging you in a DDOS against someone else's web server?
0
Reply
HDR avatar
cz flag
HDR
when I started nethogs it didn't show me the PID of processes, as you can see in the updated question (FYI, I have executed it with sudo). As far as I know, the only exposed services on the internet are apache2 and SSH (I've even checked by running a Nmap scan on the server). For the SSH, I accept only public keys authentication and the SSH port is not the default one. For Apache, I've followed the instruction of DigitalOcean that I've putted in the question.
0
Reply
Sabre avatar
cn flag
Sabre
If I am not mistaken, that would mean that the process is the kernel, someone may have to correct me if wrong there. So then lets examine the packets themselves. Can you get a packet capture of the packets leaving the system? Filter protocol and port, capture soem of those packets and lets look at what they are for more clues. install tcpdump if you do nto already have it, then 'tcpdump -i [your interface goes here] udp port 80 -w [filename to write here]", you can then transfer that to another machine, and view in wireshark, or provide the packet capture for analysis.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.