Score:0

nmap shows 8080/tcp open http-proxy on centos 7 VPS but lsof shows nothing for that port

za flag
Tim

Im running a Centos 7 VPS server for the past several years. Originally when I set it up I didnt install any firewall (iptables specifically) because I was under the impression the host (linode) needed them open to admin the box from their side. Yes I know, really dumb mistake. Other things like ssh, apache etc I locked down and hardened well along with using fail2ban and I keep things patched on a daily basis where required.

At this point I believe Im likely hacked after closing things up via iptables and locking things down. It's a LAMP box. I have no proof at this point just suspicion. Im the only user and things are working fine.

When I run nmap against the server now I have everything locked down the way I want it except I see the following open even though iptables should be blocking 8080 via a DROP:

sudo nmap -sS -O -p8080 x.x.x.x

Starting Nmap 7.94 ( https://nmap.org ) at 2023-06-09 19:52 MST
Nmap scan report for mydomain.com (x.x.x.x)
Host is up (0.018s latency).

PORT     STATE SERVICE
8080/tcp open  http-proxy
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Aggressive OS guesses: Linux 2.6.32 (94%), Linux 2.6.32 or 3.10 (94%), Linux 2.6.35 (94%), Linux 3.5 (94%), Linux 4.2 (94%), Tandberg VCS video conferencing system (94%), Synology DiskStation Manager 5.1 (94%), Linux 2.6.39 (92%), Linux 3.10 (92%), Linux 3.10 - 3.12 (92%)
No exact OS matches for host (test conditions non-ideal).

Running lsof and doesnt show me anything on port 8080 however I do see some rpcbind listed and am unsure if any of these could be related?

COMMAND   PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
rpcbind   451    rpc    6u  IPv4  25873      0t0  UDP *:sunrpc 
rpcbind   451    rpc    7u  IPv4  25922      0t0  UDP *:compaq-evm 
rpcbind   451    rpc    8u  IPv4  25923      0t0  TCP *:sunrpc (LISTEN)
rpcbind   451    rpc    9u  IPv6  25924      0t0  UDP *:sunrpc 
rpcbind   451    rpc   10u  IPv6  25925      0t0  UDP *:compaq-evm 
rpcbind   451    rpc   11u  IPv6  25926      0t0  TCP *:sunrpc (LISTEN)

From the outside I can telnet x.x.x.x 8080 and it responds, however trying to use a common GET HTTP/1.1 command doesnt reply at all and it times out after a minute or so.

When I telnet in on port 8080, I run lsof | grep http and I see the following:

lsof | grep http
NetworkMa   514          root  mem       REG                8,0    162144       5038 /usr/lib64/libnghttp2.so.14.17.0
gmain       514   517    root  mem       REG                8,0    162144       5038 /usr/lib64/libnghttp2.so.14.17.0
gdbus       514   519    root  mem       REG                8,0    162144       5038 /usr/lib64/libnghttp2.so.14.17.0

I dont run nghttp2 so Im guessing that it part of Linode admin package they preinstalled?

Any ideas on how to find this process and kill it? A bonus would be understanding what it is as well but Id be happy just to get rid of it.

Thank you for any advice or pointers.

Ginnungagap avatar
gu flag
Does this answer your question? [How do I deal with a compromised server?](https://serverfault.com/questions/218005/how-do-i-deal-with-a-compromised-server)
Tim avatar
za flag
Tim
This is a little helpful, though much of it I already know. I'll have to re-install for sure but would like to learn forensically from this mistake and understand it better and hopefully plug this particular hole until I can find time to re-install.
Score:1
re flag

looks like lsof doesn't show sockets owned by user who created that 8080 binding.
From this output it says NetworkMa[nager?] with pid=514 (if I got it correctly without header)

lsof | grep http
NetworkMa   514 ...

Does this service restart automatically? pstree can show a chain with that process (probably it can clarify a bit)

p.s. cannot put it in comments, so just added in answers

Tim avatar
za flag
Tim
Yes, NetworkManager starts automatically. I didnt set it up, it came installed on the machine. I looked at pstree and didnt see anything that stuck out as being the process thats causing this. I know Ill have to re-install the machine at some point I just dont have several days to do so at this point, so plugging the hole is my best choice ATM (realizing that there might be others I dont know about).
yvs2014 avatar
re flag
yes, maybe it's the right choice to be sure
yvs2014 avatar
re flag
another one suggestion to get some extra info if that NetworkManager is problematic: compare checksums of the local /usr/sbin/NetworkManager and its original binary (extracting this file from .deb package of the same version)
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.