Suspicious users with numbers are devouring entire CPU

Davood Falahati

10/16/22, 7:49 AM

On my test server which I have docker-run gitlab-ce, redis server and some other important services, I noticed I have an uninvited guest, kdevtmpfsi. I tried everything proposed by the community but I see kind of intelligence in this one.

I runs some processes under nonexisting users, it started by gitlab-+, but I killed all the processed with this user. Now, I see a different behaviour. It runs some processes under some users with numbers, 998, 997, 996, etc.

All the commands they run are not existing on my machine. I don't have a local postgres, redis-server,gitlab-exporter etc.

28741 999       20   0 2873420 2.289g      0 S 331.8 29.4   1:31.19 kdevtmpfsi

Can anyone help?

205

1 + 4

linux

malware

docker

A.B

10/16/22, 8:18 AM

kdevtmpfsi is probably a coin miner, but your screenshot (please use text instead) doesn't display it. Are you asking a question because 1/ you had a kdevtmpfsi but don't have it anymore? 2/ you have a running gitlab inside docker but expect it's not running anymore? 3/ something else?

A.B

10/16/22, 8:21 AM

Anyway the answer is probably there: https://serverfault.com/questions/218005/how-do-i-deal-with-a-compromised-server

Pit

10/16/22, 9:50 AM

Please check [Gitlab Application Architecture](https://docs.gitlab.com/ee/development/architecture.html#simplified-component-overview), it may be a compromised server or it might not. Gitlab is a fantastic piece of software but it's really heavy, multiple other servers are involved (like Redis, PostgreSQL, ...)

Håkan Lindqvist

10/16/22, 9:57 AM

You mentioned docker. Are you sure these are not processes in containers which have a different set of users, unknown to the host?

Score:1

Server

Simon Richter

10/16/22, 12:08 PM

There are two things happening here:

There is indeed a miner running. Googling for kdevtmpfsi gives a lot of results.
It is likely that this is happening inside a container, so the numerical UID and that the file doesn't exist on the host are both normal.

So, likely one of the containers got compromised. Whether they broke out of it is unknown.

I'd bet on "no", because it is extra effort and more chance to get caught (container hosts have a lot better security than containers) and doesn't gain them much -- this is a fire-and-forget miner that they will not contact again, when it is shut down, not much is lost.

Still, you can't be sure, so the proper and diligent thing to do would be to nuke the site from orbit.

0 + 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Suspicious users with numbers are devouring entire CPU

TH: ผู้ใช้ที่น่าสงสัยที่มีตัวเลขกำลังกิน CPU ทั้งหมด

RO: Utilizatorii suspecți cu numere devorează întreg CPU

RU: Подозрительные пользователи с номерами пожирают весь ЦП

VI: Người dùng đáng ngờ với những con số đang nuốt chửng toàn bộ CPU

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.