kafka + File handle leak due a large number of pipe/eventpoll leak

we have kafka cluster on linux rhel version - 7.6

we notice too many open files on kafka machines

but when we did lsof >/tmp/res_file

form file /tmp/res_file , we can see many lines as

java      172301 172722           kafka *770w     FIFO                0,9        0t0   303639156 pipe
java      172301 172722           kafka *771u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *772u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *773r     FIFO                0,9        0t0   303741913 pipe
java      172301 172722           kafka *774u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *775w     FIFO                0,9        0t0   303637483 pipe
java      172301 172722           kafka *776w     FIFO                0,9        0t0   306043296 pipe
java      172301 172722           kafka *777w     FIFO                0,9        0t0   313046116 pipe
java      172301 172722           kafka *778r     FIFO                0,9        0t0   304446234 pipe
java      172301 172722           kafka *779u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *780r     FIFO                0,9        0t0   303800963 pipe
java      172301 172722           kafka *781w     FIFO                0,9        0t0   303300350 pipe
java      172301 172722           kafka *782w     FIFO                0,9        0t0   305039138 pipe
java      172301 172722           kafka *783r     FIFO                0,9        0t0   304659538 pipe
java      172301 172722           kafka *784r     FIFO                0,9        0t0   303453024 pipe
java      172301 172722           kafka *785w     FIFO                0,9        0t0   303737027 pipe
java      172301 172722           kafka *786u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *787u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *788u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *789r     FIFO                0,9        0t0   303258041 pipe
java      172301 172722           kafka *790w     FIFO                0,9        0t0   303258041 pipe
java      172301 172722           kafka *791u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *792u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *793u  a_inode               0,10          0       11398 [eventpoll]
java      172301 172722           kafka *794r     FIFO                0,9        0t0   305651339 pipe

and when we did

cat /tmp/res_file | egrep "pipe|eventpoll" | wc -l

we get huge number of lines that contain pipe or eventpoll

cat /tmp/res_file | egrep "pipe|eventpoll" | wc -l
143173534

from my understand when lsof show lines that ended with pipe|eventpoll its indicate on a problem

but I still not understand what is the root cause for this

anyway we increased the number of files on kafka , but this isn't solution

since the number of files are increased every day due the lines with - pipe|eventpoll

any idea how to know the reason for this behavior?

reference:

https://stackoverflow.com/questions/58690422/kafka-producer-consumer-are-opening-too-many-file-descriptor

https://stackoverflow.com/questions/15956452/troubleshooting-too-many-files-open-with-lsof

https://access.redhat.com/solutions/4072141

https://issues.apache.org/jira/browse/KAFKA-3317