Running out of open files with sshd

The Archetypal Paul

6/21/24, 11:52 AM

I have a Java application running in WSL2

I also have an ssh connection (in Windows) to a app server, and tunnel ports from the app server for a web server hosted on a second app server back to my application.

The application makes many connections to the web server as it loads pages. I am pretty confident it closes the connections correctly.

And yet, after a while, I get many "too many open files" errors printed on the terminal of the ssh session.

I've increased the open files ulimit from 1024 to 524288 (and confirmed this with /proc//limits). I am confident the application does not open that many connections!

It looks to me like sshd is not closing the connections. If I run the application, and then re-run it, I get "too many open files" more quickly which suggests the ones from the previous execution are still open in sshd.

Is my diagnosis correct? Is there anything I can do about it? I can't see anything in sshd cofiguration files etc that would impact this.

(Windows 10/WSL2 on the client side, Ubuntu 22.04 on the app server side)

EDIT: I ran the application on the server I was ssh'ing to. It worked, without running out of open files. So the application use of connections is probably not the issue. I suspect for some reason the sshd process that's opening the tunneled connections has some lower limit on open files still

1 + 0

ubuntu

ssh

windows-subsystem-for-linux

Score:2

Server

buchanan

6/21/24, 7:48 PM

A ssh tunnel creates channels for each connection and needs a file descriptor for each on both the client and on the server side. You may need to adjust the max open file limit on your application WSL environment.

lsof is a fantastic tool for easily examining the open files for a process lsof -c ssh to list files for all ssh processes lsof -p $PID to check a specific process ID

Additionally to exclude file types that don't contribute to open file limits. The following command would list all open files for all ssh processes excluding those that don't contribute to the open file count:
lsof -c ssh -d '^cwd,^err,^ltx,^mem,^mmap,^pd,^rtd,^txt' -a

You may be able to use a ssh watchdog like autossh to restart your tunnel when needed

+ 1

The Archetypal Paul

6/22/24, 7:43 AM

Thanks. But; "You may need to adjust the max open file limit on your application WSL environment." that's not where I'm running out of open files so I don't see how that helps. Also, see edit to the question

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Running out of open files with sshd

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.