Different ulimit between Ubuntu on WSL when in the shell vs. calling WSL from Windows command prompt

Sean McKenzie

10/19/23, 8:43 PM

I am attempting to use a Linux based binary called tippecanoe to do some vector map tiling. My computer runs on a Windows 10 operating system.

To integrate with the rest of my mapping functions, I am using R 4.2.0 and calling Tippecanoe through the mapboxapi package for R. This package has a function that makes a system call to tippecanoe. For this to work in Windows, I have to make the system call to WSL. I keep getting an error from Tippecanoe that it is unable to open database files.

Following advice on the Github page for Tippecanoe, I found out that this was a result of having the number of open files limit too low. Following the advice here, I was able to reset my hard and soft nofile limit to 65535 when working in the Ubuntu shell. I confirmed in Ubuntu with ulimit -n after closing and relaunching Ubuntu.

However, if I attempt to make a call from the Windows command prompt with wsl ulimit -n, I somehow get the default value of 1024, instead of the updated values of 65535. I think this must have something to do with WSL configuration. I took a look at how to configure /etc/wsl.conf to configure the default user, but that didn't make a difference.

Is there a way to change my WSL configuration so that a command from the Windows command prompt will respect the ulimit that I set in Ubuntu?

352

0 + 8

command-line

bash

windows-subsystem-for-linux

matigo

10/19/23, 9:52 PM

If you are running Ubuntu 20.04 in WSL, then there is no real solution. Microsoft says it’s fixed and won’t take “no” for an answer. That said, [this little hack](https://github.com/microsoft/WSL/issues/1576#issuecomment-643804974) may help. Be warned that doing this may affect you elsewhere; it breaks VSCode integrations, for example. An alternative would be to install Ubuntu in a virtual machine such as HyperV or VirtualBox, where you can then modify the upper limit to any value (so long as you have enough memory for the larger file buffers).

NotTheDr01ds

10/20/23, 1:47 AM

Could you confirm a few things for me? First of all, would you confirm whether Ubuntu is running as WSL1 or WSL2 via a `wsl -l -v`? Second, would you confirm again that your `ulimit -Sn 65535` really survived exiting the shell and restarting? My understanding is that `ulimit` is a shell construct, and only changes the limits in the *current* shell. Exiting and restarting should have reset it back to the default soft limit of 1,024. With those two pieces of information, I think I can get you an answer.

NotTheDr01ds

10/20/23, 1:51 AM

@matigo *"Microsoft says it’s fixed and won’t take “no” for an answer."* -- Are you sure you aren't thinking about WSL1? If you are referring to either [#1576](https://github.com/microsoft/WSL/issues/1576) or [#1688](https://github.com/Microsoft/WSL/issues/1688), then those were WSL1 issues that were closed. As mentioned by therealkenc in [this comment](https://github.com/microsoft/WSL/issues/1576#issuecomment-717509966) when it was closed, *"Full support for ulimit and prlimit is available in WSL2."*

Sean McKenzie

10/20/23, 11:26 PM

@NotTheDr01ds, I am running WSL2 (confirmed with call to `wsl -l -v`). After exiting Ubuntu, restarting my computer, and relaunching Ubuntu, `ulimit -n` returns the desired 65535. Interestingly, following the advice @matigo pointed me to, when I changed prlimit, and ran `wsl -d Ubuntu -lc 'ulimit -n'` in widows command prompt, I got the desired 65535, however, if I change the second flag, so that I run `wsl -d Ubuntu -c 'ulimit -n' I get the default 1024. I hackishly got R to send this command to WSL, but I still had the "can't open database files" error, so maybe it's something else?

NotTheDr01ds

10/21/23, 12:50 AM

@SeanMcKenzie Hmm. Can you `grep -ri ulimit ~`? Is there a `ulimit` statement in one of the startup files such as `~/.bashrc` perhaps?

Sean McKenzie

10/21/23, 3:58 PM

@NotTheDr01ds, Here is the output of `grep -ri ulimit ~` : /root/.bash_history:ulimit -n /root/.zcompdump:'ulimit' '_ulimit' /root/.zcompdump: _ulimit _unhash _user_math_func _value _vared \. . I also checked `sudo nano ~/.bashrc`, but there is no call to `ulimit`

NotTheDr01ds

10/21/23, 5:13 PM

@SeanMcKenzie Hmmm. I'll keep noodling on this. Also a curiosity question -- What does `wsl -e bash -lic "ulimit -n"` show? Same as, or different than, `wsl ulimit -n`? Thanks!

Sean McKenzie

10/21/23, 7:07 PM

@NotTheDr01ds `wsl -e bash -lic "ulimit -n"` gives 65535, `wsl ulimit -n` gives 1024. But happily, I found the solution to my issue. As it turns out, the reason tippecanoe was unable to open the database files, was actually unrelated to the `ulimit`, but rather because it was inheriting a Windows path to the the database files from R and trying to use that path in the linux command line. I did a little reworking of the R code to change the output and input file path sent to tippecanoe to be the linux paths, and now tippecanoe works!! Thanks so much for all of your help.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Different ulimit between Ubuntu on WSL when in the shell vs. calling WSL from Windows command prompt

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.