Score:8

Is it possible to run a WSL app in the background?

bz flag

I installed LXD in ubuntu with the snap store and then conncted it to the windows client, but for me to be able to run containers from Windows the Ubuntu app has to be open. Is there a way to have the LXD server run in the background or will I have to have Ubuntu open for it to work? wsl --status returns:

Default Distribution: Ubuntu
Default Version: 2

wsl --version returns:

WSL version: 0.70.0.0
Kernel version: 5.15.68.1
WSLg version: 1.0.45
MSRDC version: 1.2.3575
Direct3D version: 1.606.4
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22000.1098

and cmd /c ver returns:

Microsoft Windows [Version 10.0.22000.1098]
ChanganAuto avatar
us flag
Although is looks and behaves like an "app", it's actually a full-fledged albeit minimal OS running in a way not dissimilar to a virtual machine. Knowing that the answer to your question should be obvious.
Luxvoo avatar
bz flag
Well yes, but I can’t seem to find how to. It clearly isn’t working if the app is closed?
ChanganAuto avatar
us flag
Well, that's because the obvious answer is the opposite of what you think it is. And again, Ubuntu is NOT an app.
Luxvoo avatar
bz flag
What about when I use Rancher desktop? It uses WSL to run containers and I know that I don’t have to have the custom distro open. It also seems that a VM should be able to run in the background (without any GUI/cli). There is a high chance that I don’t understand how VMs work, but I have no idea why VMs couldn’t run in the background. Again I’m new to this and I’m not qualified to say anyhing. Thanks for the help I guess.
ChanganAuto avatar
us flag
Rancher Desktop is (quoting) *an open-source desktop **application** for Mac, Windows and Linux. Rancher Desktop runs Kubernetes and container management on your desktop.*
ChanganAuto avatar
us flag
You're confusing software that runs in an OS (Ubuntu in this case, could be any other Linux distro) with the OS itself and the layer that allows the OS to run inside another OS (WSL).
Luxvoo avatar
bz flag
Well I need the LXD server to be avaliable when the cli for Ubuntu is closed. Not when WSL is completely shut down. I see absolutely no reason why closing the cli would shut down Ubuntu thus I don’t see why the server is not avaliable. The only thing you’ve told me is that it doesn’t work. But I would greatly appreciate it If you explain why this could never work.
ChanganAuto avatar
us flag
LXD runs under Ubuntu therefore you need Ubuntu running, end of story.
Luxvoo avatar
bz flag
The fact that I still have a question will bother you, but didn’t Microsoft add deamon support after the last bash.exe process has been killed? Shouldn’t the lxd deamon continue running in the background even if the bash process has been killed, please note that Ubuntu shouldn’t close if you have a deamon running in the background, at least thats what I understood from an arcticle I found?
ChanganAuto avatar
us flag
1. `bash.exe` has nothing to do with Ubuntu (the .exe should be clue enough); 2. Please understand this quite obvious situation: *when the cli for Ubuntu is closed*, **Ubuntu is closed** (granted, maybe not obvious for you because you're very confused). Then 3. anything you install in Ubuntu needs Ubuntu running, the same way any software installed in the guest OS (VM) CANNOT run by itself in the host OS. There's absolutely nothing else to understand here.
NotTheDr01ds avatar
vn flag
@Luxvoo Welcome to Ask Ubuntu! Can you provide your WSL and Windows version? The answer to your question probably depends a bit on this. `wsl --status`, `wsl --version`, and `cmd.exe /c ver`? Would you add that info to your question? Note that `wsl --version` may not work, depending on the actual version. Thanks!
Luxvoo avatar
bz flag
@NotTheDr01ds I added the versions. All of them ran. (I am on the preview version of WSL)
Luxvoo avatar
bz flag
@ChanganAuto The bash.exe does have to do a lot with ubuntu, It's the cli I think. But the point is when you close the cli (the last bash.exe proccess) it SHOULDN'T close as far as I'm concerned.
NotTheDr01ds avatar
vn flag
@ChanganAuto You know I answer a lot of Ubuntu-on-WSL questions, and trust me, this is a pretty *good* question. Also `bash.exe` is the legacy (now deprecated) way of launching Ubuntu on WSL. It was a horrible name, clearly, but that's why it was changed to `wsl.exe`.
Luxvoo avatar
bz flag
@NotTheDr01ds Okay thank you so much! Don’t rush it I have time. Again thanks for looking into it.
Luxvoo avatar
bz flag
@NotTheDr01ds I know how it feels, as I am a type of person who forgets to save their work and then loses everything lmao.
Score:7
vn flag

It's honestly a great question, as I realized when I went through the various scenarios below. The way that WSL works here is, I have to agree, confusing. There are a number of "issues" that come up related to this on the WSL Github repo, but the reality is that this behavior is "by design".

From the WSL/Systemd announcement:

It is also important to note that with these change, systemd services will NOT keep your WSL instance alive. Your WSL instance will stay alive in the same way it did before, which you can (read more about here](https://devblogs.microsoft.com/commandline/background-task-support-in-wsl/).

In other words, WSL is designed to terminate Ubuntu in certain situations, but the linked post isn't very clear about exactly what those scenarios are. To understand them (and how WSL behaves) a bit better, let's start out with the (at least) four different ways you can start a background task or service (a.k.a. daemon) on Ubuntu on WSL:

  1. Interactively, from the command line via the command itself or using the service helper. E.g.:

    sudo service cron start
    # or even
    sudo /usr/sbin/cron
    

    This, of course, will work on any version of WSL, but the SysVInit service may only work for certain distributions (such as Ubuntu) and services.

    (Important) note that commands run inside your shell startup scripts are considered "interactive" for this purpose. That will come in handy in my ultimate (hacky) solution.

  2. Interactively, as a background task from the shell. E.g.:

    nohup sleep 100000 &
    
  3. Under Windows 11, you can start a daemon automatically at WSL startup via /etc/wsl.conf. E.g.:

    [boot]
    command = service cron start
    
  4. And starting with WSL 0.67.6, using Systemd. This can happen either interactively (sudo systemctl cron start), automatically via /etc/systemd links, or as a dependency of another service. E.g.

    sudo systemctl cron start
    

    Note that the Snap daemon (snapd) starts automatically via Systemd once enabled in WSL.


With those scenarios in mind, we need to look at when WSL will terminate Ubuntu. As I said, it's "confusing". I wish I could come up with a summary statement to describe the behavior, but I struggle with it. The best I can do is provide examples:

Summary

  1. Will not terminate
  2. Will not terminate
  3. Will terminate
  4. Will terminate

Details and examples

  1. A service started interactively via the commandline will prevent WSL from terminating Ubuntu. We can demonstrate this by:

    • Temporarily disabling Systemd in /etc/wsl.conf
    • Exiting Ubuntu
    • Start PowerShell
    • wsl --shutdown
    • Restart Ubuntu via wsl ~ (if needed, wsl ~ -d Ubuntu-22.04, etc.)
    • sudo service cron start
    • Exit the shell
    • Wait a minute (25 seconds, by default, should be sufficient) and check wsl -l -v to show that Ubuntu is still in the Running state.
    • wsl -e ps axjff should show that cron is still running, even though no shell is running.
  2. A background task started interactively via the commandline will also prevent WSL from terminating Ubuntu. Following the previous example:

    • With Systemd still disabled, from PowerShell, wsl --shutdown
    • Start Ubuntu (e.g. wsl ~)
    • Run nohup sleep 100000 &
    • Hit Enter one extra time to dismiss the nohup message
    • Exit the shell/Ubuntu back to PowerShell.
    • Wait 30 seconds to a minute and wsl -l -v to show that Ubuntu is still running.
    • wsl -e ps axjff should show that the sleep command is still running, even though no shell is running.
  3. A service/process started via /etc/wsl.conf, on the other hand, will not prevent WSL from terminating Ubuntu.

    • Starting with the previous example (Systemd disabled)

    • In Ubuntu, sudo -e /etc/wsl.conf and add the following:

      [boot]
      command = service cron start
      
    • Exit Ubuntu, and from PowerShell, wsl --shutdown

    • Restart Ubuntu via wsl ~

    • ps axjff to show that cron is running as a subprocess of one of the /init parents.

    • Exit the shell/Ubuntu

    • After 30 seconds to a minute wsl -l -v will show that Ubuntu is no longer running. It has been terminated.

  4. A service/process started via Systemd will also not prevent WSL from terminating Ubuntu. Demonstration (although you are experiencing this):

    • Edit /etc/wsl.conf and re-enable Systemd while disabling (commenting out or removing) the command.

    • Exit Ubuntu

    • In PowerShell, wsl --shutdown

    • wsl ~ to start Ubuntu.

    • ps axjff will show the Systemd services running and/or starting up.

    • After a minute:

      sudo systemctl stop cron
      sudo systemctl start cron
      

      This has the effect, obviously, of simply stopping cron then restarting it. The point here is to show that even though we explicitly started cron interactively, this will not prevent WSL from terminating Ubuntu.

    • Exit the shell/Ubuntu

    • After 30 seconds to a minute, wsl -l -v will show that Ubuntu has been Stopped.

    As for your particular question, since lxd is simply a Snap, which is running in a containerized environment started by Systemd, it is getting terminated as well. Well, bummer ...

Solution

Hopefully there's a more permanent fix incoming, as one of the WSL developers did indicate a desire to keep distros running.

In the meantime, a solution (and there may, of course, be more) is to start a long-running process via your shell startup-files. In my case, I typically run keychain.

There are a few things about Keychain that allow it to work nicely in this case:

  • It's available in the default Ubuntu repositories
  • It already starts a singleton instance of ssh-agent. In other words, if ssh-agent is already running, it will attach to it rather than starting a new one.
  • It can run quietly, and without interaction if no keys are added

To set this up:

sudo apt update && sudo apt install keychain -y
sudo -e /etc/profile.d/keep_wsl_running.sh

And add the following:

#!/usr/bin/env sh
eval $(keychain -q)

This script:

  • Runs when you start your shell (assuming most shells such as Bash and Zsh). Note that it will not run if you use Fish as your shell. If you (or any future reader) does want to use Fish (or another shell that doesn't process /etc/profile.d), ask a separate question on how to handle this scenario. I do have a solution, but this answer is (as usual) long enough already ;-).

  • Evaluates the output of keychain, so that it finds any existing ssh-agent (or starts a new one, if not) and sets the environment variables for the shell to point to it.

  • -q, as you probably imagine, simply runs keychain quietly, without output

Having a the ssh-agent running (that was started via /etc/profile.d) will prevent WSL from terminating the Ubuntu distribution/instance/container. Ubuntu will continue running after you close the terminal window, and your lxd (and other daemons/services installed by Snap) will stay up as well.

To allow Ubuntu to terminate gracefully, just get rid of the ssh-agent. This can be done several ways, including:

keychain -k all

Of course, make sure that any other services that you started interactively are also gone. Once that condition is met, WSL will gracefully terminate Systemd and its units/services.

There's one corner case here, and that's if you start a wsl -u root session. In that case, ssh-agent will also be running as root, and you'll need to sudo keychain -k all.

Luxvoo avatar
bz flag
Thanks! Do you think that systemd support will be added some time in the future? I saw that one of the devs added the systemd label.
NotTheDr01ds avatar
vn flag
@Luxvoo Oh, Systemd is clearly supported (and you are using it). It's just that services started that way won't prevent WSL from closing. But by using the "hack" solution in my answer, even *Systemd* services will keep running until you `--terminate` or `--shutdown` (or kill the `sleep inf`). I'll edit a bit more on this to see if I can make it more clear.
Luxvoo avatar
bz flag
I meant that you wouldn't have to do the "hack" to keep them running you know (it would be nice if you just started them and then they would keep running even without the script that you wrote) EDIT: Also I understand this it's just that I can't phrase my questions right.
NotTheDr01ds avatar
vn flag
@Luxvoo No worries! And yes, at some point I think they are going to work on a better way of doing this. Personally, once they get the important Systemd bugs knocked out (and they seem to be making progress), I'm hoping that [9P performance](https://stackoverflow.com/a/68974497/11810933) is next on the list so that we can finally truly remove the need for WSL1.
Luxvoo avatar
bz flag
Yeah I've been experiencing that bug as well.
Luxvoo avatar
bz flag
Would compiling LXD from source make it work with service lxd start? Or does it need systemd? If it could be started via SysVInit wsl wouldn’t terminate it right? Is that a possibility? I think the package version of LXD (now not avaliable) worked by using SysVInit, but I’m not sure
NotTheDr01ds avatar
vn flag
@Luxvoo I'm not terribly familiar with LXD itself, but what I saw is that the prepackaged Linux versions (since 2019 or so) are only available via Snap. But yes, you can also install from source, and running just the daemon (as in method #1) *should* allow WSL to keep Ubuntu running.
Luxvoo avatar
bz flag
Ok I’ll try that and if it doesn’t work then I’ll use your option. Thanks for all the help. :)
Score:2
us flag
ws_

Run this vbs script on windows startup. Just put it in the windows startup folder (crl+r -> shell:startup)

wsl-startup.vbs:

# change '<Distro>' to the distro name you are using.
set ws=wscript.CreateObject("wscript.shell")
ws.run "wsl -d <Distro>", 0

form: https://github.com/microsoft/WSL/issues/8854#issuecomment-1421910739

After that, everytime you start your machine and login. It will start wsl and waiting for input. The wsl instance would keep alive.

Then you can use systemd to run whaterever you want in the background.

Rember to run the vbs again if you restart the wsl instance manually with wsl -t <distro> / wsl --shutdown

GammaGames avatar
in flag
This worked! I had to enable the docker service with `sudo systemctl enable docker`
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.