Score:2

Ubuntu

`apt-mark showmanual` shows almost all packages. messed up?

Albert

4/27/23, 9:43 AM

apt-mark showmanual lists a lot of packages for me:

root@hcompneu:~# apt-mark showmanual | wc -l
4013
root@hcompneu:~# apt-mark showmanual | head
a11y-profile-manager-indicator
abiword
abiword-common
abiword-plugin-grammar
account-plugin-facebook
account-plugin-flickr
account-plugin-google
account-plugin-identica
account-plugin-twitter
accountsservice
root@hcompneu:~# apt-mark showmanual | tail
yelp-xsl
zeitgeist
zeitgeist-core
zeitgeist-datahub
zenity
zenity-common
zip
zlib1g
zlib1g-dev
zlib1g:i386

As I understand, it should only list those packages which I manually installed. I don't remember in each case anymore but I'm pretty sure I did not install all of those manually. Actually, I rarely installed any software on this system, despite Chrome, OpenOffice and some other very basic things. So I would expect a list of maybe 10-100 packages but not more. Maybe it also includes ubuntu-desktop, grub and linux or so from the base Ubuntu installation.

Maybe in the past though I wrote some scripts to fix up a broken system by automatically calling apt-get install XXX on various packages. Or maybe I did other things. I don't really remember anymore.

Could that be the case here? So I somehow messed it up? If that is the case, what would you suggest? Mark all to automatically installed, and then by hand add some selected packages like ubuntu-desktop?

Or is such output correct and I understand it wrongly? I read this and I'm not sure. It looks like it is much more complicated to get the list of manually installed packages? Or why is the answer so complicated?

Or is apt-mark showmanual buggy?

root@hcompneu:~# ls -ld /var/log/installer/
drwxr-xr-x 2 root root 4096 Dez 27 11:27 /var/log/installer/
root@hcompneu:~# grep -r cdrom /etc/apt/ --include=*.list
root@hcompneu:~# dpkg -l | grep ^ii | wc -l
3812

267

0 + 0

package-management

apt

nobody

4/27/23, 11:15 AM

Any chance you use this update-applet (i never know the name) to update your system?

0

Reply

N0rbert

4/27/23, 1:10 PM

How old is your system? Please show output from `ls -ld /var/log/installer/` and `grep -r cdrom /etc/apt/ --include=*.list` and `dpkg -l | grep ^ii | wc -l`. On my 7 year system I have 3953 manually installed packages of 5680 in total; on my 11 year system I have 3947 manually installed packages of 6395 in total. Some of them were installed by me, some - as dependencies. So there is no reason to be afraid.

3

Reply

Albert

4/27/23, 2:13 PM

@N0rbert But those installed as dependencies should not be in that list. Only those which you directly installed via `apt-get install XXX`, not the dependencies. Or actually that is my question, if this is the case. That is my understanding at least.

0

Reply

N0rbert

4/27/23, 2:34 PM

I was mistaken about dependencies, they are not marked as manually installed. But you do not answer the questions and do not provide the requested command output. We both share a love to Gentoo, but please provide additional info.

0

Reply

Albert

4/27/23, 3:15 PM

@N0rbert I added the outputs of the commands. I don't really know how old the system is but I think it is old. Maybe started with Ubuntu 8 or so and went through 10, 12, 14 and 16, while I recently updated to 18. However, despite Chrome, OpenOffice and some other very basic things, I never installed anything manually. So definitely most things in the list were not manually explicitly installed by me. So maybe the answer to my question is simply *yes, it is messed up*?

0

Reply

Score:4

Ubuntu

N0rbert

4/27/23, 3:34 PM

TL;DR No, your system is not messed up. Moreover, you can leave all things as they are, so in the current state.

Let's consider three systems:

Current System	Installation media	Age	`dpkg -l \| grep ^ii \| wc -l`	`apt-mark showmanual \| wc -l`
Ubuntu 18.04.6 LTS with MATE desktop(*)	Ubuntu 12.04.4 LTS	7 y.o.	5680	3953
Ubuntu 18.04.6 LTS with MATE desktop(*)	Ubuntu 10.04 LTS	11 y.o.	6395	3947
Yours Ubuntu 18.04 LTS (**)	~Ubuntu 8.04 LTS	~13 y.o.	3812	4013

Notes:

(*) my systems were originally installed from Ubuntu desktop installation disks, thus shipped with GNOME and/or Unity (not a flavour). Then I have installed MATE desktop using official PPAs for 12.04 LTS, 14.04 LTS and their meta-packages; then when 16.04 LTS were released I have re-installed MATE desktop as task package using commands like sudo apt-get install ubuntu-mate-desktop^. This installation of task package causes all such packages to be marked as manually installed and thus protects them from removal. All other software were installed manually during life-span of the both systems.
(**) the strange thing about your system is that you have total number of installed packages 3812 less than manually installed packages 4013. You have to investigate this using Synaptic.

I tried to conduct some research with fresh installations of old and new systems. Below is the table about clean installations of ordinary Ubuntu desktop ISOs.

Ubuntu version	`dpkg -l \| grep ^ii \| wc -l`	`aptitude search '~i!~M' \| wc -l`	`apt-mark showmanual`	Synaptic: Status → Installed (manual)
6.06 LTS	1046	1046	n/a	n/a
8.04 LTS	1102	1087	n/a	n/a
10.04 LTS	1296	1286	n/a	1286
12.04 LTS	1392	1367	1367	1342
14.04 LTS	1722	1690	1690	1666
16.04 LTS	1745	220	220	195
18.04 LTS	1602	1557	1557	1529
20.04 LTS	1607	49	49	49
22.04 LTS (dev)	1700	44	44	31

We can understand here that functionality of Ubiquity installer and APT by itself have changes over time.

From the above table we can see that old systems have huge numbers of manually installed packages from their day 0 of installation.
Newer Ubuntu versions have smaller numbers, this ease their maintenance. They probably have more meta-packages installed which prevent package removal because of dependencies.

Another experiment I have done: I upgraded fresh Ubuntu installation from 8.04 LTS to 10.04 LTS → 12.04 LTS → 14.04 LTS → 18.04 LTS → 20.04 LTS and in the resulting 20.04.3 LTS system I have 1867 manually installed packages of 2913 total packages.

The "manual" installation status of package prevents its removal, so you should not set package as auto-installed.

Good resources to read:

Chapter 2. Debian package management of Debian Reference.
man apt-mark locally or online, the below quote is essential:

apt-mark - show, set and unset various settings for a package
[...]
showmanual can be used in the same way as showauto except that it will print a list of manually installed packages instead.

0 + 0

Albert

4/27/23, 4:06 PM

I wonder, do you really have installed all those almost 4k packages manually? Or is it maybe also messed up in your case? As said before, dependencies should not be in that list.

1

Reply

N0rbert

4/27/23, 4:24 PM

I did some experiments on fresh Ubuntu MATE 22.04 LTS daily VM - it has the following numbers `apt-mark showmanual | wc -l` returns 30 (I suppose that these packages were installed by Ubiquity to support needed locales and so on), while total number of packages is 1971. Installation of each next package increase the list of manually installed packages by 1 (as expected). For 18.04 LTS it needs further investigation.

0

Reply

Someone

4/28/23, 5:13 AM

@Albert Yes, it is possible to have more than 4k manually installed packages. For eg:- If N0rbert installs kde desktop (or any desktop) they will also install various apps with it (around 100!) And all those apps will also be shown in the manually installed list. So it doesn't mean that having 4k manually installed packages means, you have run `apt install` 4k times! (The suggested application packages and other extra packages (not including dependencies) are also manually installed packages ;)

0

Reply

Someone

4/28/23, 5:15 AM

@N0rbert How did you installed Ubuntu Mate 22.04? Is it released?

0

Reply

N0rbert

4/28/23, 7:31 AM

@someone I have installed Ubuntu MATE 22.04 LTS for testing purposes from http://cdimage.ubuntu.com/ubuntu-mate/daily-live/current/ using daily ISO.

0

Reply

Albert

4/28/23, 12:55 PM

@someone No, if you do `apt-get install kubuntu-desktop`, that will install a lot of packages, but only one single package will be added to the manual list, namely `kubuntu-desktop`. All the others will be marked as auto, as they are only dependencies.

1

Reply

Someone

4/28/23, 1:28 PM

@Albert Is Konsole a dependency for Kubuntu desktop?

0

Reply

Albert

4/28/23, 5:05 PM

@someone: I don't really know. Why is that relevant? See [here](https://packages.ubuntu.com/focal/kubuntu-desktop) for direct dependencies but there are much more indirect dependencies. It is for example a dependency of the [`kde-full`](https://packages.ubuntu.com/focal/kde-full) package.

0

Reply

Someone

4/29/23, 3:21 AM

@N0rbert Can you look at https://askubuntu.com/a/1384019/1460940. Is it true? Please see the comments also. Am I missing something? Aren't all packages installed during the installation of OS marked "non-auto" or "manual" by the installer, so that they don't get removed by `sudo apt autoremove`?

0

Reply

N0rbert

4/29/23, 8:16 AM

During Ubuntu 18.04 LTS installation many packages are marked as manual. In newer versions only bunch are marked as manual. For me it seems that there some changes occur in the internals of APT for last 3 years.

0

Reply

Albert

4/30/23, 1:18 PM

You now changed the answer quite a bit, and reversed the meaning. Now from your description, it sounds like the system is messed up, but somehow you get to the conclusion it is not messed up? It also looks like the fresh installation of Ubuntu <=18 is messed up, as it should not list such a big list of manual installed packages. If almost all packages are listed as manual installed, that should not be, i.e. is messed up. But somehow you conclude just because that is how it is, that it is not messed up?

0

Reply

Albert

4/30/23, 1:20 PM

To add, I just found [this](https://askubuntu.com/questions/12279/why-are-almost-all-packages-marked-as-manually-installed), and there is indeed a bug report about this behavior, [bug/424643](https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/424643).

0

Reply

N0rbert

4/30/23, 1:22 PM

I conducted comprehensive analysis to understand why it goes this way. I use such a messed system for years. There is no reason to worry.

0

Reply

Albert

4/30/23, 1:36 PM

Well, the whole point of this functionality is that `autoremove` works, also in case of further updates where some packages might not need some other dependencies anymore. So you say, I should not care that `autoremove` will not work, at least for those packages by the base installation? Also, I fear that it could break some distribution upgrade later. I already had a couple of problems due to packages which had problems in the upgrade which I did not really need at all. For example it wanted to have `upstart` together with `systemd`.

0

Reply

N0rbert

4/30/23, 2:07 PM

Then you have to describe exact "problems in the upgrade" first. Upstart and Systemd may stay in the system if there are packages depending on them. You can run `aptitude why upstart` to understand why this package is installed. Currently I do not understand exact problem in your system.

0

Reply

Albert

5/4/23, 7:46 AM

Upstart also stays in the system when it is wrongly flagged as manual, which was the case, which is the exact problem I'm describing here. Only because Ubuntu 8 or whatever flagged **all** packages as manually installed does not make it correct, or not messed up.

0

Reply

Score:1

Ubuntu

ubfan1

4/29/23, 10:29 PM

The output of the showmaual option includes any package that ever got updates, so not really what you want. The below command I copied from some other answer (don't recall where or I'd attribute it), and compares the output against the initial install list, producing the packages you installed yourself. Put it into a file in your bin area for convenience.

comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)

The first time I ran apt-mark showmanual, I knew the list was too big because things like "tar" were included, which I knew I had never manually touched. "tar" however had had an update applied, which caused it to be output. Since "tar" was included in the initial system packages, it gets removed by the script, just leaving things that "showed up" after the initial system installation.

0 + 0

Albert

4/30/23, 1:09 PM

"showmaual option includes any package that ever got updates" - I don't really understand what that means. As I understood it, it ends up in this list exactly iff I did `apt install X`, which is thus exactly what I want.

0

Reply

Score:0

Ubuntu

Tom V

4/27/23, 6:48 PM

The system is not "messed up", but basically your understanding of what manually installed means is incorrect. A package is never actually marked manual or not-manual. It can only ever be marked auto or not-auto.

A package is marked auto when it is installed automatically to satisfy a dependency. This means that it can (and implies that it should) be removed automatically when no other package depends on it any longer.

On a normal desktop system the majority of packages are installed when the OS is first installed by the installer. These packages have to be installed not-auto, or else they would be removed because nothing depends on them.

Take for example 'abiword' near the top of your list. This is a word processor that is probably quite normal to have installed on a desktop system. I assume that you used one of many flavours of desktop installers. It probably didn't ask you specifically if you wanted abiword. You probably didn't install abiword yourself by name. You still chose to have it by choosing the install medium that you used.

Since there is no other package on your system that depends on you having abiword installed in order to work, it cannot be marked auto or else it would be removed. This means it is not marked auto. apt-mark showmanual lists the packages that are not marked auto.

Note also that many installers install everything non-auto even if they could mark them auto because something depends on them. Even the important core packages that hundreds of things depend on get marked non-auto. In this sense, maybe the installer is "messsed up", but it is nothing special about your system.

My best attempt at getting a list of things I might have installed myself is:

aptitude search -F '%p' --disable-columns '~i !~M !~E !~prequired !~pimportant !~pstandard !~tminimal !~tstandard'

To exclude packages that are depended on, use:

aptitude search -F '%p' --disable-columns '~i !~M !~R~i !~E !~prequired !~pimportant !~pstandard !~tminimal !~tstandard'

0 + 0

Albert

4/27/23, 7:28 PM

I don't exactly understand. Your description matches my understanding. Yes, `abiword` in that list is maybe sth I installed manually. However, definitely not `abiword-common`, `yelp-xsl ` or `zeitgeist` or so. Those should all be marked as auto because they were probably installed to satisfy a dependency.

1

Reply

Tom V

4/27/23, 9:04 PM

The point I am making is that if you run apt-mark showmanual on a brand new freshly installed system it will list many hundreds of packages because the system install media mark everything as non-auto so that it doesn't get removed.

1

Reply

Albert

4/27/23, 9:46 PM

No, that is not true. On a fresh install, showmanual only lists about 30 packages, not more. Not more are needed. You just need ubuntu-desktop, linux, and a few others. All the remaining ones are then dependencies.

0

Reply

Someone

4/28/23, 5:20 AM

@Albert No, that is not true. On a fresh VM of Ubuntu 21.10, I can see almost 150 packages. Also, all the applications such as firefox and gedit are on the list. So, preinstalled applications are also manually installed. I agree with TomV. +1

0

Reply

Tom V

4/28/23, 6:50 AM

150 isn't 4000 though, I would have expected more. Maybe it depends on the installer version. The real question is: are any packages marked auto on this system?

0

Reply

Someone

4/28/23, 7:34 AM

@TomV No... Off course not. The number of packages depend on minimal and full install (I was telling about minimal install). If the packages were marked as auto then a simple `apt autoremove` will delete the entire system!

0

Reply

Albert

4/28/23, 12:53 PM

Surely on a fresh install, there are many packages marked as auto. Much more than those marked as manual. The number 30 of manual packages is from N0rberts comment above for a fresh Ubuntu MATE 22.04 LTS daily VM. I still don't exactly understand the discussion here. In my original post, I'm quite sure there are packages which I have never installed manually and which are also not marked as manual on any fresh installation.

0

Reply

Tom V

4/28/23, 5:27 PM

The last time I ran a release installer (ubuntu 21.04 mate) every package flagged essential or important and every package with priority higher than optional was marked non-auto by the installer, even libraries that have no purpose unless you install something that depends on them. As well as this the mate gui packages were non-auto too. showmanual listed many many hundreds of packages.

0

Reply

Albert

4/30/23, 1:24 PM

@TomV I just found [this](https://askubuntu.com/questions/12279/why-are-almost-all-packages-marked-as-manually-installed), and there is a bug report saying that this is wrong and buggy behavior, [bug/424643](https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/424643). Also, see the [answer above](https://askubuntu.com/a/1384000/6878) which indicates that this is not a problem in Ubuntu 20 and 22 anymore?

0

Reply

Tom V

4/30/23, 9:39 PM

That question and bug report are over 10 years old (but I think you said your system is very old too). I've definitely seen this much more recently on Debian and Ubuntu.

0

Reply

Score:0

Ubuntu