Score:1

ESM 401 Unauthorized error on Ubuntu 16.04

de flag

I want to upgrade my Ubuntu 16.04 server to Ubuntu 18.04 and I'm running the following commands to do so;

apt update -y
apt upgrade -y
do-release-upgrade

apt update command runs fine with the following output;

# apt update -y
Hit:1 https://esm.ubuntu.com/infra/ubuntu bionic-infra-security InRelease
Hit:2 https://esm.ubuntu.com/infra/ubuntu bionic-infra-updates InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
5 packages can be upgraded. Run 'apt list --upgradable' to see them.

However when running the apt upgrade command, it returns the following errors;

# apt upgrade -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done

*The following packages could receive security updates with UA Infra: ESM service enabled:
  libkrb5-3 libgssapi-krb5-2 libk5crypto3 libkrb5support0 libzstd1
Learn more about UA Infra: ESM service for Ubuntu 16.04 at https://ubuntu.com/16-04

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

The following packages have been kept back:
  libk5crypto3 libkrb5support0
The following packages will be upgraded:
  libzstd1
1 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
1 esm-infra security update
Need to get 189 kB of archives.
After this operation, 132 kB of additional disk space will be used.
Err:1 https://esm.ubuntu.com/infra/ubuntu bionic-infra-security/main amd64 libzstd1 amd64 1.3.3+dfsg-2ubuntu1+esm1
  401  Unauthorized
E: Failed to fetch https://esm.ubuntu.com/infra/ubuntu/pool/main/libz/libzstd/libzstd1_1.3.3+dfsg-2ubuntu1+esm1_amd64.deb  401  Unauthorized

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

I'm not sure what to do at this point. How can I solve this?

Mark Kirby avatar
cn flag
How long ago did you try? 401 can mean that a repo is down for whatever reason, it seems up now, alternatively you could try just removing `esm.ubuntu.com` form your sources list, since you are upgrading, you don't need extended support.
Marry Jane avatar
de flag
Well I tried a few minutes ago but I have multiple servers and I'm doing them all at the same time and while some of them works fine, some of them gives this error so I don't think the repo is down, I think it's related to my server.
Marry Jane avatar
de flag
@MarkKirby also I don't have `esm.ubuntu.com` in my sources list at all.
user535733 avatar
cn flag
You say you're running 16.04 ESM, but those look like 18.04 ESM sources (they say `bionic` instead of `xenial`). 18.04 does not need to be ESM, since it's still in Community Support. This suggests that somebody already tried a `do-release-upgrade` that failed. Perhaps the upgrader left the altered sources in place (it happens). You can change the sources back to Xenial.
Marry Jane avatar
de flag
@user535733 Yes, I tried that myself but it failed. How can I revert this back?
user535733 avatar
cn flag
The fact that it failed suggests that you might need to be prepared for a backup-and-reinstall. Depends upon the reason (check the logs!) Revert the sources change by simply substituting release names in /etc/apt/sources.list and /etc/apt/sources.list.d/*
Score:3
tn flag

On an 18.04 system (at least), look for a /etc/apt/sources.list.d/ubuntu-esm-infra.list file. That file erroneously lists sources for ESM updates that you would need to pay for, and apt is dutifully trying to download them.

I disabled those sources by putting a # before each of the two deb lines. The file now looks like this:

# Written by ubuntu-advantage-tools
#deb https://esm.ubuntu.com/infra/ubuntu bionic-infra-security main
# deb-src https://esm.ubuntu.com/infra/ubuntu xenial-infra-security main

#deb https://esm.ubuntu.com/infra/ubuntu bionic-infra-updates main
# deb-src https://esm.ubuntu.com/infra/ubuntu xenial-infra-updates main

I think deleting the file would have worked, too.

Now, when I update the system again, using my usual update command:

$ sudo apt update && sudo apt autoremove && sudo apt upgrade

...the apt upgrade portion doesn't try to download those unauthorized sources, and the upgrade is able to finish!

Score:2
zw flag

Two ways exist depending on currently running Ubuntu version.

(a) Ubuntu 16.04 LTS as currently running version

You have to backup your sources.list by

sudo mv /etc/apt/sources.list ~/
sudo mv /etc/apt/sources.list.d/*.list ~/

and then fill main sources.list with correct URLs using one of commands below:

  • plain sources.list replace

    cat <<EOF | sudo tee /etc/apt/sources.list
    deb http://archive.ubuntu.com/ubuntu/ xenial-backports main universe multiverse restricted
    deb http://archive.ubuntu.com/ubuntu/ xenial main universe multiverse restricted
    deb http://archive.ubuntu.com/ubuntu/ xenial-updates main universe multiverse restricted
    deb http://security.ubuntu.com/ubuntu/ xenial-security main universe multiverse restricted
    EOF
    
  • using add-apt-repository

    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ xenial-backports main universe multiverse restricted"
    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ xenial main universe multiverse restricted"
    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ xenial-updates main universe multiverse restricted"
    sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu/ xenial-security main universe multiverse restricted"
    

and then resume upgrade to 18.04 LTS:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get purge ubuntu-advantage-tools --autoremove
sudo rm /etc/apt/sources.list.d/ubuntu-esm-infra.list
sudo do-release-upgrade

Then check ~/*.list files from previous release and partially move the lines to /etc/apt/sources.list.

(b) Ubuntu 18.04 LTS as currently running version

You have to backup your sources.list by

sudo mv /etc/apt/sources.list ~/
sudo mv /etc/apt/sources.list.d/*.list ~/

and then fill main sources.list with correct URLs using one of commands below:

  • plain sources.list replace

    cat <<EOF | sudo tee /etc/apt/sources.list
    deb http://archive.ubuntu.com/ubuntu/ bionic-backports main universe multiverse restricted
    deb http://archive.ubuntu.com/ubuntu/ bionic main universe multiverse restricted
    deb http://archive.ubuntu.com/ubuntu/ bionic-updates main universe multiverse restricted
    deb http://security.ubuntu.com/ubuntu/ bionic-security main universe multiverse restricted
    EOF
    
  • using add-apt-repository

    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ bionic-backports main universe multiverse restricted"
    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ bionic main universe multiverse restricted"
    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ bionic-updates main universe multiverse restricted"
    sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu/ bionic-security main universe multiverse restricted"
    

and then install all necessary upgrades by:

sudo apt-get purge ubuntu-advantage-tools --autoremove
sudo rm /etc/apt/sources.list.d/ubuntu-esm-infra.list

sudo apt-get update
sudo apt-get upgrade

Then check ~/*.list files from previous release and partially move the lines to /etc/apt/sources.list while replacing xenial with bionic.

Marry Jane avatar
de flag
Thank you for your answer. It is very detailed. For my case, only running the command `sudo apt-get purge ubuntu-advantage-tools --autoremove` alone solved my issue. I didn't have to do the other changes.
Lambart avatar
tn flag
On an 18.04 system, the source of the problem was the `/etc/apt/sources.list.d/ubuntu-esm-infra.list` file. Commenting out the sources in there did the trick. I'm sure deleting it would have worked, too.
Score:1
cn flag

When you register a system with Ubuntu Pro a /etc/apt/auth.conf.d/90ubuntu-advantage file is created. This file contains credentials tied to your Ubuntu account which gives that system permission to retrieve ESM updates.

I registered a system and initially it had no issues, but after some weeks attempts to retrieve packages from https://esm.ubuntu.com/ gave a HTTP 401 "Unauthorized" error.

I resolved the issue by detaching and reattaching the machine using the current Token value from https://ubuntu.com/pro/dashboard.

This regenerated the /etc/apt/auth.conf.d/90ubuntu-advantage file with current credentials.

  1. sudo pro detach
  2. sudo pro attach TOKEN_VALUE_HERE
  3. sudo apt-get update

At this point I could apply updates (e.g., via apt, apt-get, aptitude) as before.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.