Score:2

Problems updating CUDA on 18.04 LTS (driver package conflict?)

kz flag

I am trying to upgrade my CUDA install on Ubuntu 18.04 LTS - more specifically, I am following the instructions provided by the TensorFlow documentation here (I followed these instructions for the original install, CUDA 10). First I did (following the instructions here)

sudo rm /etc/apt/sources.list.d/cuda*
sudo apt remove --autoremove nvidia-cuda-toolkit
sudo apt remove --autoremove nvidia-*

sudo apt-get purge nvidia*
sudo apt-get autoremove
sudo apt-get autoclean

sudo rm -rf /usr/local/cuda*

Then (from the TensorFlow instructions)

sudo apt install ./nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb

But after executing that I get the following error:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies.
 libnvidia-extra-450 : Depends: libnvidia-extra-460 but it is not installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

Then trying sudo apt --fix-broken install gives me:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  g++-6 libaccinj64-9.1 libcublas9.1 libcudart9.1 libcufft9.1 libcufftw9.1 libcuinj64-9.1 libcurand9.1 libcusolver9.1
  libcusparse9.1 libnppc9.1 libnppial9.1 libnppicc9.1 libnppicom9.1 libnppidei9.1 libnppif9.1 libnppig9.1 libnppim9.1
  libnppist9.1 libnppisu9.1 libnppitc9.1 libnpps9.1 libnvblas9.1 libnvgraph9.1 libnvidia-common-440 libnvidia-extra-440
  libnvrtc9.1 libnvtoolsext1 libnvvm3 libstdc++-6-dev libthrust-dev libvdpau-dev nvidia-cuda-doc nvidia-cuda-gdb
  nvidia-opencl-dev nvidia-profiler nvidia-visual-profiler ocl-icd-opencl-dev opencl-c-headers
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libnvidia-extra-460
The following NEW packages will be installed
  libnvidia-extra-460
0 to upgrade, 1 to newly install, 0 to remove and 273 not to upgrade.
2 not fully installed or removed.
Need to get 0 B/40.9 kB of archives.
After this operation, 120 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
(Reading database ... 196344 files and directories currently installed.)
Preparing to unpack .../libnvidia-extra-460_460.106.00-0ubuntu1_amd64.deb ...
Unpacking libnvidia-extra-460:amd64 (460.106.00-0ubuntu1) ...
dpkg: error processing archive /var/cache/apt/archives/libnvidia-extra-460_460.106.00-0ubuntu1_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/x86_64-linux-gnu/libnvidia-allocator.so', which is also in package libnvidia-compute-450:amd64 450.36.06-0ubuntu1
Errors were encountered while processing:
 /var/cache/apt/archives/libnvidia-extra-460_460.106.00-0ubuntu1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

I guess this is just a driver conflict. Following the suggested fix from this issue I am minded to run

sudo dpkg -P nvidia-driver-450

Or force the install with

sudo dpkg -i --force-overwrite /var/cache/apt/archives/libnvidia-compute-450:amd64 450.36.06-0ubuntu1
sudo apt -f install

The output of dpkg -l | grep -i nvidia is:

ii  cuda-nsight-compute-10-1                   10.1.243-1                                       amd64        NVIDIA Nsight Compute
ii  cuda-nsight-systems-10-1                   10.1.243-1                                       amd64        NVIDIA Nsight Systems
ii  cuda-nvtx-10-1                             10.1.243-1                                       amd64        NVIDIA Tools Extension
ii  libaccinj64-9.1:amd64                      9.1.85-3ubuntu1                                  amd64        NVIDIA ACCINJ Library (64-bit)
ii  libcublas9.1:amd64                         9.1.85-3ubuntu1                                  amd64        NVIDIA cuBLAS Library
ii  libcudart9.1:amd64                         9.1.85-3ubuntu1                                  amd64        NVIDIA CUDA Runtime Library
ii  libcufft9.1:amd64                          9.1.85-3ubuntu1                                  amd64        NVIDIA cuFFT Library
ii  libcufftw9.1:amd64                         9.1.85-3ubuntu1                                  amd64        NVIDIA cuFFTW Library
ii  libcuinj64-9.1:amd64                       9.1.85-3ubuntu1                                  amd64        NVIDIA CUINJ Library (64-bit)
ii  libcurand9.1:amd64                         9.1.85-3ubuntu1                                  amd64        NVIDIA cuRAND Library
ii  libcusolver9.1:amd64                       9.1.85-3ubuntu1                                  amd64        NVIDIA cuSOLVER Library
ii  libcusparse9.1:amd64                       9.1.85-3ubuntu1                                  amd64        NVIDIA cuSPARSE Library
ii  libnppc9.1:amd64                           9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives core runtime library
ii  libnppial9.1:amd64                         9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Arithmetic and Logic
ii  libnppicc9.1:amd64                         9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Color Conversion
ii  libnppicom9.1:amd64                        9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Compression
ii  libnppidei9.1:amd64                        9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Data Exchange and Initialization
ii  libnppif9.1:amd64                          9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Filters
ii  libnppig9.1:amd64                          9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Geometry transforms
ii  libnppim9.1:amd64                          9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Morphological operations
ii  libnppist9.1:amd64                         9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Statistics
ii  libnppisu9.1:amd64                         9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Support
ii  libnppitc9.1:amd64                         9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives lib for Image Threshold and Compare
ii  libnpps9.1:amd64                           9.1.85-3ubuntu1                                  amd64        NVIDIA Performance Primitives for signal processing runtime library
ii  libnvgraph9.1:amd64                        9.1.85-3ubuntu1                                  amd64        NVIDIA Graph Analytics library (nvGRAPH)
ii  libnvidia-cfg1-450:amd64                   450.36.06-0ubuntu1                               amd64        NVIDIA binary OpenGL/GLX configuration library
ii  libnvidia-common-440                       450.119.03-0ubuntu0.18.04.1                      all          Transitional package for libnvidia-common-450
ii  libnvidia-common-450                       460.91.03-0ubuntu0.18.04.1                       all          Transitional package for libnvidia-common-460
ii  libnvidia-common-460                       460.91.03-0ubuntu0.18.04.1                       all          Shared files used by the NVIDIA libraries
rc  libnvidia-compute-440:amd64                440.82-0ubuntu0~0.18.04.2                        amd64        NVIDIA libcompute package
ii  libnvidia-compute-450:amd64                450.36.06-0ubuntu1                               amd64        NVIDIA libcompute package
ii  libnvidia-decode-450:amd64                 450.36.06-0ubuntu1                               amd64        NVIDIA Video Decoding runtime libraries
ii  libnvidia-encode-450:amd64                 450.36.06-0ubuntu1                               amd64        NVENC Video Encoding runtime library
iU  libnvidia-extra-440:amd64                  450.119.03-0ubuntu0.18.04.1                      amd64        Transitional package for libnvidia-extra-450
iU  libnvidia-extra-450:amd64                  460.91.03-0ubuntu0.18.04.1                       amd64        Transitional package for libnvidia-extra-460
ii  libnvidia-fbc1-450:amd64                   450.36.06-0ubuntu1                               amd64        NVIDIA OpenGL-based Framebuffer Capture runtime library
ii  libnvidia-gl-450:amd64                     450.36.06-0ubuntu1                               amd64        NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
ii  libnvidia-ifr1-450:amd64                   450.36.06-0ubuntu1                               amd64        NVIDIA OpenGL-based Inband Frame Readback runtime library
ii  libnvrtc9.1:amd64                          9.1.85-3ubuntu1                                  amd64        CUDA Runtime Compilation (NVIDIA NVRTC Library)
ii  libnvtoolsext1:amd64                       9.1.85-3ubuntu1                                  amd64        NVIDIA Tools Extension Library
ii  libnvvm3:amd64                             9.1.85-3ubuntu1                                  amd64        NVIDIA NVVM Library
ii  nsight-compute-2020.1.0                    2020.1.0.33-1                                    amd64        NVIDIA Nsight Compute
rc  nvidia-compute-utils-440                   440.82-0ubuntu0~0.18.04.2                        amd64        NVIDIA compute utilities
ii  nvidia-compute-utils-450                   450.36.06-0ubuntu1                               amd64        NVIDIA compute utilities
ii  nvidia-cuda-doc                            9.1.85-3ubuntu1                                  all          NVIDIA CUDA and OpenCL documentation
ii  nvidia-cuda-gdb                            9.1.85-3ubuntu1                                  amd64        NVIDIA CUDA Debugger (GDB)
rc  nvidia-dkms-440                            440.82-0ubuntu0~0.18.04.2                        amd64        NVIDIA DKMS package
ii  nvidia-dkms-450                            450.36.06-0ubuntu1                               amd64        NVIDIA DKMS package
ii  nvidia-driver-450                          450.36.06-0ubuntu1                               amd64        NVIDIA driver metapackage
rc  nvidia-kernel-common-440                   440.82-0ubuntu0~0.18.04.2                        amd64        Shared files used with the kernel module
ii  nvidia-kernel-common-450:amd64             460.91.03-0ubuntu0.18.04.1                       amd64        Transitional package for nvidia-kernel-common-460
ii  nvidia-kernel-common-460                   460.91.03-0ubuntu0.18.04.1                       amd64        Shared files used with the kernel module
ii  nvidia-kernel-source-450                   450.36.06-0ubuntu1                               amd64        NVIDIA kernel source package
ii  nvidia-machine-learning-repo-ubuntu1804    1.0.0-1                                          amd64        nvidia-machine-learning repository configuration files
ii  nvidia-modprobe                            450.36.06-0ubuntu1                               amd64        Load the NVIDIA kernel driver and create device files
ii  nvidia-opencl-dev:amd64                    9.1.85-3ubuntu1                                  amd64        NVIDIA OpenCL development files
ii  nvidia-profiler                            9.1.85-3ubuntu1                                  amd64        NVIDIA Profiler for CUDA and OpenCL
ii  nvidia-settings                            470.57.01-0ubuntu0.18.04.1                       amd64        Tool for configuring the NVIDIA graphics driver
ii  nvidia-utils-450                           450.36.06-0ubuntu1                               amd64        NVIDIA driver support binaries
ii  nvidia-visual-profiler                     9.1.85-3ubuntu1                                  amd64        NVIDIA Visual Profiler for CUDA and OpenCL
ii  xserver-xorg-video-nvidia-450              450.36.06-0ubuntu1                               amd64        NVIDIA binary Xorg driver

I tried explicitly removing and purging the libnvidia-compute-450 package, but that did nothing. Not sure what to do, tempted to just remove that driver but having read of people being stung by similar problems I'm wary, don't want to have to completely reinstall Ubuntu!

Score:0
kz flag

OK in the end I managed to fix this by forcing overwrite of the offending package (purging it didn’t work). I ran

sudo dpkg -i --force-overwrite /var/cache/apt/archives/libnvidia-extra-460_460.106.00-0ubuntu1_amd64.deb

Then I was able to successfully run the rest of the TensorFlow CUDA install instructions. Annoyingly, these instructions actually install CUDA 11.0 for TF 2.4 (why?), which are not the current versions of either - it’s easy to miss this!

A very frustrating experience.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.