Which CUDA version should I install? Do they have back compatibility?

This is the first GPU I am using and unfortunately, I am using Ubuntu 20.04 not easy easy Windows. I want to use my GPU (Nvidia Quadro 2000 1GB GDDR5) for very basic machine learning models. I've got a supercomputer from my university to train bigger models.

When I type nvidia-smi in my terminal I can see the following.

+---------------------------------------------+
| NVIDIA-SMI 390.144                Driver Version: 390.144                   |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro 2000         Off  | 00000000:01:00.0  On |                  N/A |
| 34%   62C    P0    N/A /  N/A |    383MiB /   963MiB |     20%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+---------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0       955      G   /usr/lib/xorg/Xorg                            93MiB |
|    0      1261      G   /usr/bin/gnome-shell                         143MiB |
|    0      3398      G   ...AAgAAAAAAAAACAAAAAAAAAA= --shared-files   142MiB |
+---------------------------------------------+

In other answers for example in this one Nvidia-smi shows CUDA version, but CUDA is not installed there is CUDA version next to the Driver version.

I want to download Pytorch but I am not sure which CUDA version should I download. Or should I download CUDA separately in case I wish to run some Tensorflow code. BTW I use Anaconda with VScode.

I found an old article which says my GPU supports CUDA 2.1. Are the newer versions back-compatible?

As per Nmath's suggestion, I went on to install CUDA from the Ubuntu repository as follows.

$ sudo apt install nvidia-cuda-toolkit

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been                Recommends: nvidia-visual-profiler (= 10.1.243-3) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.oing to be installed
                       Recommends: nvidia-visual-profiler (= 10.1.243-3) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

Here, I do understand that it need some dependencies. How do I fix it?

Don't overthink this. Just use the version of CUDA that is in repos for your version of Ubuntu. Unless you have a very specific technical reason for doing so, you shouldn't need to install a specific version, especially not one that isn't in Ubuntu's repositories. This is as true with CUDA as it is any other software. Think about this: why would developers release new versions of software that stop supporting features and hardware that most people are still using?

In fact, if you try to explicitly install software versions (especially older ones) that are different than what's in Ubuntu's repositories, this is often a recipe for disaster as lots of software in Ubuntu/Linux rely on dependencies and expect the version that is in official repositories.