How can I reserve my GPU for only executing Python scripts?

dextervex

3/8/24, 2:33 PM

I'm on Ubuntu 22.04 trying to run a PyTorch script and get

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 86.00 MiB (GPU 0; 3.81 GiB total capacity; 3.18 GiB already allocated; 80.81 MiB free; 3.18 GiB reserved in total by PyTorch)

prime-select query yields the following:

on-demand

lspci | egrep 'VGA|3D' yields the following:

00:02.0 VGA compatible controller: Intel Corporation Alder Lake-P Integrated Graphics Controller (rev 0c)
01:00.0 3D controller: NVIDIA Corporation GA107M [GeForce RTX 3050 Ti Mobile] (rev a1)

nvidia-smi yields the following:

+---------------------------------------------+
| NVIDIA-SMI 525.85.05    Driver Version: 525.85.05    CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   37C    P0    N/A /  35W |      7MiB /  4096MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+---------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1464      G   /usr/lib/xorg/Xorg                  4MiB |
|    0   N/A  N/A      1743      G   ...ome-remote-desktop-daemon        1MiB |
+---------------------------------------------+

In my Ubuntu About settings, "Graphichs" is shown as:

NVIDIA Corporation GA107M [GeForce RTX 3050 Ti Mobile] / Mesa Intel® Graphics (ADL GT2)

How can i stop my gpu being used for Xorg and for the gnome-remote-desktop-daemon? I only want to use my gpu with PyTorch. Using the torch.cuda.memory_allocated and torch.cuda.memory_reserved functions shows that I'm using 0.0GB of my gpu despite recognizing the Nvidia gpu, so I'm confused haha.

If anyone needs more info please let me know (:

Thanks!

167

0 + 4

python

nvidia

cuda

drivers

Terrance

3/8/24, 3:00 PM

You might want to see https://stackoverflow.com/questions/15197286/how-can-i-flush-gpu-memory-using-cuda-physical-reset-is-unavailable but you might not have enough memory on your device to run that script properly depending on the script you are running. However, killing Xorg from being used by the GPU may render your GUI not working and you may not be able to see what you are doing. That being said, you might be able to get it to fully work without the GUI or maybe use a different desktop version, i.e. Xubuntu, Lubuntu, etc that doesn't use as much resources.

dextervex

3/8/24, 3:17 PM

@Terrance thanks! Do you know if there's a way to force Xorg to use the cpu instead? I tried the `nvidia-smi --gpu-reset` command from the link but the gpu is in use (by Xorg/desktop daemon I assume). You're right about killing Xorg, definitely messes up the gui haha

Terrance

3/8/24, 3:23 PM

I wish I had integrated graphics on my system to actually help on things like this, but unfortunately, I don't. There are answers out there were you can use the Intel for the display then use the NVIDIA for CUDA. One to see is https://askubuntu.com/questions/869496/force-xorg-to-use-cpu-not-gpu and I am certain that there are others out there. Just to let you know, even if an answer was written for 14.04 there is still a good chance that it will work with newer versions like 22.04.

dextervex

3/8/24, 3:25 PM

Okay that's super helpful thank you so much, I'll play around with it and hopefully not blow everything up (:

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: How can I reserve my GPU for only executing Python scripts?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.