Score:0

Running time-critical executable (CPU+GPU) I receive huge time divergence among execution runtimes

ph flag

Under Ubuntu 20.04.3 LTS I am running a multithreading application that is using both CPUs and GPU. My question is that among sequencing identical runs, I get huge time divergence among execution runtimes (benchmarks). I checked the processes running in the background which seems very normal and the CPU idle time is almost 99%.

For example

1st Run: 573 secs 2nd Run: 750 secs 3rd Run: 986 secs

I tried to run the application with lower priority using nice -n15 but I didn't see any differences.

So, 1) how can I explain this phenomenon? and 2)how to set the application to Real-time priority?

Regards,

Maverick avatar
ph flag
@guiverc yes Ubuntu 20.04.3 LTS
TD15 avatar
cn flag
This is probably related to your CPU/GPU temps. A possible explanation is: in the first run your CPU/GPU started near ambient and had the ability to boost to max turbo for a decent time then trottled due to heat. In the second run maybe the CPU/GPU were already hot from the first run hence it didn't turbo. The third one maybe had even worse throttling. It is actually hard to determine what is the cause without more information. What is your app doing? Only CPU/GPU bound or it accesses the disk frequently? Is it processing the same data?
Maverick avatar
ph flag
yes they processing the same data and also using disk frequently (video encoding). GPU load is about 60%
Maverick avatar
ph flag
I will re-run the tests with `chrt -f 99` command and check again.
TD15 avatar
cn flag
Assuming you have an SSD, SSDs (Non SLC ones) use SLC cache to cover for the slower write speed of TLC and QLC cells. Maybe you are exhasting your SLC write cache and writing to the slower TLC cells. Run your app while monitoring CPU/GPU load, frequency, temps and disk bandwidth.
Maverick avatar
ph flag
thanx, I will be monitoring the statistics. SSD is the primary disk used only for boot. The application read/write IOs are on a secondary mechanical HD.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.