Score:1

Computer cluster admin: how to limit users running program but permit file transferring

ua flag
wdg

I am managing a small computer cluster with slurm on CentOS 7. I want to discourage users to run programs on login node. This can be achieved by adding user hard cpu 1 to file /etc/security/limits.conf. However, I do not want file transferring from another cluster through sshd to be disrupted. Is there a way to reconcile these seemingly conflicting goals?

Score:1
nc flag

CPU limits seem like the obvious answer, but an effective answer is to use memory limits instead. Tools like ssh tend to have a constant memory profile, so a suitable ulimit for memory can be found that will not affect them, where most real computing applications have large memory requirements that will quickly go over the limit.

Also, the operating system tends to be pretty good at balancing cpu use between processes under heavy cpu use. A per user total processes ulimit may also be necessary.

If you are dead set on limiting cpu use, instead of a hard cpu limit, it is possible to renice heavy cpu processes, or use cgroups to throttle them.

If you want to be draconian, use a script in cron that uses ps -o and filter by pcpu, total cpu use, and whitelist some commands like ssh, screen, tmux, and system users, and kill remaining suspect processes automatically. (I recommend printing candidates and manually reviewing them for a few months before flipping the kill switch.)

However, the key is to educate users to use slurm. Things like the above may be a form of hard persuasion to accomplish that.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.