I am currently unable to connect to my remote workstation (I unfortunately do not have admin privileges) via SSH, instead receiving the error message shell request failed on channel 0 when trying to open an SSH connection normally and exec request failed on channel 0 when trying to remotely execute commands.
However, I am still able to access the remote system via RDP using Remmina (the server has xrdp installed). Additionally, as far as I know, this problem only affects me - other users are able to access the workstation both via RDP and SSH.
Both the local and remote systems use Ubuntu (20.04.3 and 18.04.3 respectively). The problem first started appearing when the remote system had to be restarted due to another user freezing the system entirely (trying to run a series of expensive calculations simultaneously), but the problem is still persisting after multiple restarts and logout/logins now that the workstation's resources have been freed up again.
As I can still connect via RDP, I tried journalctl and found the following entries just after each SSH connection attempt (the stack trace seems unrelated to me at first glance, but after trying to connect a few times, it almost always seems to appear after the sshd error):
Nov 28 10:59:18 <hostname> sshd[80953]: error: do_exec_pty: fork: Resource temporarily unavailable
Nov 28 10:59:18 <hostname> gnome-shell[79418]: Object St.Bin (0x55a0c655e260), has been already finalized. Impossible to set any property to it.
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: == Stack trace for context 0x55a0c3a70340 ==
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #0 0x55a0c3e92910 i resource:///org/gnome/shell/ui/userWidget.js:59 (0x7fdda0655cd0 @ 212)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #1 0x7ffc5a863090 b resource:///org/gnome/gjs/modules/_legacy.js:82 (0x7fdde40b5de0 @ 71)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #2 0x55a0c3e92890 i resource:///org/gnome/shell/ui/components/polkitAgent.js:342 (0x7fdd90068ab0 @ 59)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #3 0x7ffc5a8643f0 b resource:///org/gnome/gjs/modules/_legacy.js:82 (0x7fdde40b5de0 @ 71)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #4 0x55a0c3e927d0 i self-hosted:916 (0x7fdde40f12b8 @ 367)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: == Stack trace for context 0x55a0c3a70340 ==
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #0 0x55a0c3e92910 i resource:///org/gnome/shell/ui/userWidget.js:60 (0x7fdda0655cd0 @ 274)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #1 0x7ffc5a863090 b resource:///org/gnome/gjs/modules/_legacy.js:82 (0x7fdde40b5de0 @ 71)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #2 0x55a0c3e92890 i resource:///org/gnome/shell/ui/components/polkitAgent.js:342 (0x7fdd90068ab0 @ 59)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #3 0x7ffc5a8643f0 b resource:///org/gnome/gjs/modules/_legacy.js:82 (0x7fdde40b5de0 @ 71)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #4 0x55a0c3e927d0 i self-hosted:916 (0x7fdde40f12b8 @ 367)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: == Stack trace for context 0x55a0c3a70340 ==
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #0 0x55a0c3e92910 i resource:///org/gnome/shell/ui/userWidget.js:65 (0x7fdda0655cd0 @ 365)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #1 0x7ffc5a863090 b resource:///org/gnome/gjs/modules/_legacy.js:82 (0x7fdde40b5de0 @ 71)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #2 0x55a0c3e92890 i resource:///org/gnome/shell/ui/components/polkitAgent.js:342 (0x7fdd90068ab0 @ 59)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #3 0x7ffc5a8643f0 b resource:///org/gnome/gjs/modules/_legacy.js:82 (0x7fdde40b5de0 @ 71)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #4 0x55a0c3e927d0 i self-hosted:916 (0x7fdde40f12b8 @ 367)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: == Stack trace for context 0x55a0c3a70340 ==
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #0 0x55a0c3e92890 i resource:///org/gnome/shell/ui/components/polkitAgent.js:343 (0x7fdd90068ab0 @ 84)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #1 0x7ffc5a8643f0 b resource:///org/gnome/gjs/modules/_legacy.js:82 (0x7fdde40b5de0 @ 71)
Nov 28 10:59:18 <hostname> org.gnome.Shell.desktop[79418]: #2 0x55a0c3e927d0 i self-hosted:916 (0x7fdde40f12b8 @ 367)
Nov 28 10:59:18 <hostname> gnome-shell[79418]: Object St.Bin (0x55a0c655e260), has been already finalized. Impossible to set any property to it.
Nov 28 10:59:18 <hostname> gnome-shell[79418]: Object St.Bin (0x55a0c655e260), has been already deallocated - impossible to access it. This might be caused by the object having been destroyed from C code using something such as destroy(), dispose(), or remove() vfuncs
Nov 28 10:59:18 <hostname> gnome-shell[79418]: clutter_actor_set_size: assertion 'CLUTTER_IS_ACTOR (self)' failed
Nov 28 10:59:18 <hostname> gnome-shell[79418]: Object St.Bin (0x55a0c655e260), has been already deallocated - impossible to access it. This might be caused by the object having been destroyed from C code using something such as destroy(), dispose(), or remove() vfuncs
Nov 28 10:59:18 <hostname> gnome-shell[79418]: clutter_actor_show: assertion 'CLUTTER_IS_ACTOR (self)' failed
The fork: Resource temporarily unavailable seemed to suggest that I was running low on resources, so I also tried checking the system limits with ulimit -a and less /etc/security/limits.conf. The results seemed slightly contradictory, especially nproc vs. ulimit -u:
ulimit -a
-----
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 514665
max locked memory (kbytes, -l) 65536
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 514665
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
/etc/security/limits.conf
-----
(entries for other users...)
<username> hard nproc 12
<username> hard as 67000000
<username> hard memlock 67000000
<username> hard rss 67000000
Running ps --no-headers auxwwwm | awk '$2 == "-" { print $1 }' | sort | uniq -c | sort -nr tells me that I have 731 running processes attributed to my user account and running lsof -u <username> 2> /dev/null | sed -E "s/^(..........).+$/\1/" | sort | uniq -c | sort -r | head gives me the following:
3740 evolution
801 gnome-she
636 goa-daemo
489 deja-dup-
338 baloo_fil
336 dbus-daem
275 gnome-sof
236 gvfs-udis
231 x-termina
216 gvfs-afc-
So it appears that I may indeed be above the limit for either open files or processes, but this seems to be due to programs that I have always had running without any problem.
I'm further confused by the fact that I can still connect via RDP and use the workstation as usual. Does anybody have any idea why this could be working while SSH isn't, and how I could get that working again?
