I seem to be unable to ssh or su as a local user on my CentOS 7 machine as of today.
$ adduser ndejay
$ passwd ndejay
Changing password for user ndejay.
New password:
Retype new password:
$ su ndejay
su: failed to execute /bin/bash: Permission denied
$ ssh ndejay@localhost
ndejay@localhost's password:
Could not chdir to home directory /home/ndejay: Permission denied
/bin/bash: Permission denied
Connection to localhost closed.
I confirmed that permissions are set properly and that SElinux is disabled.
$ ls -ld /home /home/ndejay
drwxr-xr-t. 10 root root 4096 Feb 27 18:13 /home
drwx------ 2 ndejay ndejay 85 Feb 27 18:13 /home/ndejay
$ ls -ld /bin /usr/bin /usr/bin/bash
lrwxrwxrwx. 1 root root 7 Oct 1 2015 /bin -> usr/bin
dr-xr-xr-x. 2 root root 36864 Jan 9 17:51 /usr/bin
-rwxr-xr-x. 1 root root 960384 Mar 5 2015 /usr/bin/bash
$ getenforce
Disabled
$ tail /var/log/secure
Feb 27 18:13:28 fs01 su: pam_unix(su:session): session opened for user ndejay by (uid=0)
Feb 27 18:13:28 fs01 su: pam_sss(su:session): Request to sssd failed. Connection refused
I tried to disable sssd, but it doesn't help:
$ authconfig --savebackup /root/230227-authconfig
$ authconfig --disablesssd --disablesssdauth --update
$ tail /var/log/secure
Feb 27 18:35:24 fs01 su: pam_unix(su:session): session opened for user ndejay by (uid=0)
Feb 27 18:35:24 fs01 su: pam_unix(su:session): session closed for user ndejay
Feb 27 18:40:25 fs01 sssd[14272]: Accepted password for ndejay from 172.21.13.11 port 53132 ssh2
Feb 27 18:40:25 fs01 sshd[14272]: pam_unix(sshd:session): session opened for user ndejay by (uid=0)
Feb 27 18:40:25 fs01 sshd[14344]: Received disconnect from 172.21.13.11: 11: disconnected by user
Feb 27 18:40:25 fs01 sshd[14272]: pam_unix(sshd:session): session closed for user ndejay
Any help would be greatly appreciated. Thanks!
Nic
EDIT 1 after @Andrew Henle suggestion to look into disk usage
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 222G 100G 123G 45% /
devtmpfs 7.8G 0 7.8G 0% /dev
tmpfs 7.8G 0 7.8G 0% /dev/shm
tmpfs 7.8G 41M 7.7G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/sdb1 4.7G 174M 4.5G 4% /boot
/dev/mapper/centos-home 47G 122M 47G 1% /home
tmpfs 1.6G 0 1.6G 0% /run/user/0
tmpfs 1.6G 0 1.6G 0% /run/user/12002
$ df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/centos-root 232411136 2806233 229604903 2% /
devtmpfs 2026378 497 2025881 1% /dev
tmpfs 2028949 1 2028948 1% /dev/shm
tmpfs 2028949 711 2028238 1% /run
tmpfs 2028949 13 2028936 1% /sys/fs/cgroup
/dev/sdb1 4882432 337 4882095 1% /boot
/dev/mapper/centos-home 48828416 889 48827527 1% /home
tmpfs 2028949 1 2028948 1% /run/user/0
tmpfs 2028949 1 2028948 1% /run/user/12002
EDIT 2 after @Andrew Henle suggestion to look into verbose sshd logs
On the server:
$ /usr/sbin/sshd -ddd -p 222
...
Server listening on :: port 222.
On the client:
$ ssh -p 222 ndejay@fs01
Last login: Mon Feb 27 18:40:25 2023 from d1p-hydratm01.ldi.lan
debug3: mm_request_send entering: type 124
debug3: mm_request_receive_expect entering: type 125
debug3: mm_request_receive entering
debug3: Copy environment: XDG_SESSION_ID=2198
Environment:
LANG=en_US.UTF-8
USER=ndejay
LOGNAME=ndejay
HOME=/home/ndejay
PATH=/usr/local/bin:/usr/bin
MAIL=/var/mail/ndejay
SHELL=/bin/bash
...
SSH_TTY=/dev/pts/1
TERM=xterm-256color
XDG_SESSION_ID=2198
Could not chdir to home directory /home/ndejay: Permission denied
/bin/bash: Permission denied
Connection to fs01 closed.
On the server end (full log on pastebin):
Connection from 172.21.13.11 port 50449 on 172.21.13.10 port 222
...
Could not stat AuthorizedKeysCommand "/usr/bin/sss_ssh_authorizedkeys": Permission denied
...
debug1: Could not open authorized keys '/home/ndejay/.ssh/authorized_keys': Permission denied
...
Accepted password for ndejay from 172.21.13.11 port 51316 ssh2
...
/dev/pts/1: Permission denied
open /dev/tty failed - could not set controlling tty: Permission denied
...
Received disconnect from 172.21.13.11: 11: disconnected by user
