My stand-alone Mattermost on Ubuntu server stops and gets frozen suddenly; no ping, no SSH, no service. It is installed on vmware with 8G RAM and 60G storage. In vmware host I see the RAM usage is less than 4G. The storage is less than 20G used.
During the fault, even vmware browser console of the vm doesn't work, so there isn't any connection to server.
How can I find the root cause?
Mattermost version is Enterprise 7.8.1
I just could power-off and then power-on to recover the server.
It occurs mostly on Saturdays!
Today I faced it again! Imagine 4 happening between 11:30 till 17:00 (local time), and now it’s more than 2 days without any problem. Last week on Saturday between 14:00 till 18:00 I had it, then it worked for 7 days none-stopped.
The latest logs in journalctl results are like these:
Apr 15 13:45:01 mattermost CRON[1740]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 15 13:45:01 mattermost CRON[1741]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Apr 15 13:45:01 mattermost CRON[1740]: pam_unix(cron:session): session closed for user root
Apr 15 13:55:01 mattermost CRON[1766]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 15 13:55:01 mattermost CRON[1767]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Apr 15 13:55:01 mattermost CRON[1766]: pam_unix(cron:session): session closed for user root
Apr 15 13:59:50 mattermost mattermost[1075]: {“timestamp”:“2023-04-15 13:59:50.961 +03:30”,“level”:…
Apr 15 13:59:51 mattermost mattermost[1075]: {“timestamp”:“2023-04-15 13:59:51.503 +03:30”,“level”:…
– Reboot –
Apr 15 16:54:47 mattermost mattermost[1089]: {“timestamp”:“2023-04-15 16:54:47.719 +03:30”,“level”…
Apr 15 16:54:48 mattermost mattermost[1089]: {“timestamp”:“2023-04-15 16:54:48.242 +03:30”,“level”…
Apr 15 16:55:01 mattermost CRON[2134]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 15 16:55:01 mattermost CRON[2135]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Apr 15 16:55:01 mattermost CRON[2134]: pam_unix(cron:session): session closed for user root
– Reboot –
And Syslog resutls:
Apr 15 13:59:50 mattermost mattermost[1075]: {“timestamp”:“2023-04-15 13:59:50.627 +03:30”,“level”:…
Apr 15 13:59:51 mattermost mattermost[1075]: {“timestamp”:“2023-04-15 13:59:51.503 +03:30”,“level”:…
Apr 15 16:54:48 mattermost mattermost[1089]: {“timestamp”:“2023-04-15 16:54:48.242 +03:30”,“level”:“info”,…
Apr 15 16:55:01 mattermost CRON[2135]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Server version are as bellow:
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
$ snap list
Name Version Rev Tracking Publisher Notes
core18 20210507 2066 latest/stable canonical✓ base
gnome-3-34-1804 0+git.3556cb3 66 latest/stable/… canonical✓ -
gtk-common-themes 0.1-52-gb92ac40 1515 latest/stable/… canonical✓ -
snap-store 3.38.0-59-g494f078 518 latest/stable/… canonical✓ -
snapd 2.50.1 12057 latest/stable canonical✓ snapd
