Need advises on VPS security - maybe some automated tools

I create application that will be served on Ubuntu 20.04 VPS server. Nginx, Python, Postgresql, nodejs, nothing special.

Unfortunately, all my knowledge about server security ends after ufw enable and fail2ban. (Because most tutorials ends after it too.)

Can you please recommend me some good modern manual on securing server in Internet? If there is some automated security-test service or anything that can help audit/monitor server - please tell me, paid is ok.

The principles of keeping your servers secure are quite simple.

In no order of priority:

• Keep your software up-to-date. (Related: run only supported/maintained versions.)
• Run only the services you need.
• Install only the software you need.
• Configure your software correctly.
• Grant access based on the principle of least privilege.
• Add monitoring.
  Establish a base line and alert on deviations.

The specifics depend quite a lot on the actual software, the services that you need to run and your requirements.

For external validation of your configuration: There are numerous vulnerability scanners and/or penetration testing toolkits as you can see on lists such as this one: https://owasp.org/www-community/Vulnerability_Scanning_Tools

Be aware that often such scans rely on determining the version numbers of your installed software and they don't test if known vulnerabilities can successfully be exploited. That can result in many false positives on Linux distributions that do security back porting, as for example explained in this Q&A PCI Compliance: install Apache 2.4.17 on Ubuntu 14.04.3?

Authenticated scans can help against that by checking the version of the package, rather than the version string an application reports.

Another approach is more from the system management perspective with centralised server management including release and patch management.
For example Ubuntu's Landscape , Red Hat Satellite and Microsoft SCCM

Tips for securing your VPS and some automated tools that you can us:

Keep your software up-to-date: Always ensure that your operating system and all software installed on your VPS are up-to-date with the latest security patches.

Use strong passwords: Use strong and unique passwords for all user accounts on your VPS, including the root account. Avoid using easily guessable passwords.

Configure a firewall: Only allow access to the necessary ports and services.

Use SSH keys: Instead of passwords, use SSH keys for authentication. This will make it harder for attackers to gain access to your VPS.

Enable 2FA: Enable two-factor authentication for all user accounts on your VPS.

Use IDS: Consider using an intrusion detection system such as OSSEC or Snort or Suricata to monitor your VPS for unusual activity.

Backup regularly: Prevent data loss in case of a security breach.

Free/Open-source Automated tools:

Fail2ban: Scan log files for failed login attempts and blocks the IP address of the attacker.

ClamAV: AV software that can scan your VPS for malware and viruses.

Lynis: Automated security auditing tool that can scan your VPS for security vulnerabilities and provide recommendations for remediation.

OpenVAS: Vulnerability scanner that can scan your VPS for known security vulnerabilities.