Changing hostname for the purpose of email delivery

I have email delivery issues, and MXtoolbox says "Reverse DNS is not a Valid Hostname". I have A and AAAA records for domain.com and mail.domain.com. The data center has set up reverse DNS to domain.com. I have found this and this thread where users reported similar issues and it appears the issues were resolved by 'changing the hostname' from 'domain.com' to 'something.domain.com'.

I am not comfortable because, to my knowledge, the hostname it is just 'some random word' that I change with the $ hostname and $ hostnamectl commands. Also, I heard here that that it should be changed in /etc/postfix/Main.cf for the purpose of email delivery. If you recommend, I'd like to change the hostname to 'server.domain.com' and, I have gathered that it should match 'helo' in the email. Right now, the hostname is of the form domain.com and helo is 'domain.com' according to a recipient of an email.

This question is about how I should configure the VPS to reach this goal. My attempt for the purpose of email delivery would be something like the following. The VPS uses debian. Everything is quite generic, I use postfix, OpenDKIM and certbot. Feel free to use it as a template to start your answer, if you want, and modify as needed.

1. Should I new hostname into the terminal?

$ su
$ hostname server
$ hostnamectl set-hostname server
$ vim /etc/postfix/main.cf

Then change myhostname = domain.com to myhostname = server.domain.com mydestination = $myhostname, domain.com, localhost.com, , localhost to mydestination = $myhostname, domain.com, localhost.com, localhost

To the following file, /etc/hosts, ensure there is a line that reads

127.0.0.1 server.domain.com server

To /etc/resolv.conf add a line that reads

domain domain.com

2. Should I set up reverse dns? I assume I should set the reverse DNS to server.domain.com. Is it correct?

Other steps How should I modify the dns records? IN ADDITION to A and AAAA records to domain.com, mail.domain.com, www.domain.com and www.mail.domain.com, currently there are four email related records:

domain.com 300 MX 10 mail.domain.com
_dmarc.domain.com 3600 TXT v=DMARC1; p=reject;rua=mailto:[email protected]; fo=1
domain.com 3600 TXT v=spf1 mx a:mail.domain.com -all
mail._domainkey.domain.com 3600 TXT v=DKIM1; k=rsa; p=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

How should these be changed? Would I have to add A and AAAA records for server.domain.com?

Which other steps are required?

And if you have time to explain what each step achieves, it would be great.

Confirm that dig -x IP and host IP return server.domain.com. If not, you will need to speak with your ISP to configure reverse DNS for this IP.

For email, the rDNS needs to be setup to point to any of the sources listed in the SPF record. Either add domain.com to the SPF record or configure your rDNS to point to mail.domain.com. Then the server will be properly authorized as a sender. For postfix, myhostname should be set to the same thing rDNS is set to(this is what gets sent in the HELO by default, among other things), and mydestination should, at minimum, list domain.com. It tells postfix what domains you'll be receiving mail on behalf of, so be sure to also add whatever that may be. As far as the server's hostname goes, it should also be set to mail.domain.com.

All that being said, if your server is hosting both domain.com and mail.domain.com, there is no reason to overcomplicate things, and just having your MX record, rDNS, etc. point to domain.com will be fine. If ever you need to scale to separate servers for email and whatever is currently served on domain.com, it won't add much time to the reconfiguration process.