Firstly, I recommend you do not link a public name to the actual VM's IP. Public names are generally linked to web sites or other endpoints on a host. By linking a public name to a machine IP, you're simply inviting malicious actors to try and crack the machine and/or fill it with malware.
However, if you want to configure different names for apps/services, presumably you bought the domain via the Azure App Service. Hopefully you didn't link it to a specific web app - if you did, you probably have to undo it.
What you need to do is configure a custom DNS record (an A record for your host). To do that, you set up a zone in Azure DNS - or a third party DNS, but I suspect that's over-complicated for your needs - and then configure the records in there to point to your host. There's a small cost for setting up an Azure DNS instance, naturally.
To continue my initial warning, from a security-by-obscurity point of view (as limited as that is), it's preferable to just manage the machine via its IP address. You can always edit a hosts file on the machine you're connecting from and provide a name for the IP there for convenience. Obviously I'm assuming you're an individual managing just one or two machines. In an enterprise environment, it'd be very different.
If you'll be managing multiple machines, it might be worth considering Azure Bastion as a more secure solution. Of course, there's an additional cost for that too.
