Azure Automation Hybrid Worker Extension Provisioning failed

I am setting up a Hybrid Worker to use in Azure Automation. The worker is added to my hybrid worker pool, and it states the platform as "Extension based (V2)". I am using a System Managed Identity. I am unsure if I need to add a User Managed Identity to the VM.

My Azure Automation account was setup as Private, but failed on adding to DNS so I recreated without adding to a PrivateLink DNS.

While the Worker shows as having the extension in Azure Automation (this is the new Extension based and not the Agent based so it's automatically installed), inspecting the VMs extensions states that the HybridWorkerExtension "Provisioning Failed" as follows:

[Internal Error] The Hybrid Worker Extension failed to execute: {"Message":"Authentication failed for private links"}. More information about the failure can be found in the logs located under 'C:\WindowsAzure\Logs\Plugins\Microsoft.Azure.Automation.HybridWorker.HybridWorkerForWindows\0.1.0.18' on the VM. For more troubleshooting, please follow the steps mentioned here: 'https://aka.ms/troubleshoot-hybrid-runbook-worker-v2'

I have searched high and low, but can't seem to find 1) where this authentication handshake needs to happen 2) the best method to set this up acknowledging that I believe the Extension agent is still in preview.

I suspect this is an issue with the Hybrid Worker communicating with Azure Automation as Azure Automation is set to Private vs. Public, but with the DNS error I received on trying to create Azure Automation originally (now skipped that step by creating a new Azure Automation as it kept stating a "conflict" on creation), how do I ensure communication?

If you have setup your Azure Automation service as private, and so using private link, then you need a solution on your vNet to resolve the privatelink.azure-automation.net zone. Without this there is no way for the worker to be able to resolve the endpoint for the service to connect to.

The easiest solution for this is to use an Azure Private DNS zone for privatelink.azure-automation.net, and add in your automation service private IP and name. You also need to make sure that there is a private endpoint and associated network interface for the automation service attached to your network.

Given the current state, it might be easiest to delete any existing private link infrastructure (but not the automation account) and recreate it, having it create the DNS zone for you.