My Win 11 Pro VPN client for IKEv2 is perpetually broken

I am tearing my hair out over this sudden refusal of Windows 11 Pro on my PC to use the appropriately configured crypto in IKEv2 negotiation. It worked fine for a long time, until it didn't. This issue persists over new installs. Meaning I have deleted partitions and reinstalled more than once. In the past this fixed the issue for a while, but no longer. This leads me to believe it may be related to updates integrated into the Win11 images I'm downloading and so can't be reversed any longer.

Back in January 2022 there was a security update that broke VPN functionality for a ton of people, and it was subsequently patched, but I wonder if it is somehow still affecting me, perhaps because of some way I configure things. The issue as far as I have been able to tell involves the hash algorithm used with the root certificate. The server is on Rocky Linux 9 using libreswan and is certificate authentication without a password. The hash algorithm showing for the certificate in the Windows system store is SHA256. However at some point Windows decided it is not going to use SHA256 anymore. It is falling back to some sort of hard-coded IPsec policy involving SHA1. This is what the logs say server-side is being proposed. SHA1 is rejected as insecure and I really have no desire to enable it. I will switch to a different VPN protocol before I do that.

I tried somewhat to configure a system IPsec policy with PowerShell, but it is a pain, and I have no idea if I am wasting my time in that, since those IPsec policies may not be applicable and I believe are more for connections with Windows Server domains and such. It has always normally followed the configuration for the VPN client itself, which I think falls under the purview of Remote Access Connection Manager (RasMan) and not IPsec policy. I was just hoping maybe I could force it with policy. I would rather just have proper configuration working properly.

The configuration it normally is supposed to use is the following:

Set-VpnConnectionIPsecConfiguration -ConnectionName "IKEv2 VPN" -AuthenticationTransformConstants GCMAES128 -CipherTransformConstants GCMAES128 -EncryptionMethod AES256 -IntegrityCheckMethod SHA256 -PfsGroup None -DHGroup Group14

However now regardless of configuration I get a this error

error processing signature payload.

I believe it would work if I allowed SHA1 on the server, though if someone more well-versed than I could explain how that is possible using a certificate with a SHA2 hash, that would be nice. Microsoft has deprecated SHA1 themselves, so it is odd that Windows clients would still fall back to it, when pretty much all the certificates in the store are going to have SHA2 hashes. There is plenty of Windows Server documentation, mostly involving L2TP and Kerberos connections and such, but very little for IKEv2 and the Windows 11 clients themselves. I have seen suggestions that replacing the system's ikeext.dll file with an older version may help, but I am not keen on installing random old DLL files from the internet in my system32 folder so I can use IKEv2.

I am unsure what to do now about a problem that can't even be fixed by completely reinstalling the OS. For a while blocking it from updating might have worked, but I think we are beyond that point now. It just doesn't work straight out of the box. Any insight would be appreciated.

It's not a problem with the signature in the certificate but the one used during IKEv2.

The classic signature scheme for RSA defined in RFC 7296 is generally limited to SHA-1. Only with the extension defined in RFC 7427 are stronger signature schemes (e.g. RSASSA-PSS with SHA-256) possible in an interoperable way. However, Windows currently doesn't support that extension, so until it does, you have to allow SHA-1 signatures for IKEv2 on the server.