If you're not deploying them via Group Policy at present, I recommend you do so. This article describes how to create a custom cert template and GPO to get your servers enrolling and binding the correct cert.
The reason I suggest it - if you have an internal CA - is that it might be cleaner to simply replace the certs you have, using the GPO to force it. If you already have this setup, then tweaking the cert template and selecting the option to force all clients to re-enroll might get them to acquire and bind the new cert.
To answer your actual question, if you have multiple valid certs with the correct EKU, I believe RDP will bind with the cert that has the longest validity interval left (I did a brief search, but I can't find a reference for that right now). It might be worth finding a few machines in that situation and checking which cert thumbprint is being used for RDP vs what valid certs are in the store.
That's part of the reason I suggest issuing new certs across the board - if they're newer/have longer to run, RDP should bind to those rather than the ones with the incorrect OSCP. If that seems consistent, hopefully that'd avoid having to script a solution to force which local cert to bind. In my experience, refreshing/reissuing a valid cert works without additional steps required.