Score:2

What is changing my Windows Server's date?

na flag

At my organisation we have a number of virtual machines on ESXi hosts in a number of sites.

For the past few months, around once or twice a week one virtual machine, seemingly at random, will update the date to something weeks in the future for a few minutes, only for W32Time to notice and change it back. This is affecting the applications running on the server and we cannot figure out what could be causing it.

This is not the usual time change of a few minutes, it's shifting the date and time to something completely different, and there's no pattern to what it moves to.

The latest example can be seen in the below event log pulled from the server:

"Event 4616, Microsoft Windows security auditing. The system time was changed. Subject: Security ID: LOCAL SERVICE Account Name: LOCAL SERVICE Account Domain: NT AUTHORITY Logon ID: 0x3E5 Process Information: Process ID: 0x174 Name: C:\Windows\System32\svchost.exe Previous Time: ‎2022‎-‎10‎-‎11T19:28:42.731783900Z New Time: ‎2022‎-‎12‎-‎06T05:38:25.439000000Z This event is generated when the system time is changed. It is normal for the Windows Time Service, which runs with System privilege, to change the system time on a regular basis. Other system time changes may be indicative of attempts to tamper with the computer."

Any help or advice would be greatly appreciated!

joeqwerty avatar
cv flag
Is the VM set to sync time with the host? What is w32Time using as it's source? What is it's configuration?
Mayzed avatar
na flag
It is not set to sync time with the host and w32Time is set to use its local Read-only DC, which as far as we can tell, always has the correct time. For what it's worth - The RODCs look for their time from Azure.
MyUserName avatar
in flag
@Mayzed have you find the reason why this happens? We have got somewhat similar issue, it's very frustrating, and we still cannot find a way to solve it unfortunately.
Rohit Gupta avatar
be flag
Are you changing the locale ?
MyUserName avatar
in flag
@RohitGupta no, we don't. These VMs run the server apps, so there is no need in such changes for us.
Score:1
ru flag

Most likely, the VM is being 'updated' to the wrong time by the ESXi host.

When a VM is 'stunned' (on pause, snapshot, migration, etc.) it inherits the host's system time, regardless of the VM settings. Possibly the 'seemingly at random' times are when snapshots are taken, backups are running or similar. And very likely the other VMs are having the same problem but you just haven't noticed it yet (been there myself).

Therefore, it's very important to keep your hosts updated by NTP at all times and check their correct system time on a regular basis.

Mayzed avatar
na flag
Hello! Thanks for the reply. We went down this rabbit hole too, and our ESX hosts are indeed showing the correct time whenever we check them. So while we don’t sync time with the host, in events where the VM is in the midst of a snapshot, the time it gets from the host should be correct. It certainly would never be getting a date weeks into the future! So unfortunately this still remains a mystery to us.
Zac67 avatar
ru flag
Then I'd definitely monitor the network connection for any wanted - and unwanted - NTP and SNTP traffic. Perhaps there's a funny upstream stratum host. If the time offset always the same or is it fluctuating?
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.