Ran into a fun issue today… I had a pair of Server 2012 R2 servers in a remote office that refused to sync the proper time for their clocks. No matter what I did they were always off by five minutes. One of them was a domain controller for the office.
In the process of fixing the issue I learned about an interesting feature in hyper-v that was the root cause of all my trouble. By sharing my experience, hopefully you will avoid the same issues I ran into.
The setup was basically this:
- Physical Domain Controller in local office running all PDC emulator roles including being the central time server. Time is correct.
- Hyper-V Physical Host Server running Server 2012R2 located in remote office. Clock is off by 5 minutes.
- Server 2012 R2 Domain Controller – Virtual Machine running on Hyper-V Host server. Clock is off by 5 minutes.
No matter what I did from the command line or the registry I couldn’t get the time to sync correctly. The time on my server in our home office was correct, and my remote domain controller was set to sync its time to the domain controller in the home office. The Hyper-V Host was set to sync its time to the virtual domain controller. All should be be well…
But actually not… There is a feature in Hyper-V I was unaware of that basically allows all of the Virtual Machines to sync their time to the physical host. This feature is set to “on” by default and it over-rules the windows 32 time service (w32time).
The clock on the Hyper-V physical host had drifted by 5 minutes. It was set to sync to the client Virtual AD server… which was having its time set by the physical host. And around and around we go. I had a time-sync loop on my hands (not nearly as cool as a quantum “time loop” but perhaps less of a headache).
This fix was to disable the “feature” that was causing the VM client machine to sync to the host.
To do this I went into Hyper-V console on the host machine, right-clicked on the client VM (the virtual AD server) and selected properties. Once in here, on the left look under management –> integration services –> untick time-synchronization –> apply/save
Once you have those services turned off, W32time service becomes authoritative again. You can open-up an administrative powershell prompt on the box and force a sync of your time with the following command:
If you need to check what your server is actually syncing to (I found this command very helpul):
If you need to change to a different sync partner (like an outside NTP server, not really recommended in a Windows environment for Domain Controllers but you can…):
net stop w32time
net start w32time
w32tm /resync
That pretty much covers it! Hopefully this will save a fellow virtual admin some headache!
I hit so much contradictory information about this and because my PDC emulator is running on Hyper-V it just went around and around, never able to get the time to update. But yes once I cleared the time sync settings on the DCs on my Hyper-V servers I was able to sort the domain time out. Guess the question is.. what kind of risk is involved by doing this? Have you encountered any issues by doing this?
My other option is to shift the PDC emulator role it to a physical DC I guess..
If your PDC emulator is a VM I would think that most definitely the option to sync to the Hyper-V host SHOULD NOT be checked. Here is my line of reasoning… (and this article’s actually)
I am assuming your Hyper-V host box is joined to the domain that your VM is hosting all the FISMO roles for including PDC emulator. So… Logic goes like this in my head…
PDC Emulator is the ultimate authority on time for the whole domain – It normally syncs to an external Microsoft Time server, in this case, it is syncing to the Hyper-V host if that option is turned on. All other member servers sync to the DC nearest them and all DC’s sync to the DC hosting the PDC role. The Hyper-V host is ALSO going to sync to the the DC housing the PDC role. And now we have a loop…
Hyper-V host syncs to the PDC emulator which is a guest virtual machine running on the Hyper-V host and it therefore is trying to sync to the hyper-v host which is trying to sync to it… and around and around it would go I would guess?
In my case, my virtualized domain controllers were not hosting the PDC role but the same issue still ensued because they weren’t syncing to the server that was, they were syncing to the Hyper-V host. The Hyper-V host was closes to the Domain Controllers that were virtualized on that host so it was trying to sync to them. Same issue. However if it is the PDC that is virtualized I would guess the issue could become a magnitude or so worse as ALL machines in your domain are going to be affected eventually.
So long story short, I think it is safe to say that anytime you virtualize a DC, that option should be turned off.
Pingback: Hyper-V – Dynamic Memory – Time Sync Issues on the VM « KiloRoot
Pingback: What ports on the firewall should be open between Domain Controllers and Member Servers? « KiloRoot
Awesome! Thanks worked for me.
Glad to hear it!
Pingback: The Windows Time Service, Hyper-V Hosts, and DCs that are VMs. – IT Blog
I want to say that the one caveat that I know of here is that snapshots use host time to sync up when they get turned back on. I think if you have this turned off, snapshots can cause issues if there’s no initial time sync when it boots up. I haven’t developed a fix for that in my environment yet so I just avoid snapshots altogether or only use them sparingly to go back a few minutes from a quick change.
This may not be an issue anymore but just in case, I figured it was worth mentioning.