New W2012 DC in W2008 AD: Failed (DFSRDiag is New Gold)
Most of you know I live in Hollywood so let me give this blog little Hollywood touch which might make it more interesting. I would say DFSRDiag tool is new Gold. Here is the reason.
This infrastructure had 2 x Windows 2008 R2 domain controllers in the only AD Site.
Now customer decided to upgrade all Windows 2008 R2 computers to Windows 2012 R2 to add more security.
We added 2 new windows 2012 R2 domain controllers in new VMware hosted VMs. This was a new hardware build in the same data center with the new subnet.
Issue:
- 1st Domain controllers named DC11 installed and replicated, now I tried to install 2nd domain controller named DC12 using DC11 as the replication partner but it failed so installed with Windows 2008 R2 DCs. Installed and restarted
- Checked shares from computer management and net share command but there was no SYSVOL or NETLOGON share.
- Ran DCDIAG which showed me the following result
-
Directory Server Diagnosis
Performing initial setup:
Trying to find home server…
Home Server = DC11
* Identified AD Forest.
Done gathering initial info.
Doing initial required tests
Testing server: Default-First-Site-Name\DC11
Starting test: Connectivity
……………………. DC11 passed test Connectivity
Doing primary tests
Testing server: Default-First-Site-Name\DC11
Starting test: Advertising
Warning: DsGetDcName returned information for \\DC.Domain.com, when we were trying to reach DC11.
SERVER IS NOT RESPONDING or IS NOT CONSIDERED SUITABLE.
……………………. DC11 failed test Advertising
Starting test: FrsEvent
……………………. DC11 passed test FrsEvent
Starting test: DFSREvent
There are warning or error events within the last 24 hours after the
SYSVOL has been shared. Failing SYSVOL replication problems may cause
Group Policy problems.
……………………. DC11 passed test DFSREvent
Starting test: SysVolCheck
……………………. DC11 passed test SysVolCheck
Starting test: KccEvent
……………………. DC11 passed test KccEvent
Starting test: KnowsOfRoleHolders
……………………. DC11 passed test KnowsOfRoleHolders
Starting test: MachineAccount
……………………. DC11 passed test MachineAccount
Starting test: NCSecDesc
……………………. DC11 passed test NCSecDesc
Starting test: NetLogons
Unable to connect to the NETLOGON share! (\\DC11\netlogon)
[DC11] An net use or LsaPolicy operation failed with error 67, The network name cannot be found.
……………………. DC11 failed test NetLogons
Starting test: ObjectsReplicated
……………………. DC11 passed test ObjectsReplicated
Starting test: Replications
……………………. DC11 passed test Replications
Starting test: RidManager
……………………. DC11 passed test RidManager
Starting test: Services
……………………. DC11 passed test Services
Starting test: SystemLog
……………………. DC11 passed test SystemLog
Starting test: VerifyReferences
……………………. DC11 passed test VerifyReferences
Running partition tests on : DomainDnsZones
Starting test: CheckSDRefDom
……………………. DomainDnsZones passed test CheckSDRefDom
Starting test: CrossRefValidation
……………………. DomainDnsZones passed test
CrossRefValidation
Running partition tests on : ForestDnsZones
Starting test: CheckSDRefDom
……………………. ForestDnsZones passed test CheckSDRefDom
Starting test: CrossRefValidation
……………………. ForestDnsZones passed test CrossRefValidation
Running partition tests on : Schema
Starting test: CheckSDRefDom
……………………. Schema passed test CheckSDRefDom
Starting test: CrossRefValidation
……………………. Schema passed test CrossRefValidation
Running partition tests on : Configuration
Starting test: CheckSDRefDom
……………………. Configuration passed test CheckSDRefDom
Starting test: CrossRefValidation
……………………. Configuration passed test CrossRefValidation
Running partition tests on : exchange
Starting test: CheckSDRefDom
……………………. exchange passed test CheckSDRefDom
Starting test: CrossRefValidation
……………………. exchange passed test CrossRefValidation
Running enterprise tests on : Domain.com
Starting test: LocatorCheck
……………………. Domain.com passed test LocatorCheck
Starting test: Intersite
……………………. Domain.com passed test Intersite
Overall Netlogon failed and Advertising failed means DC will not server any operation.
Resolution:
-
This was a new deployment so we thought it could be a non-windows related issue. In fact, there should not be any issue in the new deployment.
-
We engaged our network and virtualization team for months to get into the route cause.
-
At this point, all our superheroes had bite the dust from Ironman to Captain America.
-
We could only find the resolution to do authoritative synchronization (D4) which was the last resort. We can say this is the Ultron (the savior).
-
To complete the authoritative steps, we needed DFSRDiag.exe which is not installed by default in Windows 2012 R2 and AD RSAT does not add it either. A very good question for Microsoft Windows Product Group team which I have already asked Rob Hindman.
-
To install DFSRDiag go to the Server Manager and add a feature here. “Remote Server Administration Tools” -> “Role Administration Tools” -> “File Services Tools” -> “DFS Management Tools”. Installed DFSRDiag on all Domain Controllers
-
Now here comes Caution: Protect yourself by backing up your AD just before this change. Simple take system state backup of one of the working DCs.
-
Opened ADSIEDIT.MSC tool and modified two attributes in the following distinguished name (DN) value on the domain controller you want to make authoritative (preferably the PDC Emulator, which is usually the most up to date for SYSVOL contents): We did it on Windows 2008 R2 DC which was the PDC E.
CN=SYSVOL Subscription,CN=Domain System Volume,CN=DFSR-LocalSettings,CN=<the server name>,OU=Domain Controllers,DC=<domain>
msDFSR-Enabled=FALSE
msDFSR-options=1
-
Modified the single attribute in the following DN on all other domain controllers in the same domain:
CN=SYSVOL Subscription,CN=Domain System Volume,CN=DFSR-LocalSettings,CN=<each other server name>,OU=Domain Controllers,DC=<domain>
msDFSR-Enabled=FALSE
-
Forced Active Directory replication throughout the domain and validated its success on all DCs.
We used AD site and services and the following command on every DC.
Repadmin /syncall /force
Most important point here to validate on every domain controller so go slow and don’t be Ironman here. I would suggest open Adsiedit in every domain controller and connect to the self so that we can verify if changes are replicating in their local AD database.
-
Restarted the DFS Replication service on PDC E:
You will see Event ID 4114 in the DFSR event log indicating SYSVOL is no longer being replicated.
-
Restarted the DFS Replication service on all other DCs and verify Event ID 4114.
-
On the same DN on the authoritative DC where we edited 2 attributes, we changed the following attribute:
msDFSR-Enabled=TRUE
-
Forced Active Directory replication throughout the domain using ADSS & Repadmin command.
Repadmin /syncall /force
-
Then validate its success on all DCs using Adsiedit. It is like verifying Ultron woke up and ready. So, ensure replication on all DCs.
-
Ran the following command from an elevated command prompt on the authoritative DC where we had changed the attribute to True:
DFSRDIAG POLLAD
-
We saw Event ID 4602 in the DFSR event log indicating SYSVOL has been initialized. That domain controller has now done a “D4” of SYSVOL.
- Restarted the DFS Replication service on the other non-authoritative DCs. We saw Event ID 4114 in the DFSR event log indicating SYSVOL is no longer being replicated on each of them.
-
Modified the following DN and single attribute on all other domain controllers in that domain:
CN=SYSVOL Subscription,CN=Domain System Volume,CN=DFSR-LocalSettings,CN=<each other server name>,OU=Domain Controllers,DC=<domain>
msDFSR-Enabled=TRUE
-
Ran the following command from an elevated command prompt on all non-authoritative DCs:
DFSRDIAG POLLAD
-
Now checked the Event log for Event ID 4604 and we could see the replication working which means we had the shares.
- We verified the Netlogon and Sysvol shares which were available now on all DCs including DC11 and DC12.
- We ran DCDiag and all the tests passed on all Domain Controllers.
The following reference link was used.
Overall, this was an unexpected issue and there is no clear cause mentioned anywhere. I would expect Product Group will come up with a solution to this and I will share here.
CTO @ Golden Five
Team@MSExchangeGuru
