I’ve been looking a lot at backing up and recovering domain controllers from disaster lately. It is something I haven’t had to do a whole lot of (thankfully) in the past so I wanted to do some research and a little labbing to understand possible actions that can be taken.
Obviously one of the most common things that can be done to recover a domain controller is restoring the system state from backup. A system state backup is actually the bare minimum that is needed to recover Active Directory in the event of disaster. However, because the system state consists of very critical components of the operating system, it can be notoriously quirky when being restored. So, in this post I just wanted to list some of the things I’ve noticed and would want to keep in mind when looking at restoring the system state.
I. When Should You Restore the System State?
First thing to consider is why are you thinking about restoring the system state? From my experience, the best reasons to perform a restore of the system state on a domain controller are if you need to authoritatively restore an object that was deleted or if you have no other functioning domain controllers online and are trying to recover a domain or forest.
For other situations where you have trouble with your DC, it may just be a whole lot easier to rebuild the domain controller, reinstall Windows, and then DCPromo the machine. This is called restoring a domain controller through reinstallation (http://technet.microsoft.com/en-us/library/cc785849%28WS.10%29.aspx) This technique allows you to perform a clean and stable installation of Windows and then lets Active Directory replication do all the work to bring the domain controller up to date.
Of course, this solution is dependent upon having at least one other functioning and non-corrupt domain controller in the domain online. It also assumes you don’t have anything else installed on the domain controller that would need to be restored from backup.
II. Shelf Life of System State Backups
Whether you are restoring the system state because you need to restore an object that was deleted or you have rebuilt a domain controller, you should make certain your system state backup is not older than the tombstone lifetime of the forest. For forests created with Windows 2003 SP1 and later DCs this is 180 days, which is usually plenty. However for pre-SP1 and earlier versions of Windows Server, it was 60 days.
Trying to restore a DC with a system state backup older than the tombstone lifetime can introduce serious problems into your environment such as lingering objects (I plan on doing a more in-depth post about these in the future, btw). As a result, Windows Server will actually try to block you from restoring a system state backup older than the tombstone lifetime, but you can bypass this by adjusting the system clock on the DC you are restoring 🙂
III. Naming Backup Files
I came across this recommendation from Microsoft about naming system state backups. I don’t know if any one in the real world actually uses it, but I thought it was kind of interesting and may be useful if you have a lot of domain controllers to keep track of.
On a side note, the MD5 Microsoft refers to is related to the MD5 checksum of the files in the SYSVOL folder. You can run ntfrsutl idtable to view all the files in SYSVOL and see if they have a checksum or not. For all the files and folders to include an MD5, you may have to modify every file in the SYSVOL tree. See http://support.microsoft.com/kb/311078 for much more in-depth information regarding the relevance of the MD5 when restoring SYSVOL.
Windows 2008 Specific: This naming convention works when you are dealing with individual BKF files that Ntbackup creates, but not so well with the new Windows Server Backup utility in Windows 2008 as it has its own way of organizing backups in the WindowsImageBackup folder.
Note: The rest of this post will talk about the requirements for restoring the system state after you have performed a fresh installation of Windows either to the same machine or possibly even another machine. Interestingly enough, this type of recovery is not technically supported by Microsoft and is suppose to be used in last ditch attempts to recover the system. System state restores are technically only supported to the same operating system instance and not new- or re-installations (http://support.microsoft.com/kb/249694).
IV. Hardware Requirements
What happens in cases where you have had to do a fresh install of Windows before restoring the system state?
In those cases, it is always best if you can restore to identical hardware. The system state you restore is going to contain information about the devices installed on the machine. Also, while the system state does not specifically include the HAL from the backed up machine, it does include the kernel (ntoskrnl.exe for example). The kernel must be compatible with the HAL. So, if the machine you are restoring the system state to has a different HAL than the machine the backup was taken from, you may end up restoring a kernel from the system state backup that is not compatible with the currently configured HAL and you will have problems booting your machine.
There are workarounds of course. For example, with Windows 2003 you could manually copy the correct HAL and kernel to the System32 folder. Or you could perform a repair installation of Windows. For Windows 2008, you can run the bcdedit /set detecthal yes command after restoring, but before rebooting to force the loader to detect what HAL and kernel to load. Ultimately the point is, if you are going to restore to different hardware, you need to test, test, test, and just as importantly document what you do. It will make things much easier in case of disaster.
Funnily enough, I have actually had some decent success restoring to different hardware. For example, in my testing, I had a DC running on an old Dell computer. I managed to create a virtual machine on my ESXi host, install Windows 2003, and then restore the system state backup of the physical machine. The differences between the physical machine and virtual machine included differing HALs, NICs, disk controllers, and video cards. Despite this, the restore worked and Active Directory functioned fine. There were a couple of kinks to work out, though.
For instance, as I mentioned the system state restores all information about devices that were installed at the time of backup. My virtual machine was using an Intel network card while my physical DC had a 3COM NIC installed. After restoring the system state, I had 2 network connections in Control Panel: one for the Intel NIC that was actually installed in the VM and one for the 3COM that was a part of the physical machine. This network connection was actually disabled since the NIC was not installed, but it did maintain all the static IP settings while the Intel NIC did not have any default settings. So I had to change the IP configuration on the Intel NIC to that of what my 3COM was configured to so my VM could work on the network.
In this case, it was a simple fix, but it could be something more complicated like different disk controllers. This could result in the always annoying INACCESSIBLE_BOOT_DEVICE blue screen and you not being able to boot your machine after performing the restore.
V. Operating System Requirements
When restoring the system state, you are also going to want to make sure that you restore to a machine with the same operating system installed. This means same SKU, build number, and service pack level. For example, if you are restoring a system state backup taken from a machine running 32-bit Windows 2003 Enterprise with SP2, you are going to want to restore to a machine running this exact same version.
As far as I can tell, you do not need to install hotfixes that were previously installed before restoring the system state backup. These will be included as part of the system state.
Note about Boot.ini: If you are restoring Windows 2003 to new hardware, you have to take care that the boot.ini file you restore from the system state is appropriate as well. Microsoft actually recommends copying the boot.ini file before restoring the system state and then restoring it to its original location after the system state is restored, but before you reboot the machine (http://support.microsoft.com/kb/249694).
VI. Disk Configuration
The paths for your Windows installation and your Active Directory logs and database files must be kept the same. So, if you make a system state backup from a machine where Windows is installed to C:\Windows, when you restore the system state, Windows must be in this same path.
I have found the disk size does not need to be the same when performing a system state restore, but obviously you will need enough free space on the drive you are restoring to. I verified this when I restored the system state of my physical DC to a VM. My physical machine had a 40GB hard drive, but my VM only had 20 GB allocated to it.
You may want to install any other drivers that are installed at the time when you took the system state backup if you restoring the system state to a new installation of Windows.
For example, in my lab I had a virtual machine that had VMWare tools installed. When I tried to restore the system state to another VM that I had done a fresh install of Windows on, my mouse did not work.
The most important drivers you may need to reinstall are any 3rd party disk controller drivers. As mentioned in the hardware requirements section, if you are using a different disk controller on the machine you are restoring to, you may have problems like the INACCESSIBLE_BOOT_DEVICE blue screen.
Before you restore the system state, you may want to install any applications that were previously installed. Because the system state restores the registry, you will find that Add/Remove Programs will list any applications that were installed at the time of backup. Of course, if these applications were not reinstalled they will not work. You may also have trouble removing them from the Add/Remove Programs applet.
In my testing, I had a DC that also had the vSphere Client installed. When I restored this DC’s system state to a fresh installation of Windows, vSphere Client was in the Add/Remove Programs, but I could not remove it. Trying to reinstall it also resulted in an error. Essentially, I was stuck. There is probably a way to manually hack away at this and reinstall it, but the point is, if your programs are not already installed when you restore your system state, you may have wacky results when you try to reinstall them.
IX. Internet Explorer
I want to make a special note of Internet Explorer. Since it is tightly integrated with the operating system, it is important to have the same version installed when the system state is restored. In my lab, I had a Windows 2003 machine with IE8 installed. I then performed a fresh installation of Windows 2003 with the default IE6 on another machine. I restored the system state to this new machine and the restore completed, but upon rebooting I could not launch any programs. MMCs, Server Manager, even programs like ipconfig and nslookup gave errors about missing DLLs when trying to start. Windows was also unable to install new devices such as the network card (which happened to be different from the NIC that was included in the system state backup). Installing IE8 solved the problem.
X. Computer Name, IP Configuration, and Domain Membership
I have seen some resources say that you need to use the same computer name when restoring the system state. But, when the system state is restored, the computer will be renamed to the name that was in use at the time of backup.
The same is true for domain membership. When you are restoring the system state of a domain controller to a fresh installation of Windows, it is important not to join the computer to the domain before restoring. The restored system state will include domain membership.
The same is true for IP configuration. If you are restoring to a machine with the same network card, the system state will include any IP address and DNS settings.
When restoring the system state, you may also have to decide whether you want to mark the restored SYSVOL as being authoritative. By default, restoring the system state non-authoritatively restores the SYSVOL.
A non-authoritative restore of SYSVOL may be fine if you have other domain controllers online that have the correct version of SYSVOL. In this case, when you bring the restored DC back online, replication will take care of updating the SYSVOL on the restored machine.
However, in some cases you may want to mark the restored domain controller’s copy of SYSVOL as authoritative so that it will actually be the master copy and will push it’s contents to all other domain controllers when replication takes place.
You will also need to mark the SYSVOL as authoritative when you perform a domain or forest recovery and you are restoring the first DC in the domain. The reason for this is that if the SYSVOL is non-authoritatively restored and the restored DC has knowledge of other replication partners, the domain controller will discard it’s current copy of SYSVOL and will have to replicate with it’s partner DCs. If it cannot replicate with it’s partner DCs to rebuild the SYSVOL, the SYSVOL folder will not be available on that DC. If the SYSVOL folder is not available on a DC, the DC will not advertise itself as a DC! By marking the SYSVOL as authoritative, you are telling the DC that it has the master copy and there is no need to have to replicate with its partner DCs.
On Windows 2003, you can mark the SYSVOL as authoritative by choosing the Advanced options of a restore job. You will then have the chance to check the “When restoring replicated data sets, mark the restored data as the primary data for all replicas” option.
In Windows 2008, you can use the -authsysvol switch of wbadmin.exe to produce the same result.
XII. Directory Services Restore Mode
In Windows 2003, after you do a new installation of Windows, you can restore the system state of a domain controller within Windows. There is no need to boot into DSRM.
However, in Windows 2008, if you try to do a system state restore from within Windows after a fresh installation, you will get an error stating “The backup contains the Active Directory Domain Service which can be recovered only when the computer is started in Directory Services Restore Mode (DSRM).”
At first glance you may be thinking how can I boot into DSRM if this machine is not a domain controller yet. Well, you can actually boot into DSRM even if the computer is not a DC. Looking at the boot menu options, you will see DSRM is a choice and once you boot into that, you will be able to perform the system state restore.
As you can see, there can be a lot of things to consider when restoring the system state to a new installation of Windows.
As such, it seems like if you wish to recover a domain controller, your best option may be to just do a reinstall of Windows and DCPromo the machine. However, restoring the system state may be needed in the following cases:
- You need to perform an authoritative restore. In this case you’d be restoring the system state to the same machine with a current installation of Windows. This shouldn’t be too much of a problem if the system state is within the tombstone lifetime and you haven’t made any major hardware changes since the last backup.
- You had to wipe the current machine and reinstall Windows in a situation where no other domain controllers are available. This shouldn’t be too difficult as you are restoring to identical hardware. You will have to ensure the OS is the correct version including service pack level, reinstall any applications or drivers that may be needed, and ensure the logical disk configuration is as it was at the time the backup was taken.
- You had to install Windows to another machine and there are no other domain controllers available such as in the case of a disaster recovery. This can be difficult to perform if the hardware is different. All the other considerations still apply such as OS version, applications that need to be installed, etc.
How to move a Windows installation to different hardware
Restoring a Domain Controller Through Reinstallation and Subsequent Restore from Backup
Restoring a Domain Controller Through Reinstallation