Tuesday, March 09, 2004
Exchange 2000 Disaster Recovery
Just as I thought I was getting an early hometime (16:30) along pops another Exchange 2000 problem. This time, the SAN bit the dust, and took the Exchange Server with it. After replacing the failed disks in the SAN, the customer did almost everything right.... Reinstalled Exchange 2000 with the /disasterrecovery option, and then applied the service packs. Then came the biggy - restoring the databases. Each time this was attempted, there was an error at the end of the restore, and no databases were mounted. The error code was very strange 9939xxxxxxxx. What was the problem? Well quite simply, there were no actual logs to play forward, only those from the backup. Now it if was me that had written Exchange, I would've included a little check to see if there are any logs and ask the user what he wants to do - hard recovery or get the logs from somewhere else. If you ever get the same problem, do the following. Restore all databases for a Storage Group at the same time, but do not set the "Last Backup Set" flag when restoring, even on the last backup set. Once the restore is finished, navigate to the temp directory that you defined during the restore operation from a command prompt. Check for the existence of the restore.env file, and then issue the command %pathtoexchangebin%\ESEUTIL /CC /T and press enter. This will do a hard recovery, only using the logs from the current directory, and will not start looking for other logs in the mdbdata directory. Took a few hours to sort out the problem, mainly because I was using the wrong syntax, and thought that you had to specify the .edb file that you want to roll the logs forward on.
Just goes to show that the worst time to test your disaster recovery is when the disaster happens. One other thing, why is there so little documentation available about Disaster Recovery? Try a google search sometime, and see what a very poor world it is. Maybe I'll get round to doing a document one day, only time will tell.
Just as I thought I was getting an early hometime (16:30) along pops another Exchange 2000 problem. This time, the SAN bit the dust, and took the Exchange Server with it. After replacing the failed disks in the SAN, the customer did almost everything right.... Reinstalled Exchange 2000 with the /disasterrecovery option, and then applied the service packs. Then came the biggy - restoring the databases. Each time this was attempted, there was an error at the end of the restore, and no databases were mounted. The error code was very strange 9939xxxxxxxx. What was the problem? Well quite simply, there were no actual logs to play forward, only those from the backup. Now it if was me that had written Exchange, I would've included a little check to see if there are any logs and ask the user what he wants to do - hard recovery or get the logs from somewhere else. If you ever get the same problem, do the following. Restore all databases for a Storage Group at the same time, but do not set the "Last Backup Set" flag when restoring, even on the last backup set. Once the restore is finished, navigate to the temp directory that you defined during the restore operation from a command prompt. Check for the existence of the restore.env file, and then issue the command %pathtoexchangebin%\ESEUTIL /CC /T and press enter. This will do a hard recovery, only using the logs from the current directory, and will not start looking for other logs in the mdbdata directory. Took a few hours to sort out the problem, mainly because I was using the wrong syntax, and thought that you had to specify the .edb file that you want to roll the logs forward on.
Just goes to show that the worst time to test your disaster recovery is when the disaster happens. One other thing, why is there so little documentation available about Disaster Recovery? Try a google search sometime, and see what a very poor world it is. Maybe I'll get round to doing a document one day, only time will tell.
