In preparing for my company’s Exchange 2013 CU8 to CU11 uprage a few weekends ago I decided during my lunch break to the run the CU11 installer through the prerequisite check on all our servers just to make sure nothing would get in the way of the install that weekend. This is something I used to do all the time in Exchange 2010 with Service packs and Roll ups. So I started the GUI setup on one of our multi role servers hosting a lagged database in our DR site first and the prerequisite check came up clean so I exited out of the installer at the section where you would normally hit “next” to install Exchange 2013. I then ran it on one of our production servers and it came up clean as well. Figuring I should be thorough, I ran the prerequisite check on the remaining production and DR servers. Within a few minutes we started getting multiple alarms for our Exchange email environment through out monitoring system. After 45 minutes of email routing and access down time I was able to piece together that the prerequisite check runs the following which puts the following 3 Exchange 2013 component states offline
Set-ServerComponentState $Target -Component Monitoring -Requester Functional -State Inactive Set-ServerComponentState $Target -Component RecoveryActionsEnabled -Requester Functional -State Inactive Set-ServerComponentState $Target -Component ServerWideOffline -Requester Functional -State InActive
Unfortunately the setup does not put them back online if you cancel it for any reason. Early in the trouble shooting process I realized that the components states were in an inactive state and I tried to bring them back online (like I normally do when patching). What I didn’t realize was that since the Requester was “Functional” and I was using “Maintenance” to try and bring them back online, it wasn’t taking. Only after I did so using “Functional” did it take.
After doing a post mortem on the issue at work we came across the following info online
This Exchange team blog post from 09/26/2013 that states the following
While an Exchange 2013 Server is updated with CU2, the setup- sets “Monitoring”, “RecoveryActionsEnabled” and “ServerWideOffline” to Inactive using the Requester “Functional” at the beginning, as can be seen in the “ExchangeSetup”-Logfile:
However, when the update exits prematurely because it encounters an unrecoverable error-condition, it does not restore the original state. Even when the Administrator restarts all stopped Exchange services or reboots the server, the Exchange components still remain in the Inactive state.
In order to recover from this situation, you must either find the root cause for the error and remove it so that the setup completes successfully, or manually set the ServerComponentStates back to Active with the Requester “Functional”.
This issue might be fixed in future CUs and SPs.
When I looked over my Exchange install log files I did not see any entries stating an abnormal end, but it appears that simply cancelling the setup after the prerequisite check is enough to cause this condition. I persoanlly wish this was actively stated in the setup process itself (e.g. “Putting this server in ServerWideOffline mode, do you want to continue?”)
This TechNet Blog from 10/21/2014 states the following about running a CU and when the server component states are set to inactive:
In Exchange 2013, we have also extended setup to perform some of the maintenance tasks prescribed here – specifically handling server health states to ensure that the server can be safely upgraded. When performing setup.exe /mode:upgrade to upgrade between Exchange 2013 CUs we now perform the following:
- Set the monitoring state of the server to inactive.
- Prevent automatic recovery actions from occurring on the server.
- Set the ServerWideOffline component state to InActive
This essentially disables all health checking against the server, all automatic recovery actions as a result of that health checking, and prevents the server from performing transport and other client functions.
….
It should be noted that these steps are executed directly at the beginning of setup – even before pre-requisite analysis etc.
So all in all, I didn’t realize running the install up to the prerequisite check would cause so many issues. This is our first CU update since we deployed Exchange 2013 and I don’t recall running into this issue in 2010 when I deployed SPs or RUs. So hopefully this will help someone else out before they make the same mistake.
