40 KiB
tags
| tags | |||||
|---|---|---|---|---|---|
|
Purpose
This workflow applies Exchange Server Subscription Edition (SE), Windows Server, and approved prerequisite updates to a three-member database availability group (DAG) by updating one DAG member at a time. The procedure drains client and transport activity, moves active mailbox databases, places the target member into maintenance mode, installs updates, validates the updated member, and restores the intended database placement before the next cycle begins.
All organization names, hostnames, FQDNs, DAG names, database names, and example values in this page describe the fictional Bunny Lab environment. The examples use the bunny-lab.io DNS namespace and do not identify another organization.
!!! warning "Update One DAG Member at a Time" Only one DAG member may be in maintenance mode at a time. Do not begin the next cycle until the previous member has returned to service, all database copies are healthy, all copy and replay queues have drained, transport queues are clear, and replication health passes across the entire DAG.
Assumptions and Risk Boundaries
- The Exchange organization is running Exchange Server SE on three Mailbox servers in one DAG.
- Every mailbox database has at least two healthy passive copies before maintenance begins.
- A current backup and a tested Exchange recovery path exist. DAG replication provides availability, but it is not a substitute for backup.
- The operator has the Exchange and local administrative permissions required by the selected update. A CU that extends the schema or prepares Active Directory may require additional directory permissions before the first server is upgraded.
- The administrative shell host remains online and is not the current maintenance target.
- The Exchange DAG maintenance scripts are available through
$ExScripts, and the administrative shell host has the Failover Clustering management tools installed. - The current Exchange release notes, prerequisites, known issues, and update-specific manual actions have been reviewed.
- The required Exchange update media and Windows updates are approved and staged before the maintenance window begins.
- Any load balancer, monitoring platform, backup platform, mail gateway, or third-party application that targets an individual Exchange server has an established drain and restore procedure.
- All DAG members are returned to the same Exchange CU, SU, and HU level during the rolling update window.
!!! danger "Do Not Use a Snapshot as the Only Recovery Plan" Do not begin the rolling update without an Exchange-aware backup and documented recovery method. If an update fails, keep the affected server isolated in maintenance mode and repair that server before continuing to another DAG member.
Example Bunny Lab Environment
Exchange Topology
| Object | Example Value |
|---|---|
| Organization | Bunny Lab |
| Active Directory DNS domain | bunny-lab.io |
| DAG | BL-DAG-01 |
| Exchange version | Exchange Server Subscription Edition |
| Update staging directory | C:\ExchangeUpdates |
| Support scripts directory | C:\Scripts |
DAG Members
| Short Name | FQDN | Normal Role |
|---|---|---|
EXCH-SE-01 |
EXCH-SE-01.bunny-lab.io |
DAG member and primary administrative shell host |
EXCH-SE-02 |
EXCH-SE-02.bunny-lab.io |
DAG member and alternate administrative shell host |
EXCH-SE-03 |
EXCH-SE-03.bunny-lab.io |
DAG member |
Exchange cmdlets in this page normally use the Exchange server object name, such as EXCH-SE-01. Commands that require an FQDN, including Redirect-Message -Target, use the corresponding bunny-lab.io FQDN.
Intended Database Placement
| Active Database | Intended Active Server | Passive Copy Servers |
|---|---|---|
BL-MBX-01 |
EXCH-SE-01 |
EXCH-SE-02, EXCH-SE-03 |
BL-ARC-01 |
EXCH-SE-01 |
EXCH-SE-02, EXCH-SE-03 |
BL-MBX-02 |
EXCH-SE-02 |
EXCH-SE-01, EXCH-SE-03 |
BL-ARC-02 |
EXCH-SE-02 |
EXCH-SE-01, EXCH-SE-03 |
BL-MBX-03 |
EXCH-SE-03 |
EXCH-SE-01, EXCH-SE-02 |
Rolling Upgrade Plan
| Cycle | Administrative Shell Host | Maintenance Target | Transport Redirect Target | Temporary Database Placement |
|---|---|---|---|---|
1 |
EXCH-SE-01 |
EXCH-SE-03 |
EXCH-SE-01.bunny-lab.io |
BL-MBX-03 to EXCH-SE-01 |
2 |
EXCH-SE-01 |
EXCH-SE-02 |
EXCH-SE-03.bunny-lab.io |
BL-MBX-02 to EXCH-SE-01; BL-ARC-02 to EXCH-SE-03 |
3 |
EXCH-SE-02 |
EXCH-SE-01 |
EXCH-SE-03.bunny-lab.io |
BL-MBX-01 to EXCH-SE-02; BL-ARC-01 to EXCH-SE-03 |
This order is specific to the fictional topology above. In another environment, choose an order that preserves quorum, keeps an administrative shell host available, and distributes active databases across healthy remaining members.
Prepare the Exchange Updates
Determine the Required Exchange Build
From Exchange Management Shell on a healthy DAG member, record the base Exchange build stored in Active Directory:
$DagMembers = @("EXCH-SE-01", "EXCH-SE-02", "EXCH-SE-03")
$DagMembers | ForEach-Object {
Get-ExchangeServer -Identity $_
} | Format-Table Name, Edition, AdminDisplayVersion -Auto
AdminDisplayVersion identifies the base release or CU, but it does not reliably identify the installed SU or HU. Query the local ExSetup.exe file on every DAG member to record the full installed file version:
Invoke-Command -ComputerName $DagMembers -ScriptBlock {
$VersionInfo = (Get-Item (Join-Path $env:ExchangeInstallPath "bin\ExSetup.exe")).VersionInfo
[PSCustomObject]@{
Server = $env:COMPUTERNAME
ProductVersion = $VersionInfo.ProductVersion
FileVersion = $VersionInfo.FileVersion
}
} | Format-Table -Auto
Compare the results with the current Microsoft build table and the release article for the intended update. Install the latest supported CU required for the target release, then install the latest applicable SU or HU according to that release article. Do not install an SU or HU that was built for a different CU.
!!! warning "Release-Specific Instructions Take Precedence" Review the selected update's prerequisites, Active Directory preparation requirements, known issues, and manual post-installation actions before changing the first DAG member. This generic workflow does not replace release-specific Microsoft instructions.
Run the Exchange Health Checker
Download and stage the current Microsoft Exchange Health Checker script before the maintenance window. From an elevated PowerShell session on the administrative shell host, run it against every DAG member:
$DagMembers | ForEach-Object {
& "C:\Scripts\HealthChecker.ps1" -Server $_
}
Resolve update-blocking findings before continuing. Preserve the generated reports with the maintenance record so the pre-update and post-update states can be compared.
Stage Update Media
Create the local staging directory on every DAG member:
Invoke-Command -ComputerName $DagMembers -ScriptBlock {
New-Item -Path "C:\ExchangeUpdates" -ItemType Directory -Force | Out-Null
}
Copy the approved Exchange CU media, SU or HU package, prerequisite installers, and any required scripts to C:\ExchangeUpdates on every DAG member. Use the exact package linked by the applicable Microsoft release article and verify that the file is complete before the maintenance window begins.
Select the Current Upgrade Cycle
Set these variables in Exchange Management Shell on the administrative shell host before running the common workflow. Run only the block for the current cycle.
Cycle 1: Update EXCH-SE-03
Run from Exchange Management Shell on EXCH-SE-01:
$DagName = "BL-DAG-01"
$DagMembers = @("EXCH-SE-01", "EXCH-SE-02", "EXCH-SE-03")
$AdminHost = "EXCH-SE-01"
$TargetServer = "EXCH-SE-03"
$RedirectTargetFqdn = "EXCH-SE-01.bunny-lab.io"
Cycle 2: Update EXCH-SE-02
Run from Exchange Management Shell on EXCH-SE-01 after Cycle 1 is fully validated:
$DagName = "BL-DAG-01"
$DagMembers = @("EXCH-SE-01", "EXCH-SE-02", "EXCH-SE-03")
$AdminHost = "EXCH-SE-01"
$TargetServer = "EXCH-SE-02"
$RedirectTargetFqdn = "EXCH-SE-03.bunny-lab.io"
Cycle 3: Update EXCH-SE-01
Run from Exchange Management Shell on EXCH-SE-02 after Cycle 2 is fully validated:
$DagName = "BL-DAG-01"
$DagMembers = @("EXCH-SE-01", "EXCH-SE-02", "EXCH-SE-03")
$AdminHost = "EXCH-SE-02"
$TargetServer = "EXCH-SE-01"
$RedirectTargetFqdn = "EXCH-SE-03.bunny-lab.io"
Confirm that the current Exchange Management Shell session is running on $AdminHost and that $AdminHost is not equal to $TargetServer:
[PSCustomObject]@{
CurrentComputer = $env:COMPUTERNAME
AdminHost = $AdminHost
TargetServer = $TargetServer
}
Do not continue if CurrentComputer does not match AdminHost, or if the administrative shell host is not fully healthy.
Capture a Pre-Update Service Baseline
Capture the Exchange-related service state on each server before the first maintenance action. Run this block once for the current target server from an elevated PowerShell session:
Invoke-Command -ComputerName $TargetServer -ScriptBlock {
Get-CimInstance Win32_Service |
Where-Object { $_.Name -like "MSExchange*" -or $_.Name -eq "FMS" } |
Select-Object Name, DisplayName, State, StartMode, ExitCode |
Export-Csv -Path "C:\ExchangeUpdates\PreUpdate-Services.csv" -NoTypeInformation
}
Do not replace this baseline with a service list copied from another organization. Exchange service startup modes can differ because of installed roles, enabled protocols, product version, and local design decisions.
Validate DAG Health Before Each Cycle
Check Database Copy Health
From the administrative shell host, check every database copy:
Get-MailboxDatabaseCopyStatus * |
Sort-Object Name |
Format-Table Name, Status, CopyQueueLength, ReplayQueueLength, ContentIndexState -Auto
The preflight passes only when:
- Every active copy reports
Mounted - Every passive copy reports
Healthy - No copy reports
Failed,Suspended,Disconnected,ServiceDown, or another unresolved failure state - Every copy queue is
0 - Every replay queue is
0, or is low and demonstrably draining before any state-changing action ContentIndexStatematches the expected Exchange SE state;NotApplicableis normal for the BigFunnel search architecture and is not, by itself, a failure
Check Replication Health
Run replication health against every DAG member:
$DagMembers | ForEach-Object {
Test-ReplicationHealth -Identity $_
}
Every applicable check must return Passed, and the Error field must be blank. Investigate any failure before moving a database or entering maintenance mode.
Check Cluster State and Primary Active Manager
Confirm every cluster node is up and identify the current Primary Active Manager (PAM):
Get-ClusterNode | Format-Table Name, State, NodeWeight, DynamicWeight -Auto
Get-DatabaseAvailabilityGroup -Identity $DagName -Status |
Format-List Name, PrimaryActiveManager, OperationalServers
All DAG members must be operational before the cycle begins. The maintenance script will move critical DAG functionality away from the target and pause its cluster node.
Check Exchange Service Health
Check Exchange service health on the target and the administrative shell host:
Test-ServiceHealth -Server $TargetServer
Test-ServiceHealth -Server $AdminHost
Resolve any required service failure before continuing.
Check Active Database Placement
List the currently mounted copies:
Get-MailboxDatabaseCopyStatus * |
Where-Object { $_.Status -eq "Mounted" } |
Sort-Object ActiveDatabaseCopy, Name |
Format-Table Name, Status, ActiveDatabaseCopy, CopyQueueLength, ReplayQueueLength -Auto
Compare the result with the intended placement table. A database may be temporarily active on another healthy member because of an earlier event, but its current state and all candidate copies must be understood before the maintenance move begins.
Check Transport Queues
Inspect transport queues on the target before draining it:
Get-Queue -Server $TargetServer |
Sort-Object MessageCount -Descending |
Format-Table Identity, Status, MessageCount, NextHopDomain -Auto
Investigate growing, retrying, or unreachable queues before maintenance. Redirecting a queue does not correct an underlying transport or name-resolution failure.
!!! warning "Preflight Stop Conditions" Stop the cycle if any database copy is failed or suspended, replication health does not pass, a required Exchange service is unavailable, a cluster node is down, quorum is at risk, transport queues are persistently growing, the update prerequisites are unresolved, or the recovery path is unavailable.
Move Active Databases Away From the Target
Confirm Candidate Copies
Before each move, confirm that the chosen destination server holds a healthy passive copy with drained queues:
Get-MailboxDatabaseCopyStatus * |
Sort-Object Name |
Format-Table Name, Status, CopyQueueLength, ReplayQueueLength, ActiveDatabaseCopy -Auto
Confirm the selected destination copy reports Healthy with CopyQueueLength and ReplayQueueLength equal to 0 before moving the active database.
!!! warning "Run Only the Current Cycle's Move Block" The following move blocks are cycle-specific. Do not run move commands for a different maintenance target.
Cycle 1 Database Move
Move BL-MBX-03 from EXCH-SE-03 to EXCH-SE-01:
Move-ActiveMailboxDatabase -Identity "BL-MBX-03" -ActivateOnServer "EXCH-SE-01" -Confirm:$false
Cycle 2 Database Moves
Distribute the two active databases from EXCH-SE-02 across the remaining healthy members:
Move-ActiveMailboxDatabase -Identity "BL-MBX-02" -ActivateOnServer "EXCH-SE-01" -Confirm:$false
Move-ActiveMailboxDatabase -Identity "BL-ARC-02" -ActivateOnServer "EXCH-SE-03" -Confirm:$false
Cycle 3 Database Moves
Distribute the two active databases from EXCH-SE-01 across the remaining healthy members:
Move-ActiveMailboxDatabase -Identity "BL-MBX-01" -ActivateOnServer "EXCH-SE-02" -Confirm:$false
Move-ActiveMailboxDatabase -Identity "BL-ARC-01" -ActivateOnServer "EXCH-SE-03" -Confirm:$false
Validate the Database Moves
Confirm that no active database remains on the target:
Get-MailboxDatabaseCopyStatus * |
Where-Object { $_.Status -eq "Mounted" -and $_.Name -like "*\$TargetServer" } |
Format-Table Name, Status, ActiveDatabaseCopy, CopyQueueLength, ReplayQueueLength -Auto
Expected output:
<no output>
Review each move result and confirm that Status is Succeeded, NumberOfLogsLost is 0, and MountStatusAtMoveEnd is Mounted. If any move fails or reports log loss, stop the cycle and investigate before entering maintenance mode.
Place the Target DAG Member Into Maintenance Mode
Drain External Client Traffic
If the environment uses a load balancer, reverse proxy, monitoring probe, backup job, or third-party connector that targets individual Exchange servers, drain or disable the target member according to that platform's documented procedure. Confirm that healthy remaining members are serving the traffic before continuing.
Drain Hub Transport
From Exchange Management Shell on the administrative shell host, set the target Hub Transport component to draining:
Set-ServerComponentState -Identity $TargetServer -Component HubTransport -State Draining -Requester Maintenance
Restart the Microsoft Exchange Transport service on the target to initiate queue draining:
Invoke-Command -ComputerName $TargetServer -ScriptBlock {
Restart-Service MSExchangeTransport
}
Run the DAG Maintenance Script
From Exchange Management Shell on the administrative shell host, run the Exchange-provided DAG maintenance script:
Set-Location $ExScripts
.\StartDagServerMaintenance.ps1 -ServerName $TargetServer -MoveComment "Rolling Exchange update" -PauseClusterNode
The script blocks database activation, pauses the target cluster node, moves any remaining active databases, and moves critical DAG functionality away from the target. The command can take time without producing continuous console output.
Redirect Pending Transport Messages
Redirect messages still pending on the target to the healthy server selected for the current cycle:
Redirect-Message -Server $TargetServer -Target $RedirectTargetFqdn -Confirm:$false
Set the Server-Wide Maintenance State
Place the target server into Exchange server-wide maintenance mode:
Set-ServerComponentState -Identity $TargetServer -Component ServerWideOffline -State Inactive -Requester Maintenance
Validate Maintenance Mode
Check effective Exchange component states:
Get-ServerComponentState -Identity $TargetServer |
Format-Table Component, State -Auto
ServerWideOffline must report Inactive. In the standard maintenance state, only Monitoring and RecoveryActionsEnabled should remain Active; investigate any other component that remains active before rebooting the server.
Confirm the database activation policy is blocked:
Get-MailboxServer -Identity $TargetServer |
Format-List Name, DatabaseCopyAutoActivationPolicy
Confirm that the target cluster node is paused:
Get-ClusterNode -Name $TargetServer |
Format-List Name, State
Confirm that no active databases remain on the target:
Get-MailboxDatabaseCopyStatus * |
Where-Object { $_.Status -eq "Mounted" -and $_.Name -like "*\$TargetServer" } |
Format-Table Name, Status, ActiveDatabaseCopy -Auto
Expected output:
<no output>
Confirm that the target transport queues have drained:
Get-Queue -Server $TargetServer |
Sort-Object MessageCount -Descending |
Format-Table Identity, Status, MessageCount, NextHopDomain -Auto
Confirm that the PAM is hosted by another DAG member:
Get-DatabaseAvailabilityGroup -Identity $DagName -Status |
Format-List PrimaryActiveManager, OperationalServers
!!! warning "Do Not Reboot Until Every Maintenance Gate Passes"
Do not install updates or reboot the target until it has no mounted databases, its database activation policy is Blocked, its cluster node is Paused, ServerWideOffline is Inactive, its transport queues are drained, and the PAM is hosted by another member.
Install Exchange and Windows Updates
Reboot Before Installing the Exchange Update
A clean reboot before Exchange Setup or an Exchange SU or HU reduces failures caused by pending file handles and services that do not stop cleanly. From the administrative shell host, reboot the target:
Restart-Computer -ComputerName $TargetServer -Force
Wait until the server is reachable, then revalidate that maintenance state persisted:
Get-ServerComponentState -Identity $TargetServer |
Where-Object { $_.Component -eq "ServerWideOffline" } |
Format-Table Server, Component, State -Auto
Get-ClusterNode -Name $TargetServer |
Format-List Name, State
Get-MailboxDatabaseCopyStatus * |
Where-Object { $_.Status -eq "Mounted" -and $_.Name -like "*\$TargetServer" }
Do not launch the update if ServerWideOffline is not Inactive, the cluster node is not Paused, or a database mounted on the target after reboot.
Install an Exchange CU or Build Upgrade
Mount the correct Exchange installation media on the target server. From an elevated Command Prompt on the target, run Setup by using the full media path so Windows does not invoke the installed Setup.exe from the Exchange binary directory.
=== "Diagnostic Data Off"
```cmd
D:\Setup.exe /Mode:Upgrade /IAcceptExchangeServerLicenseTerms_DiagnosticDataOFF
```
=== "Diagnostic Data On"
```cmd
D:\Setup.exe /Mode:Upgrade /IAcceptExchangeServerLicenseTerms_DiagnosticDataON
```
Replace D: with the actual mounted-media drive. Wait for Exchange Setup to complete successfully and review C:\ExchangeSetupLogs\ExchangeSetup.log before continuing.
!!! danger "A CU Upgrade Is Not Reversible by Uninstalling It" Do not treat a CU as an ordinary removable patch. If the CU fails, keep the server in maintenance mode and repair or recover that server by following Microsoft Exchange recovery guidance.
Install an Exchange SU or HU
Run the exact update package specified by the applicable release article from an elevated Command Prompt on the target server:
C:\ExchangeUpdates\<EXCHANGE_UPDATE_FILENAME>.exe
Replace <EXCHANGE_UPDATE_FILENAME> with the staged package name. Do not launch the installer from a non-elevated shell or by double-clicking it in File Explorer.
Do not stop IIS, WMI, Windows Event Log, or other Windows services preemptively unless the release article or a matching Microsoft troubleshooting procedure explicitly instructs you to do so. Wait for the installer to report successful completion before continuing.
Install Approved Windows Updates
Install the approved Windows Server updates while the target remains in maintenance mode. Apply prerequisites in the order required by the Exchange release notes, and continue rebooting as required until the server has no remaining approved updates or pending restart.
Perform the Final Maintenance Reboot
After Exchange and Windows updates have completed, reboot the target one final time:
Restart-Computer -ComputerName $TargetServer -Force
Wait for Windows, Active Directory connectivity, the Cluster service, and Exchange services to initialize before beginning return-to-service validation.
Return the Updated DAG Member to Service
Verify the Installed Exchange Build
From Exchange Management Shell on the administrative shell host, confirm the base build:
Get-ExchangeServer -Identity $TargetServer |
Format-Table Name, Edition, AdminDisplayVersion -Auto
Query the local ExSetup.exe file on the target to identify the installed SU or HU file version:
Invoke-Command -ComputerName $TargetServer -ScriptBlock {
$VersionInfo = (Get-Item (Join-Path $env:ExchangeInstallPath "bin\ExSetup.exe")).VersionInfo
[PSCustomObject]@{
Server = $env:COMPUTERNAME
ProductVersion = $VersionInfo.ProductVersion
FileVersion = $VersionInfo.FileVersion
}
}
Compare both values with the intended Microsoft build. An unchanged AdminDisplayVersion does not, by itself, prove that an SU or HU failed to install.
Validate Exchange Services
From the administrative shell host, run:
Test-ServiceHealth -Server $TargetServer
Inspect Exchange-related services on the target:
Invoke-Command -ComputerName $TargetServer -ScriptBlock {
Get-CimInstance Win32_Service |
Where-Object { $_.Name -like "MSExchange*" -or $_.Name -eq "FMS" } |
Select-Object Name, DisplayName, State, StartMode, ExitCode
} | Format-Table -Auto
Compare the result with C:\ExchangeUpdates\PreUpdate-Services.csv from that same server. Do not enable a service merely because it is stopped; protocol services and role-specific services may intentionally be manual or stopped.
!!! warning "Keep the Server in Maintenance Mode During Repair" If required Exchange services are disabled, fail to start, or report dependency errors, keep the target in maintenance mode. Repair the service state and complete the failed update before running the return-to-service commands.
Remove the Server-Wide Maintenance State
From Exchange Management Shell on the administrative shell host, restore the server-wide component state:
Set-ServerComponentState -Identity $TargetServer -Component ServerWideOffline -State Active -Requester Maintenance
Run the DAG Return-to-Service Script
Run the Exchange-provided DAG maintenance exit script:
Set-Location $ExScripts
.\StopDagServerMaintenance.ps1 -ServerName $TargetServer
The script resumes the cluster node, restores database activation policy, and resumes database copies hosted by the member.
Resume Hub Transport
Return Hub Transport to active state:
Set-ServerComponentState -Identity $TargetServer -Component HubTransport -State Active -Requester Maintenance
Restart transport on the target:
Invoke-Command -ComputerName $TargetServer -ScriptBlock {
Restart-Service MSExchangeTransport
}
Validate the Restored Member
Confirm Exchange component states:
Get-ServerComponentState -Identity $TargetServer |
Format-Table Component, State -Auto
Confirm database activation is unrestricted:
Get-MailboxServer -Identity $TargetServer |
Format-List Name, DatabaseCopyAutoActivationPolicy
Confirm the cluster node is up:
Get-ClusterNode -Name $TargetServer |
Format-List Name, State
Run service and replication health:
Test-ServiceHealth -Server $TargetServer
Test-ReplicationHealth -Identity $TargetServer
Inspect transport queues:
Get-Queue -Server $TargetServer |
Sort-Object MessageCount -Descending |
Format-Table Identity, Status, MessageCount, NextHopDomain -Auto
Do not restore the target to a load balancer or other external traffic source until required Exchange components are active, required services are running, the cluster node is up, replication health passes, and transport queues are processing normally.
Restore External Client Traffic
Return the target to its load balancer pool, monitoring platform, backup schedule, mail gateway, and any other system that was intentionally drained. Confirm the external health checks recognize the server as healthy before restoring active mailbox databases to it.
Restore the Intended Database Placement
Wait for Database Copies to Catch Up
Before moving an active database back to the updated member, confirm every copy is healthy and all queues have drained:
Get-MailboxDatabaseCopyStatus * |
Sort-Object Name |
Format-Table Name, Status, CopyQueueLength, ReplayQueueLength, ContentIndexState -Auto
Do not activate a copy on the updated member while it is Failed, Suspended, Disconnected, ServiceDown, or building a persistent queue.
!!! warning "Run Only the Current Cycle's Restore Block" The following restore blocks are cycle-specific. Do not move databases for a different cycle.
Cycle 1 Database Restore
Move BL-MBX-03 back to EXCH-SE-03:
Move-ActiveMailboxDatabase -Identity "BL-MBX-03" -ActivateOnServer "EXCH-SE-03" -Confirm:$false
Cycle 2 Database Restore
Move both databases back to EXCH-SE-02:
Move-ActiveMailboxDatabase -Identity "BL-MBX-02" -ActivateOnServer "EXCH-SE-02" -Confirm:$false
Move-ActiveMailboxDatabase -Identity "BL-ARC-02" -ActivateOnServer "EXCH-SE-02" -Confirm:$false
Cycle 3 Database Restore
Move both databases back to EXCH-SE-01:
Move-ActiveMailboxDatabase -Identity "BL-MBX-01" -ActivateOnServer "EXCH-SE-01" -Confirm:$false
Move-ActiveMailboxDatabase -Identity "BL-ARC-01" -ActivateOnServer "EXCH-SE-01" -Confirm:$false
Validate the Restored Placement
Confirm active database placement:
Get-MailboxDatabaseCopyStatus * |
Where-Object { $_.Status -eq "Mounted" } |
Sort-Object ActiveDatabaseCopy, Name |
Format-Table Name, Status, ActiveDatabaseCopy, CopyQueueLength, ReplayQueueLength -Auto
Expected Bunny Lab placement:
Name Status ActiveDatabaseCopy CopyQueueLength ReplayQueueLength
---- ------ ------------------ --------------- -----------------
BL-ARC-01\EXCH-SE-01 Mounted EXCH-SE-01 0 0
BL-MBX-01\EXCH-SE-01 Mounted EXCH-SE-01 0 0
BL-ARC-02\EXCH-SE-02 Mounted EXCH-SE-02 0 0
BL-MBX-02\EXCH-SE-02 Mounted EXCH-SE-02 0 0
BL-MBX-03\EXCH-SE-03 Mounted EXCH-SE-03 0 0
Temporary queues may appear immediately after activation. Wait until all copy and replay queues return to 0 before declaring the cycle complete.
Validate the Completed Cycle
Run replication health against every DAG member:
$DagMembers | ForEach-Object {
Test-ReplicationHealth -Identity $_
}
Check all database copies:
Get-MailboxDatabaseCopyStatus * |
Sort-Object Name |
Format-Table Name, Status, CopyQueueLength, ReplayQueueLength, ContentIndexState -Auto
Run the Exchange Health Checker against the updated member:
& "C:\Scripts\HealthChecker.ps1" -Server $TargetServer
The cycle is complete only when:
- The target reports the intended Exchange build
- Required Exchange services pass
Test-ServiceHealth - The target cluster node is
Up - Database activation policy is
Unrestricted - Required Exchange components are active
- Every applicable replication-health check returns
Passed - Every active database copy reports
Mounted - Every passive database copy reports
Healthy - Every copy and replay queue is
0 - Transport queues are processing normally
- The intended active database placement is restored
- External health checks and monitoring report the target as healthy
- The post-update Health Checker report contains no unresolved update-blocking finding
Do not begin the next cycle until all conditions are satisfied. Repeat the common workflow with the next cycle's variables and database move block.
Final DAG Validation
Confirm a Consistent Exchange Build
After all three cycles are complete, confirm the base build recorded for every DAG member:
$DagMembers | ForEach-Object {
Get-ExchangeServer -Identity $_
} | Format-Table Name, Edition, AdminDisplayVersion -Auto
Confirm the local ExSetup.exe file version on every member:
Invoke-Command -ComputerName $DagMembers -ScriptBlock {
$VersionInfo = (Get-Item (Join-Path $env:ExchangeInstallPath "bin\ExSetup.exe")).VersionInfo
[PSCustomObject]@{
Server = $env:COMPUTERNAME
ProductVersion = $VersionInfo.ProductVersion
FileVersion = $VersionInfo.FileVersion
}
} | Sort-Object Server | Format-Table -Auto
All three servers must report the same intended Exchange build unless Microsoft explicitly documents a temporary mixed-build state during the active maintenance window.
Validate Database Health and Placement
Run:
Get-MailboxDatabaseCopyStatus * |
Sort-Object Name |
Format-Table Name, Status, CopyQueueLength, ReplayQueueLength, ContentIndexState -Auto
Get-MailboxDatabaseCopyStatus * |
Where-Object { $_.Status -eq "Mounted" } |
Sort-Object ActiveDatabaseCopy, Name |
Format-Table Name, Status, ActiveDatabaseCopy, CopyQueueLength, ReplayQueueLength -Auto
Confirm every active copy is Mounted, every passive copy is Healthy, all queues are 0, and placement matches the Bunny Lab topology table.
Validate Replication, Services, Components, and Cluster State
Run:
$DagMembers | ForEach-Object {
Test-ServiceHealth -Server $_
Test-ReplicationHealth -Identity $_
}
Get-ClusterNode |
Format-Table Name, State, NodeWeight, DynamicWeight -Auto
$DagMembers | ForEach-Object {
Get-ServerComponentState -Identity $_ |
Where-Object { $_.State -ne "Active" } |
Select-Object Server, Component, State
}
Investigate every unexpected non-active component state. Protocol components that are deliberately disabled must match the documented Bunny Lab design rather than being assumed healthy merely because they were disabled before the update.
Validate Mail Flow
From Exchange Management Shell on each source server, test mail flow to another DAG member. The following examples cover all three members:
Test-Mailflow -Identity "EXCH-SE-01" -TargetMailboxServer "EXCH-SE-02"
Test-Mailflow -Identity "EXCH-SE-02" -TargetMailboxServer "EXCH-SE-03"
Test-Mailflow -Identity "EXCH-SE-03" -TargetMailboxServer "EXCH-SE-01"
Each test must return Success. Also validate inbound and outbound mail flow through the organization's real mail gateways and confirm the client-access namespaces used by the environment are healthy.
Run the Final Health Checker Audit
Run the current Exchange Health Checker against every DAG member:
$DagMembers | ForEach-Object {
& "C:\Scripts\HealthChecker.ps1" -Server $_
}
Archive the final reports with the change record.
Troubleshooting
The Installer Reports Files in Use or Cannot Stop Services
Do not select an installer option that ignores locked files, and do not terminate the Windows Event Log service. Exit the installer, confirm the server remains in maintenance mode, reboot it, and rerun the update from an elevated Command Prompt.
If the problem persists:
- Confirm the update package matches the installed CU
- Confirm the server was rebooted immediately before the installation attempt
- Review the selected update's known issues and antivirus exclusion guidance
- Preserve
C:\ExchangeSetupLogs - Use the Microsoft SetupAssist or Setup Log Reviewer tooling appropriate to the failure
- Follow the matching procedure in Fix Failed Exchange Server Updates
Do not reuse process IDs from an earlier attempt or another server. Process IDs are transient, and terminating an unidentified Windows or Exchange process can leave the installation in a worse state.
Exchange Services Are Disabled After the Update
Compare the current service configuration with C:\ExchangeUpdates\PreUpdate-Services.csv from the same server:
Invoke-Command -ComputerName $TargetServer -ScriptBlock {
$Baseline = Import-Csv "C:\ExchangeUpdates\PreUpdate-Services.csv"
$Current = Get-CimInstance Win32_Service |
Where-Object { $_.Name -like "MSExchange*" -or $_.Name -eq "FMS" }
foreach ($Before in $Baseline) {
$After = $Current | Where-Object { $_.Name -eq $Before.Name }
if ($After -and ($After.State -ne $Before.State -or $After.StartMode -ne $Before.StartMode)) {
[PSCustomObject]@{
Name = $Before.Name
BeforeState = $Before.State
AfterState = $After.State
BeforeStartMode = $Before.StartMode
AfterStartMode = $After.StartMode
}
}
}
} | Format-Table -Auto
Restore only a service whose required startup mode is confirmed by the server's baseline, current Exchange role, and Microsoft guidance:
Set-Service -Name <SERVICE_NAME> -StartupType Automatic
Start-Service -Name <SERVICE_NAME>
If the service fails with dependency error 1068, inspect its required services:
Get-Service -Name <SERVICE_NAME> -RequiredServices |
Format-Table Name, DisplayName, Status, StartType -Auto
Correct the failed dependency before retrying the dependent service. Do not automatically enable IMAP, POP, EdgeSync, or another optional service that was intentionally disabled before the update.
An Exchange Component Remains Inactive
Inspect the effective and requester-specific component states:
Get-ServerComponentState -Identity $TargetServer |
Format-Table Component, State -Auto
Get-ServerComponentState -Identity $TargetServer -Component <COMPONENT_NAME> |
Format-List Component, State, LocalStates, RemoteStates
Correct the requester that actually holds the component inactive. Do not repeatedly issue Requester Maintenance commands when the inactive state belongs to Functional, HealthAPI, or another requester.
If a failed Exchange update left ServerWideOffline, Monitoring, or RecoveryActionsEnabled inactive under the Functional requester, and the failed installation has already been repaired, restore only the affected states:
Set-ServerComponentState -Identity $TargetServer -Component ServerWideOffline -State Active -Requester Functional
Set-ServerComponentState -Identity $TargetServer -Component Monitoring -State Active -Requester Functional
Set-ServerComponentState -Identity $TargetServer -Component RecoveryActionsEnabled -State Active -Requester Functional
Rerun Test-ServiceHealth and Test-ReplicationHealth after the correction.
High Availability Remains Offline After Maintenance Ends
If a database move fails with an error stating that the HighAvailability component is offline, inspect its requester-specific state:
Get-ServerComponentState -Identity $TargetServer -Component HighAvailability |
Format-List Component, State, LocalStates, RemoteStates
Confirm StopDagServerMaintenance.ps1 completed successfully, the cluster node is Up, database activation is Unrestricted, and MSExchangeRepl is running. If every requester reports Active but Active Manager state remains stale, restart the replication service on the target:
Invoke-Command -ComputerName $TargetServer -ScriptBlock {
Restart-Service MSExchangeRepl
Get-Service MSExchangeRepl
}
Rerun replication health and retry the database move only after every applicable check passes.
Exchange Setup Reports Success but the Build Does Not Change
Confirm Setup was launched from the mounted media by using an absolute path such as D:\Setup.exe or, from PowerShell in the media root, .\Setup.exe. Running only Setup.exe can invoke the installed copy under the Exchange binary path instead of the intended installation media.
Review C:\ExchangeSetupLogs\ExchangeSetup.log, correct the launch path, and rerun Setup while the server remains in maintenance mode.
Outlook on the Web or the Exchange Admin Center Fails After the Update
First confirm the Exchange update completed successfully and required services are running. Review the matching symptom in Microsoft's failed-update guidance rather than running a generic post-update command sequence.
For an IIS state that specifically requires a service restart, run from an elevated PowerShell session on the affected server:
Restart-Service -Name WAS, W3SVC
Do not make UpdateCas.ps1, UpdateConfigFiles.ps1, IISADMIN, ADSI changes, or forced process termination part of the normal update workflow. Use a repair command only when an authoritative troubleshooting procedure identifies the same failure condition and explains the required validation.
Recovery and Stop Conditions
If a target server cannot be returned to service:
- Keep
ServerWideOfflineinactive and keep the cluster node paused - Keep database activation blocked on the failed member
- Leave active databases on healthy DAG members
- Do not begin maintenance on another member
- Preserve Exchange Setup logs, Windows event logs, Health Checker reports, and the pre-update service baseline
- Repair the update or perform the documented Exchange server recovery procedure
- Revalidate quorum, database redundancy, transport, client access, and backup status before resuming the rolling upgrade
The rolling update is complete only after all three members report the intended Exchange build, every required health check passes, the documented database placement is restored, and Bunny Lab mail flow and client access are validated end to end.