Mello's IT Musings

Over the summer my company moved our email ingress and egress from on premises Cisco IronPort Gateways to Exchange Online Protection. While this moved us to a modern and cloud-based mail hygiene solution, we did lose some of custom mail routing schemes we implemented when we had full control of our mail flow. One such custom feature was the ability to selectively block SMTP addresses on any given mail object in our on-premises Exchange environment from external mail without having to block all external mail to the object.

This was accomplished by adding a second proxy address for each address we wanted to block from receiving external email at the Cisco Iron port level with ‘HIDE’. Such that when our Cisco IronPort’s did an LDAP lookup for an internal proxy address, if the same address was returned twice, (an SMTP and HIDE entry) then drop the message since that address should be internal only. In this example of the proxyaddresses on a given mail object:

hide:John.Mello@contoso.com
smtp:mello@mountainview.usa.com
smtp:mello@contosocloud.mail.onmicrosoft.com
smtp:mello@tailspintoys.com
SMTP:John.Mello@contoso.com

john.mello@contso.com would be blocked at the Cisco IronPort level since an LDAP search would return the following two addresses

hide:John.Mello@contoso.com
SMTP:John.Mello@contoso.com

The remaining address would be allowed through

smtp:mello@mountainview.usa.com
smtp:mello@contosocloud.mail.onmicrosoft.com
smtp:mello@tailspintoys.com

Once we moved to Exchange Online Protection, and the HIDE proxy addresses no longer functioned, we had to clean-up all the entries we had in our environment. To do so we created an Exchange Management Shell PowerShell script to do the following

1. Run the script from the Exchange management shell
   1. The script assume you are getting non-deserialized objects. If you are not using the Exchange Management shell, then you will need to update it to take into account that you are getting deserialized objects
2. Ensure you are viewing the entire forest
   1. This was needed for us since we have Exchange on premises in a resource domain and thus our public folders are in this domain while everything else is in a separate domain
3. Grab all recipients
4. Check if there are any HIDE proxyaddresses on the mail object. If not, then skip if so then
5. Determine the Recipient detail type so we know which Set-* command to use to remove the addresses
   1. We could have just used Set-ADObject and instead to update the proxyaddresses property, but I wanted to keep this strictly within the commands available to Exchange Administrator
   1. In our environment we had a lot of contacts that required updating before we could edit the email addresses. So while we tried to force an update whenever we came across one, it still required manual intervention at the console to approve the change. We only had a few dozen in this state so after coming across the first one we filtered them out in the big run and then addressed them and manually accepted the update. In this case Set-ADObject would have allowed us to automate it, but wouldn’t fix the underlying need to update them
6. Save the results to an array for later referencing in case we need to roll back

Outside the need to update our mail contacts here are some issues we ran into while running the script

1. We encountered a number of distribution groups that gave the following error when we tried to remove the addresses : ‘Members can’t add themselves to security groups. Please set the group to Closed or Owner Approval for requests to join.’
   • In some research this was due to the fact that the distribution group was flipped to a Security using ADUC (Active directory users and computers) or vice verse, as opposed to done in Exchange
   • This put the distribution group on an inconsistent state as far as Exchange was concerned and we needed to run the following parameters when running the set-distributiongroup with the following parameters: -MemberJoinRestriction closed -MemberDepartRestriction closed
2. We had exactly 1 object with a plus sign (‘+’) in it’s name, which the various set-* cmdlets do not like when using a distinguished name with a plus sign. In this case switching the script to leverage the Alias instead for each set-* cmdlet fixed that issue

Script

#This will ensure mail objects in other forests are included
Set-ADServerSettings -ViewEntireForest:$TRUE
$AllObjects = Get-Recipient -ResultSize Unlimited
[System.Collections.ArrayList]$Changelist = @()

ForEach ($MailObject in $AllObjects) {
    $TableEntry = [PSCustomObject]@{
        Name        = $MailObject.Name
        DistinguishedName = $MailObject.DistinguishedName
        PreviousEmailAddresses = $MailObject.EmailAddresses
        RecipientTypeDetails = $MailObject.RecipientTypeDetails
        RemovedAddress      = ""
        Result = "UNPROCESSED"
    }#$TableEntry = [PSCustomObject]@
    #If you were looking for specific doamin you could use a regex like such
    #[String[]]$ProxyAddressesToRemove = ($MailObject.EmailAddresses | Where-object SMTPAddress -match '^.*@DOMAIN\.com$').ProxyAddressString
    [String[]]$ProxyAddressesToRemove = ($MailObject.EmailAddresses | Where-object prefix -eq 'hide').ProxyAddressString
    If ($ProxyAddressesToRemove.count -ne 0) {
        $TableEntry.RemovedAddress = $ProxyAddressesToRemove
        Try{
            switch -regex ($MailObject.RecipientTypeDetails) {
                '^(MailUniversalSecurityGroup|MailUniversalDistributionGroup|RoomList)$' { $SetCommand = "Set-DistributionGroup"; break }
                '^(UserMailbox|TeamMailbox|SharedMailbox|RoomMailbox|EquipmentMailbox|DiscoveryMailbox)$' { $SetCommand = "Set-Mailbox"; break }
                'MailUser' { $SetCommand = "Set-MailUser"; break }
                '^(RemoteUserMailbox|RemoteRoomMailbox|RemoteEquipmentMailbox|RemoteSharedMailbox)$' { $SetCommand = "Set-RemoteMailbox"; break }
                'MailContact' { $SetCommand = "Set-MailContact"; break }
                'PublicFolder' { $SetCommand = "Set-MailPublicFolder"; break }
                'DynamicDistributionGroup' { $SetCommand = "Set-DynamicDistributionGroup"; break }
                Default {Throw "Unhandled RecipientTypeDetails: $($MailObject.RecipientTypeDetails)"}
            }#switch -regex ($MailObject.RecipientTypeDetails) {
            #We noticed that a good majority of the mail contacts in our enviroment were of 'ExchangeVersion' '0.0 (6.5.6500.0)'
            #Any attempt to update them resulted in a prompt to update, so in all instances we force an upgrade to our current version, which is '0.20 (15.0.0.0)'
            if ($SetCommand -ne 'Set-MailContact"' ) {
                .$SetCommand $MailObject.DistinguishedName -EmailAddresses @{Remove=$ProxyAddressesToRemove} -erroraction stop
            }#if ($SetCommand -ne 'Set-MailContact"' ) {
            Else {
                .$SetCommand $MailObject.DistinguishedName -EmailAddresses @{Remove=$ProxyAddressesToRemove} -ForceUpgrade:$true -Confirm:$false -erroraction stop
            }#Else
            $TableEntry.Result = 'HIDES REMOVED'
        }#Try
        Catch {
            $TableEntry.Result = $_
        }#catch
        $Changelist.add($TableEntry) | Out-Null
    }#If ($ProxyAddressesToRemove.count -ne 0) {
    Else{
        $TableEntry.Result = 'NO ACTION NEEDED'
        $Changelist.add($TableEntry) | Out-Null
    }#Else
    [System.GC]::Collect()
}#ForEach ($MailObject in $AllObjects) {

Recently our helpdesk received a report of a user leveraging Outlook 2013 (x86) in a VM on a Mac to access a local mailbox (i.e. Not O365) crashing with an out of memory error. Before they reached out to my team (which are the stewards of Exchange 2016 and O365) they tired the following

• Recreated the OST file
• Remove all attached PST files
• Created a new VM with Outlook 2013 (x64)
• Then Outlook 2016
• Doubled the memory of the VM in question (8 to 16GB) as well as the virtual CPUS (2 to 4)
• Applied the fixes laid out in Outlook performance issues when there are too many items or folders in a cached mode .ost or .pst file

The issue only went away once we removed all attached shared mailboxes (2 total). When adding them back, we noticed one of the shared mailboxes generated a lot of entries in the ‘Sync Issues’ folder. When we investigated the shared mailbox in question, it had 4207 user created folders. And while the user had access to this mailbox for years, it wasn’t until the gained access to another shared mailbox that this issue starting to occur. So we surmised that this mailbox in particular was the cause and was ripe for clean-up due to the amount of folders it contained. Using the get-mailboxfolderstatistics cmdlet we generated a CSV of all the user created folders along with some metrics around them for the owners of the shared mailbox to review for deletion

get-mailboxfolderstatistics PROBLEM_MAILBOX | Where-Object FolderType -eq "User Created" | Select Name, FolderPath, ItemsInFolder, ItemsInFolderAndSubfolders,@{N='FolderSize(MB)';E={$_.FolderSize.tomb()}}, @{N="FolderAndSubfolderSize(MB)";e={$_.FolderAndSubfolderSize.toMB()}}, NewestItemReceivedDate, OldestItemReceivedDate | Export-CsV -Path C:\Temp\FolderReview.csv

The owners then edited that list down to 3551 folders to delete. At that point we leverage EWS (Exchange Web Services) in PowerShell to delete the folders in question . This was a quick and dirty process as I didn’t have much time to create something more elegant. One of wish list items I had was figuring out how to properly weed out only top level folder paths to delete, but after spending 30 minutes trying to figure that out I decided it wasn’t worth the effort for this one off task. If I did delete a top level folder and the list had multiple child folders then our deletion attempt would just error out. We also used an account setup for application impersonation for this effort. Bare bones script below:

Make an Exchange EMS Connection
$ExchSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri YOUR_EXCHANGE_PS_URL -Authentication Kerberos -Name Exchange -ErrorAction Stop
Import-PSSession $ExchSession -allowclobber -ErrorAction Stop -DisableNameChecking -WarningAction SilentlyContinue *> $NULL

#Variables for EWS connection
$ExchangeVersion = "Exchange2013"
[String]$EWSManagedApiPath = "C:\Program Files\Microsoft\Exchange\Web Services\2.2\Microsoft.Exchange.WebServices.dll"

#Impersonation Account
$ImpersonationAccountNAme = "YOUR IMPERSONATION ACCOUNT"
$ImpersanationAccountPassword = "YOUR IMPERSONATION ACCOUNT"

#Mailbox clean up info
$Mailbox = 'PROBLEM_MAILBOX@YOURDOMAIN.COM'
$TaggedFolders = Import-Csv -Path 'YOUR_CSV.CSV'
$AllFolders = get-mailboxfolderstatistics $Mailbox |
    Where-Object FolderType -eq "User Created"
$FoldersToRemove = $AllFolders |
    Where-Object FolderPath -in $TaggedFolders.FolderPath

# Load the needed dlls
Add-Type -Path $EWSManagedApiPath
Add-Type -AssemblyName System.Web

# Set EWS Service
$ExchangeVersion = [Microsoft.Exchange.WebServices.Data.ExchangeVersion]::$ExchangeVersion
$service = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService($ExchangeVersion)  

# Setup impersonation
$Service.Credentials = New-Object Microsoft.Exchange.WebServices.Data.WebCredentials($ImpersonationUserName, $ImpersonationPassword) -ErrorAction Stop
$Service.ImpersonatedUserId = New-Object -TypeName Microsoft.Exchange.WebServices.Data.ImpersonatedUserId -ArgumentList "SmtpAddress", $Mailbox

$service.AutodiscoverUrl($Mailbox, { $true })  

#Function to convert the folderID from the Exchange Management shell to one we can us in EWS
function ConvertId {    
    param (
        $OwaId = "$( throw 'OWAId is a mandatory Parameter' )"
          )
    process {
        $aiItem = New-Object Microsoft.Exchange.WebServices.Data.AlternateId      
        $aiItem.Mailbox = $Mailbox     
        $aiItem.UniqueId = $OwaId   
        $aiItem.Format = [Microsoft.Exchange.WebServices.Data.IdFormat]::OwaId 
        $convertedId = $service.ConvertId($aiItem, [Microsoft.Exchange.WebServices.Data.IdFormat]::EwsId) 
        return $convertedId.UniqueId
    }
}

# Run through the folders ot delete
$FoldersToRemove | 
    ForEach-Object {
        Write-Output "Deleting Folder $($_.FolderPath)"
        try {
            $urlEncodedId = [System.Web.HttpUtility]::UrlEncode($_.FolderId.ToString())
            $folderid = New-Object Microsoft.Exchange.WebServices.Data.FolderId((Convertid $urlEncodedId))  
            $ewsFolder = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service, $folderid)        

            $ewsFolder.Delete([Microsoft.Exchange.WebServices.Data.DeleteMode]::HardDelete)
            "Folder Deleted"
        }
        catch {
            Write-Warning "Problems deleting $($_.FolderPath)"
        }
    }