Hybrid free busy not working for just one user

A quick break down of the environment where this issue occurred

  • Office 365 in Hybrid with Exchange 2016
  • Local 2019 AD Domain with Azure AD Connect syncing to Office 365 every 30 minutes, with no AD write back
  • 350+ could mailboxes with 2200+ still on prem

I recently had an issue at work with my Exchange online mailbox not being able to pull free busy information from any of my company’s onprem mailboxes. My mailbox was migrated a little over 2 years ago and I’m not exactly sure when it stopped, but I only noticed as my company started to enact it’s return to work plan early in the summer of 2021 and I couldn’t see the free busy information of any of our conference rooms. All of which we haven’t migrated to Exchange Online yet. At first, I checked in with my teammates and discovered the issue was just with my account. From there I tried using the Microsoft Remote Connectivity Analyzer to troubleshoot, but got a generic error when attempting the various Exchange tests. I then checked for any odd attributes set on my account, but that turned up nothing obvious. I then tried doing a New-MoveRequest PowerShell cmdlet, which sometimes helps with fixing odd issues in Exchange Online. The hope in doing so is that the post move validations might correct whatever was causing the free buys look up issue. So, after exhausting all the options I could think of I opened a ticket with Microsoft Support. After going through the normal troubleshooting steps, they suggested that I clear out the following AD attributes and wait for a sync from our on premises AD to Azure AD

  • msExchRecipientDisplayType
  • msExchRecipientTypeDetails
  • msExchMailboxGuid

Before clearing these attributes, I made a note of the values and then waited for 2 Azure AD connect sync session. Unfortunately the issue persisted and my companies various other integrated SAAS platforms were throwing errors since those attributed weren’t populated for my account. At first, I tired running the Update-Recipient PowerShell cmdlet in both Exchange Online and Exchange on premises to see if it would re-populate those attributes. Sadly it did not so I added back the values I saved for each. About 24 hours later I checked if the issue was still present, and it was fixed! MS support wasn’t exactly sure why the issue was resolved, but the next time I run into an odd Exchange issue I’m going to try clearing those attributes first.

Posted in Uncategorized | Tagged as: , , | Leave a comment

Adventures in PowerShell dot sourcing

As my company has started our journey to Office 365, we have had to go back and update a lot of our daily checks, scripts, and automations we put in place for user accounts. As a specific example we have a disable user script that performs various actions on a user’s mailbox depending on the wishes of the manager of said user. Depending on where the mailbox is located (on premises or in the cloud), we need to use either the set-mailbox or set-remotemailbox commands. In order support both scenarios in our existing scripts we leveraged dot sourcing to store the needed commands per mailbox type to the same variables used throughout the script. So that when we need to perform those mailbox actions, we don’t need to add a separate if or case statement for each action for the location of the mailbox. We instead leverage the variable holding the command via dot sourcing. The snippet below shows that we use the msExchRemoteRecipientType attribute of the user to determine the location of the user mailbox, and then save the appropriate commands to the matching variables.

        #Checking which Set-*mailbox command to use depending on the user is on prem or in the cloud
        if ($FoundAccount.msExchRemoteRecipientType -ne $NULL) {
            Write-Verbose "$UserAccount : Has a remote mailbox, using *-remotemailbox for all Exchange tasks"
            $SetMBXCommand = "Set-RemoteMailbox"
            $GetMBXCommand = "Get-RemoteMailbox"
            $DisableMBXCommand = "Disable-RemoteMailbox"
        }#if ($FoundAccount.msExchRemoteRecipientType -ne $NULL) {
        Else {
            Write-Verbose "$UserAccount : Has a local mailbox, using *-mailbox for all Exchange tasks"
            $SetMBXCommand = "Set-Mailbox"
            $GetMBXCommand = "Get-Mailbox"
            $DisableMBXCommand = "Disable-Mailbox"
        }#Else

When performing an action on the user mailbox we do source the variable for the command, which will expand the variable and treat it as the command store in the variable.

.$DisableMBXCommand $UserAccount -ErrorAction Stop -Confirm:$FALSE
Posted in Exchange, Office 365, PowerShell | Leave a comment

Issues installing Windows Admin Center on Windows Server 2016

At my Job we have been playing around with Windows Admin Center, and while it’s pretty neat it has been a pain to install. We ran into multiple issues trying to get it to properly install in Gateway Mode on our Windows Servers 2016 management host that weren’t called out in the troubleshooting steps. Here are the steps we had to go through in order to get it to actually install

  1. You can’t install it over Remote Desktop
  2. You must have the following services enabled and running
    1. Windows Update
    2. Windows firewall

The last two were interesting since they aren’t really called out during the install. They are sorta called out in in the install log, but it’s not obvious that it’s directly related to the services not being enabled and running. Hopefully this helps out anyone else who has issues installing it on Windows server 2016.

Posted in Windows Admin Center | Leave a comment

Hunting down personal Microsoft accounts using an corporate Email address in O365

Recently my company started endeavored on setting up an O365 (Office 365) tenant in full Hybrid mode with our on-premise Exchange infrastructure. One of the issues we ran into as we started migrating pilot users was that some of our various Exchange integrations didn’t handle the authentication when the user in question had an existing personal Microsoft account associated with their corporate email address.  As an example, users in this state would se something like this when they logged into the O365 portal

My company leverages Blackberry UEM as our mobile email solution, and for any user migrated to Office 365 that was in this state could not user mobile email access via BlackBerry work due to the fact that it would receive this prompt and not be able to process it. We contacted MS and ask if we could be provided a list of all personal Microsoft accounts that were using a domain owned by my company, but for obvious legal reasons they could not. So the only option left to use was to test every email address in our environment against the office 365 login page (https://login.microsoftonline.com). After some testing we were able to properly inspect the login process and create a PowerShell script that could present an email address to this page and return not only if it was associated with a personal account but the federated gateway it was associated with. Here is a link to the Github repository for the PowerShell function in question. We tested it against about 8000 email addresses with no issues, but your mileage may vary.

https://github.com/Iczer1/O365-Scripts/blob/master/Check-PersonalAccount.ps1

Posted in Office 365 | Leave a comment

Properly scoping get-inboxrule for a custom RBAC role group

After creating a custom RBAC role in Exchange for our help desk so that they could handle simpler end user requests like mailbox size, conference room permissions, etc. after deploying it we got reports that the get-inboxrule command was not working as expected and was throwing a “You may need elevated permissions. isn’t within your current write scopes. Can’t perform save operation.” Error whenever a member of the help desk ran that command against another user. At first we weren’t exactly sure what the issue was because roles like “View-Only recipients” (which was one of the roles that was used to create the custom role group) seemed to be in the right scope for other commands. After some searching we came across a blog post by Pawet Jarosz explaining the problem. So we needed to recreate the role group with a new management role for get-inboxrule that was in the proper scope. Once we did that our help desk was now able to properly view other user’s inbox rules. Below is the custom role group we created

Add all the Get and test commands from “View-Only Configuration”
New-ManagementRole –Name “Helpdesk View-Only Configuration” –Parent “View-Only Configuration”
Get-ManagementRoleEntry “Helpdesk View-Only Configuration*” |
Where Name -notmatch ‘(Get)|(Test)|(Write-AdminAuditLog)|(Start-AuditAssistant)(Get-InboxRule)’ |
Remove-ManagementRoleEntry -Confirm:$False
Add all the Get and test commands from “View-Only recipients”
New-ManagementRole –Name “Helpdesk View-Only recipients” –Parent “View-Only recipients”
Get-ManagementRoleEntry ” Helpdesk View-Only recipients*” |
Where Name -notmatch ‘(Get)|(Test)|(Write-AdminAuditLog)|(Start-AuditAssistant)(Get-InboxRule)’ |
Remove-ManagementRoleEntry -Confirm:$False
Add all the Get and test commands from “Monitoring”
New-ManagementRole –Name “Helpdesk Monitoring” –Parent “Monitoring”
Get-ManagementRoleEntry ” Helpdesk Monitoring*” |
Where Name -notmatch ‘(Get)|(Test)|(Write-AdminAuditLog)|(Start-AuditAssistant)(Get-InboxRule)’ |
Remove-ManagementRoleEntry -Confirm:$False
Get-Inbox rules has scope issues so we need to create a separate role for that
New-ManagementRole –Name “Helpdesk View Inbox Rules” –Parent ‘Mail Recipients’
Get-ManagementRoleEntry ” Helpdesk View Inbox Rules*” |
Where Name -notmatch ‘Get-InboxRule’ |
Remove-ManagementRoleEntry -Confirm:$False
$GroupSplat = @{
Name = ‘Helpdesk Exchange Tasks’
Roles = @(” Helpdesk View-Only Configuration”, ” Helpdesk View-Only recipients”, ” Helpdesk Monitoring”, ” Helpdesk View Inbox Rules”)
ManagedBy = ‘GROUP OWNER ACCOUNT
Description = ” specific collection of cmdlets needed for view only Exchange management”
members = @(“Global Helpdesk”)
}
New-RoleGroup @GroupSplat

Posted in Exchange | Leave a comment

Checking Exchange IIS logs for users who downloaded files via OWA

At my work we use a 3rd party application that rides ontop of OWA in Exchange in order to block attachment downloads across certain IP ranges (namely all external access). We recently had an issue with it silently failing and we needed to figure out who might have downloaded attachments via OWA while the application was down. In looking over the IIS logs we determined that we could accurately figure this out by looking for

  1. Any cs-uri-stem like ‘/owa*Get*Attachment*’. In particualr these were the ones we want to capture
    1. /owa/service.svc/s/GetFileAttachment
    2. /owa/service.svc/s/GetAllAttachmentsAsZip
    3. We did not want /owa/service.svc/s/GetAttachmentThumbnail
  2. Any cs-uri-query like ‘*&ClientRequestId=*&encoding=*’
  3. In our parsing of the logs the same cs-uri-stem was used for uploading and downloading files, but downloads had that near the of the query while uploads had ‘*&ClientId=*’

With this info we used the following log parser query to look for any successful file downloads from the IP ranges we considered external as well as any non internal IP range. We also included some other items to weed out any actions a health mailbox performed

SELECT Distinct
TO_LOCALTIME(TO_TIMESTAMP(date, time)) AS [DateTime],
s-ip as [Server-IP],
REVERSEDNS(s-ip) as [ExchangeServer],
cs-uri-stem as [OWAUrl],
cs-username as [UserName],
c-ip as [Client-IP],
REVERSEDNS(c-ip) as [RemoteHostName],
cs(User-Agent) as [Browser]
FROM '[LOGFILEPATH]'
Where cs-uri-stem Like '/owa%Get%Attachment%'
and cs-uri-stem <> '/owa/service.svc/s/GetAttachmentThumbnail'
and cs-uri-query like '%&ClientRequestId=%&encoding=%'
and sc-status < 400 and cs-username NOT LIKE 'HealthMailbox%' and cs-method = 'GET' and (c-ip like '192.172.%' or c-ip like '189.18.5.%') and (c-ip not like '10.%' or c-ip not like '172.20.2%') and cs(User-Agent) <> 'AMProbe/Local/ClientAccess'
Posted in Exchange, LogParser | Leave a comment

A single server can serve as a witness for multiple DAGs. However, each DAG requires its own witness directory.

This recently trip us up when bringing up new 2016 servers in our existing 2013 environment. We and used the same witness directory for both DAGS (e.g. \\MGMT500\C$\DAGFileShareWitnesses) under the assumption that the witness directories would be different.  While it does create a folder with a different GUID for each DAG. it treats the full path as owned by either DAG. So if any issues occur and the DAG decides to remove the fileshare witness directory it will remove the whole path. So if you are going to use the same server for your fileshare witness make sure to specify a different path for each.

Posted in Uncategorized | Tagged as: , | Leave a comment

Sending an email with X-Headers in EWS via powershell

I recently needed to test sending an email via EWS with a bunch of custom X-headers. While I have worked with EWS in PowerShell before I couldn’t find any examples of how to do so but found plenty in Java and .Net. So, after stumbling through trying to recreate what I found in PowerShell I came with up with the following and inserted into this script for sending EWS emails.

In the code below, I’m

  1. Creating a new property set so I have access to edit all the parts of the new email. I think all the needed property sets must be loaded at the same time. That is you can’t load one set and add another without it overwriting the first property set
  2. I then save the email since I learned in my testing you can’t add a property to the email without an instance key associated with it, which it doesn’t get until it’s saved.
  3. Create a internet header object and it’s name
  4. Load the property set
  5. Insert the X-Header and its value in the headers of the email
  6. I then update the saved email (though I don’t think this necessary)

 

#Tested by using this : https://gallery.technet.microsoft.com/office/Send-e-mail-from-67a714aa

#TO view more properties

# https://msdn.microsoft.com/en-us/library/microsoft.exchange.webservices.data.emailmessage_members(v=exchg.80).aspx

$propertyset = New-Object Microsoft.Exchange.WebServices.Data.PropertySet (

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::InternetMessageHeaders,

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::ExtendedProperties,

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::MimeContent,

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::Attachments,

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::ToRecipients,

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::Body,

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::BccRecipients,

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::CcRecipients,

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::From,

    [Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::ReplyTo,

    [Microsoft.Exchange.WebServices.Data.ItemSchema]::InternetMessageHeaders,

    [Microsoft.Exchange.WebServices.Data.ItemSchema]::ExtendedProperties

)

#You need to save to load more properties (this creates a draft)

$message.Save()

#Create New Xheader object

$XHeader = New-object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition([Microsoft.Exchange.WebServices.Data.DefaultExtendedPropertySet]::InternetHeaders,'X-CustomHeader',[Microsoft.Exchange.WebServices.Data.MapiPropertyType]::String)

$message.load($propertyset)

#Set the property

$message.SetExtendedProperty($XHeader,"Sent Via EWS")

#Update the draft version

#https://docs.microsoft.com/en-us/dotnet/api/microsoft.exchange.webservices.data.conflictresolutionmode?view=exchange-ews-api

$message.Update('AlwaysOverwrite')
Posted in EWS, PowerShell | Leave a comment

Undocumented Change in 2016 MessageID header behavior

At my company, we recently migrated from 2013 to 2016, but had to keep a 2013 footprint around for support of older Blackberry 5 devices. In our environment, we have a SendMail farm that is an internal open relay as well as being responsible for the send and receiving of external mail. We used various config files on this SendMail farm to distinguish if an email came from Exchange and not one of the various internal application servers that relays off this farm. One of those methods was using the MessageID header of the email. In 2013 that message header would look something like GUID@ServerFQDN, as explained in this technet article. The same article also states this is true for 2016, but this does not appear to the case in our mixed environment.  While 2013 still follows this pattern, 2016 instead uses the primary SMTP address of the sending object for its domain on the MessageID header, so GUID@Primary_SMTP_Address_of_Sending_Object. Whenever the primary SMTP address of an Exchange object changes, and that change is reflected in the Global Address Book, any message sent by that object will be in that format. I’m guessing that this was a change meant to help better track messages in Office 365, but I’m curious as to why this behavior change hasn’t been called out in any of the Cumulative updates for 2016 or the official documentation.

Posted in Exchange | Leave a comment

How to remove a user from a security group in a different domain in PowerShell

Recently I ran into an issue at my company removing a user in our primary domain from a group in our root domain using the AD cmdlets in PowerShell. All my company’s user, computer, and group objects are in our primary domain and our root domain is more of a resource forest. The group in question was an Exchange RBAC role in the resource forest. So, when I first attempted the removal as such

Remove-ADGroupMember -Identity “HelpDesk Exchange Tasks” -members doej

I got the following error

Remove-ADGroupMember : Cannot find an object with the Identity: ‘HelpDesk Exchange Tasks’ under: ‘DC=corp,DC=contoso,DC=com’.

At first it seemed obvious that the solution was to use a domain controller in our resource domain to perform the task. So, I tried referencing a DC in the resource domain

Remove-ADGroupMember -Identity “HelpDesk Exchange Tasks” -members doej -server FRDC500.root.contoso.com

But got the following error

Remove-ADGroupMember : Cannot find an object with the Identity: ‘CN=doel,OU=US,OU=CORP,DC=corp,DC=contoso,DC=com’’ under: ‘DC=root,DC=contoso,DC=com’.

At that point I didn’t know how to proceed so I did some searching on the internet and came across an MS blog entry entitled Adding/removing members from another forest or domain to groups in Active Directory

Basically, you need to

  1. Choose against what domain server you want to run the command against.
  2. Get the default returned property set of the object in the other domain, referencing a domain controller in that domain if needed
  3. Run the command referencing just the name/samaccountname/CN/DN of the object that will be referenced by the server in the command and for the object in the other domain use the full object
    1. Referencing just the name/samaccountname/CN/DN OR even just selecting those properties on the object will not work. It needs to be the full default object as returned by the get-AD* command you are using to get the object

So, in my example I pulled the PDCEmulator from the resource domain (where the group was) and the default domain (where the user object was)

$DC_In_Root = (Get-ADDomain root.contso.com).PDCEmulator
$DC_In_Default = (Get-ADDomain corp.contso.com).PDCEmulator

Then I saved the default returned property set of the user object in the current domain (I didn’t need to reference a DC in this domain since it was my default working domain, but it’s done here for clarity’s sake)

$Default_Domain_User = Get-Aduser doej -server $DC_In_Default

In my example, I’m going to use the DC in my root domain to remove the user from the group. So, I only need to reference the group in this domain by name/samaccountname/CN/DN BUT the user needs to be referenced as an object with it’s complete default returned property set. The opposite can be done if needed

Remove-ADGroupMember -Identity “HelpDesk Exchange Tasks” -members $Default_Domain_User -server $DC_In_Root

I’m not sure why it needs to be the complete default property set. In my limited testing, removing just one of the properties caused it to fail.

Posted in Exchange, PowerShell | Leave a comment
  • Archives

  • December 2021
    M T W T F S S
     12345
    6789101112
    13141516171819
    20212223242526
    2728293031  
  • Page list