Mello's IT Musings

At my company, we recently migrated from 2013 to 2016, but had to keep a 2013 footprint around for support of older Blackberry 5 devices. In our environment, we have a SendMail farm that is an internal open relay as well as being responsible for the send and receiving of external mail. We used various config files on this SendMail farm to distinguish if an email came from Exchange and not one of the various internal application servers that relays off this farm. One of those methods was using the MessageID header of the email. In 2013 that message header would look something like GUID@ServerFQDN, as explained in this technet article. The same article also states this is true for 2016, but this does not appear to the case in our mixed environment.  While 2013 still follows this pattern, 2016 instead uses the primary SMTP address of the sending object for its domain on the MessageID header, so GUID@Primary_SMTP_Address_of_Sending_Object. Whenever the primary SMTP address of an Exchange object changes, and that change is reflected in the Global Address Book, any message sent by that object will be in that format. I’m guessing that this was a change meant to help better track messages in Office 365, but I’m curious as to why this behavior change hasn’t been called out in any of the Cumulative updates for 2016 or the official documentation.

Like most companies, my organization uses a proxy for all internet traffic. This presented a problem when we tried using Add-ins for Exchange 2013. At the time we could not figure out how to get the subsystem that pulled down apps in Exchange to use the proxy server despite trying the following methods

·        Configuring the proxy in IE

·        Setting the proxy at the Exchange server level via set-exchangeserver -internetwebproxy

·        Using netsh or proxycfg

Since it was not needed at the time we migrated to Exchange 2013, I dropped the effort. Recently though, after we migrated to 2016, an actual request came in for an app from the Office Outlook app store. Since the servers could not get through the proxy we would see errors in the applications logs (Event ID 3018, see below for an example) and we would get errors every time we tried to add an app via the EMS or EAC. In regard to the Event log error, we would see a different URL referenced each time. When we logged into the Exchange host we could easily get to the URL in Internet Explorer (as along as long as our company’s  proxy settings were in place) but the Exchange server could not reach it

Log Name:      Application
Source:        MSExchangeApplicationLogic
Date:          11/19/2017 1:13:29 AM
Event ID:      3018
Task Category: Extension
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      xchsrv01.contso.com
Description:
Scenario[ServiceHealth]: GetConfig. CorrelationId: e0bc58ff-f87e-4f73-a3df-814b4681bbfb. The request failed. Mailbox:  Url: https://officeclient.microsoft.com/config16?CV=15.1.1034.26&Client=WAC_Outlook&corr=e0bc58ff-f87e-4f73-a3df-814b4681bbfb Exception: System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 40.83.182.229:443
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at Microsoft.Exchange.Data.ApplicationLogic.Extension.BaseAsyncOmexCommand.<>c__DisplayClass2.<EndGetResponseCallback>b__1()
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="MSExchangeApplicationLogic" />
    <EventID Qualifiers="49156">3018</EventID>
    <Level>2</Level>
    <Task>3</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2017-11-19T06:13:29.933295500Z" />
    <EventRecordID>993987</EventRecordID>
    <Channel>Application</Channel>
    <Computer>xchsrv01.contso.com</Computer>
    <Security />
  </System>
  <EventData>
    <Data>GetConfig</Data>
    <Data>e0bc58ff-f87e-4f73-a3df-814b4681bbfb</Data>
    <Data>
    </Data>
    <Data>https://officeclient.microsoft.com/config16?CV=15.1.1034.26&amp;Client=WAC_Outlook&amp;corr=e0bc58ff-f87e-4f73-a3df-814b4681bbfb</Data>
    <Data>System.Net.WebException: Unable to connect to the remote server ---&gt; System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 40.83.182.229:443
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket&amp; socket, IPAddress&amp; address, ConnectSocketState state, IAsyncResult asyncResult, Exception&amp; exception)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at Microsoft.Exchange.Data.ApplicationLogic.Extension.BaseAsyncOmexCommand.&lt;&gt;c__DisplayClass2.&lt;EndGetResponseCallback&gt;b__1()</Data>
  </EventData>
</Event>

After some digging we found out that we needed to set the proxy for the account that is running the app pools for Exchange (which in most cases is LOCALSYSTEM) and that it needed to be set with bitsadmin /setproxysetting. When using this command you will be given a message that it is deprecated but I couldn’t find another method to set the proxy for the LOCALYSTSTEM account. Using bitsadmin You can configure the proxy either manually, like so

bitsadmin /util /setieproxy localsystem MANUAL_PROXY http://http-contso.com:80 "*.corp,contso.com; <local>"

Or using a PAC file

bitsadmin /util /setieproxy localsystem AUTOSCRIPT http://security/webproxy/BalaPAC.pac

We had trouble in our environment getting the PAC file to work with Windows Server 2012R2 and it worked half the time with Windows Server 2016. So, we stuck with the manual method. Our exclusions list was really long and apparently was too big for the buffer to read the settings back using

bitsadmin /util /getieproxy localsystem

Or you can check the following registry entry for to verify the setting took: HKEY_USERS\.DEFAULT\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings

After setting the proxy and restarting IIS, we could install Add-ins via PowerShell and the EAC. Though occasionally we still get the 3018 Application log errors for some URLs, but at least we can now install Add-ins.

At my company we are currently in the early stages of an Exchange 2013 to Exchange 2016 migration and we needed to identify any Outlook clients below a certain patch level (ones we identified as having issues with Mapi over HTTP via a proxy). So we used the following log parser query to gather a list of all clients past a certain patch level after a certain date and ran it against the RPC and MAPI logs on all our Exchange servers.

SELECT EXTRACT_SUFFIX(client-name,0,'=') as User,
client-name as DN,client-software,
client-software-version as Version,
client-mode,
client-ip,
REVERSEDNS(client-ip) as ClientName, protocol,
TO_LOCALTIME(TO_TIMESTAMP(EXTRACT_PREFIX(TO_STRING([#Fields: date-time]),0,'T'), 'yyyy-MM-dd')) AS [Day]
FROM '[LOGFILEPATH]'
WHERE (operation='Connect')
And Day > TimeStamp('2017-07-11','yyyy-MM-dd')
And (Version between '15.0.0000' and '15.0.4849.0000') OR (Version between '14.0.0000' and '14.0.7172.4000')
GROUP BY User,DN,client-software,Version,client-mode,client-ip,ClientName,protocol,Day
ORDER BY User