Category Archives: Exchange

Top 10 Fixes for troubleshooting free/busy between Exchange on-premises and Exchange Online in Office 365

Free/busy often fails to work out-of-the-box after configuring Hybrid Exchange with Office 365. Here are my top ten fixes:

 

  1. Set the sharing policy to match on-premises and cloud.

    First, Connect to Exchange Online Remote Powershell and run get-sharingpolicy

    Then connect to on-premises Exchange Management Shell and run get-sharing policy

    Then make the two match on both sides.

 

Set-SharingPolicy -Identity SharingPolicy01 -Domains ‘contoso.com: CalendarSharingFreeBusySimple’, ‘atlanta.contoso.com: CalendarSharingFreeBusyReviewer’, ‘beijing.contoso.com: CalendarSharingFreeBusyReviewer’

 

  1. Set the organization relationship domains to include all accepted domains on both on-premises and cloud (always requires an IISRESET for it to take effect)
    This script helps identify missing domains in an existing relationship:

     

    if ( (Get-OrganizationRelationship).DomainNames -contains (Get-Mailbox user).PrimarySmtpAddress.Domain) { write-host “The domain was found” -ForegroundColor Green } else { write-host (Get-Mailbox user).PrimarySmtpAddress.Domain “was not found” -ForegroundColor Yellow}

     

    $OrgRel = Get-OrganizationRelationship Contoso

    $OrgRel.DomainNames += “contoso.com”

    Set-OrganizationRelationship $OrgRel.Name -DomainName $OrgRel.DomainNames

     

     

    1. If the autodiscover DNS name is not published in external DNS, and if the client doesn’t want to do that, then manually configure TargetSharingEpr to use the published EWS path

      Get-OrganizationRelationship -Identity “O365 to On-premises – (GUID)” | Set-OrganizationRelationship -TargetSharingEpr https://mail.contoso.com/ews/exchange.asmx

    4) For ‘401 errors’ try disabling the IOC connector in Exchange 2013 to have oAuth fall back to dAuth


    5) Sometimes it’s necessary to set the on-premises EWS virtual directory “WSSecurityAuthentication” value back to defaults (some clients change this if they do load balanced CAS)
    (this is commonly a last resort)

    Need to change WSSecurityAuthentication to False for EWS Virtual directory.

        a.       Set-WebServicesVirtualDirectory “Exch CAS\ews*” –WSSecurityAuthentication $false

        b.      Need to Stop MSExchangeServicesAppPool.

        c.       Need to Start  MSExchangeServicesAppPool.

     

      Need to change WSSecurityAuthentication to True again for EWS Virtual Directory.

        a.       Set-WebServicesVirtualDirectory “Exch CAS\ews*” –WSSecurityAuthentication $True

        b.      Need to Stop MSExchangeServicesAppPool.

        c.       Need to Start  MSExchangeServicesAppPool.

     

      Need to change WSSecurityAuthentication to False for Autodiscover Virtual directory.

        a.       Set-AutodiscoverVirtualDirectory “Exch CAS\Auto*” –WSSecurityAuthentication $false

        b.      Stop MSExchangeAutodiscoverAppPool.

        c.       Start  MSExchangeAutodiscoverAppPool.

     

      Change WSSecurityAuthentication to True again for Autodiscover Virtual Directory.

        a.       Set-AutodiscoverVirtualDirectory “Exch CAS\Auto*” –WSSecurityAuthentication $true

        b.      Stop MSExchangeAutodiscoverAppPool.

        c.       Start  MSExchangeAutodiscoverAppPool.

     

    6) If the Exchange Server is behind a web proxy then it is usually necessary to configure InternetWebProxy Set-ExchangeServer <Server Name> -InternetWebProxy:http://<Proxy Address>:<Proxy Port>

     

    7)  Verify the availability address space and see required SMTP domain with access method.

        Get-AvailabilityAddressSpace (Run this on-prem)

     

    8) Try running diagnostic commands:
    You can also use the Test-FederationTrust (on prem only) and Test-OrganizationRelationship  (run this both on prem and in cloud too)

    And you can also use this website to run tests: https://www.testexchangeconnectivity.com/

    9) Make sure that the cloud user you are searching for has a valid (tenant).mail.onmicrosoft.com alias on their target mailbox (make sure Azure AD Connect is properly replicating that attribute, and/or, that the Exchange Address Policy is not blocking inheritance on that particular user/object).

     

    10) Run these commands to gather diagnostic information:

    Onpremises:

    Start-Transcript

    Get-FederationTrust | fl

    Get-FederatedOrganizationIdentifier | fl

    Get-OrganizationRelationship | fl

    Get-WebServicesVirtualDirectory | Export-Clixml C:\temp\WebVdir.xml

    Get-AutoDiscoverVirtualDirectory | Export-Clixml C:\temp\AutoDVdir.xml

    Get-RemoteMailbox bobc_sync | fl

    Get-Mailbox “on-premises John Doe User” | fl

    Test-FederationTrust -UserIdentity user@domain.com | fl

    Test-FederationTrustCertificate | fl

    Get-IntraOrganizationConnector | fl

    Stop-Transcript

     

    Online:

    Start-Transcript

    Get-FederationTrust | fl

    Get-FederatedOrganizationIdentifier | fl

    Get-OrganizationRelationship | fl

    Get-MailUser “on-premises John Doe User” | fl

    Get-Mailbox “Cloud user” | fl

    Get-IntraOrganizationConnector | fl

    get-OrganizationRelationship | Test-OrganizationRelationship -UserIdentity “cloud user”

    Stop-Transcript

     

     

     

    And when all else fails I reference these two blog articles:

    https://blogs.technet.microsoft.com/exchange/2018/02/06/demystifying-hybrid-freebusy-what-are-the-moving-parts/

    and

    https://blogs.technet.microsoft.com/exchange/2018/03/02/demystifying-hybrid-freebusy-finding-errors-and-troubleshooting/

How to fix Exchange Online Hybrid Outbound Connector 454 4.7.5 Certificate Validation Failure

While recently helping a client setup an Exchange Hybrid, the cloud to on-premises mail flow was failing validation due to 454 4.7.5 Certificate Validation Failure.

The next step was to verify that the TlsCertificateName value was properly set on the send and receive connectors to match the certificate name, following these articles:

https://blogs.technet.microsoft.com/lalitbisht/2017/06/03/mailflow-issue-from-exchange-on-prem-to-office-365/

https://practical365.com/exchange-server/configuring-the-tls-certificate-name-for-exchange-server-receive-connectors/

In this case, the TlsCertificateName was already set correctly to match the certificate name (the Hybrid Exchange Wizard does a good job at setting that correctly).

The next step was to enable Verbose logging on the on-premises receive connector so that we can get a better look at the error.

To save time, I restarted the “Microsoft Exchange Frontend Transport” service so that the logging would take effect sooner.

Then navigating to the log directory can be a bit tricky:

C:\Program Files\Microsoft\Exchange Server\v15\TransportRoles\Logs\FrontEnd\ProtocolLog\SmtpReceive

Opening up the file revealed a very helpful bit of information! The SSL Certificate that Microsoft Office 365 is presenting to the Exchange server for the TLS encrypted email is not a trusted root. How can this be?

 

To cut to the chase, the root cause was that the server had not had windows updates run in a LONG time and therefore was really far behind in its root certificates.

The least disruptive solution was to download the Office certificate chains from Microsoft (here) and install them on the on-premises Exchange Server. Then after restarting the “Microsoft Exchange Frontend Transport” service and waiting a few minutes, the validation was successful.

PowerShell script to automatically heal non-deliverable emails (NDRs) as X500

One of the possible causes for a non-delivery report (NDR) is when a mail object in Exchange is moved or removed (Contact, Mailbox, MailUser, etc). When an object is moved, any internal user who had previously emailed that object will have a cached entry in their autocomplete cache that no longer matches up to what now exists. This happens because the autocomplete cache stores the value of the LegacyExchangeDN attribute of the original object before it is moved. When an object is moved, a new LegacyExchangeDN is created.

When sending a message, Outlook will check the Global Address List (GAL), and if it can’t find a match in the GAL, IMCEA encapsulation is used (Internet Mail Connector Encapsulated Addressing). An IMCEA encapsulated address looks like:

IMCEAEX-_O=CONTOSO_OU=First+20Administrative+20Group_cn=Recipients_cn=JDOE@contoso.com

So when an email is sent to the original cached object, an NDR will be sent back to the user containing a construct of the LegacyExchangeDN that it failed to reach, in a slightly different format:

A trailing “EX” at the end of IMCEAEX indicates that a non-SMTP address was encapsulated.

While a quick fix is to have the sender clear their autocomplete cache and re-send the message, a more automated solution is desirable when there are hundreds of potential senders who cached the old object.

I discovered a nifty PowerShell script (here) written by Michael England on 2/11/2013 that searches the Message Tracking Logs for NDRs, and then reconstructs the LegacyExchangeDN from the IMCEA format and adds that as an X500 proxy alias on the target recipient object (if it can be found). This works great in a scenario where the contact object was replaced by a Mailbox or MailUser object.

One of the things I appreciate about how he wrote this script is that when you run the script it reports what would be modified, then when you are ready to modify you just add the -Autoheal parameter to the end of the script. This way you can get an idea of what would be modified before it happens.

I had to modify Michael’s script because Microsoft changed the behavior of the LegacyExchangeDN value in Exchange 2010 SP1 Rollup 6 (released 10/27/2011) to add 3 random hex characters for uniqueness at the end of the LegacyExchangeDN. So when I attempted to use Michael’s script, it was not finding the destination object to add the X500 to it because the 3 random characters threw the search off. So I have posted a very minor change to an otherwise awesome script to strip the 3 characters when searching for the object to place the proxy alias on.

            Write-Host “looking for: ” $user.Substring(0,$user.Length-3)

            $user = $user.Substring(0,$user.Length-3)

 

I also made three separate copies, depending on the recipient object type (Mailbox, MailContact or MailUser).

You can download my slightly modified version of Michael’s script from the Microsoft Technet Gallery here.

https://gallery.technet.microsoft.com/Search-Message-Tracking-6be6d1b7

I left all original credit to Michael in the script, since I only slightly changed the code to strip the 3 random hex characters out, and made separate copies of the script to search for MailUser and MailContact since his script only searched for Mailboxes. I was stoked to discover this awesome script by Michael.

Also shout out to a different Michael, Michael de Rooij for an excellent blog article on this topic here: https://eightwone.com/2013/08/12/legacyexchangedn-attribute-myth/