Category Archives: Uncategorized

Attack Simulator for Office 365

Microsoft has released Attack Simulator (currently in Preview) to allow Office 365 Global Administrators to simulate phishing campaigns and other attack simulations.

The obvious value is finding out which users are most susceptible to phishing attacks so that you can educate them before an actual attacker exploits them.


  • Your organization’s email is hosted in Exchange Online (Attack simulator is not available for on-premises email servers)
  • You have an E5 license, or have signed up for an E5 trial license (here)
  • You have the security administrator role or Global Administrator role assigned to you
  • You have multi-factor authentication enabled (make sure to first read the MFA prerequisites here, such as enabling oAuth via powershell)

Getting Started

To access Attack Simulator, in the Security & Compliance Center, choose Threat management > Attack simulator. If you don’t see it yet, you can browse to it directly here:

There are currently three attackers offered by Attack Simulator:

  1. Display name spear-phishing attack
  2. Brute Force password attack
  3. Password spray attack

In this blog post we will quickly cover the first simulation. Feel free to click on the documentation link in the reference table below to read about the other two attack simultaneous.

Display name spear-phishing attack

One of the more common and successful phishing methods is to spoof the Display Name field in Outlook. This is very effective because Sender Policy Framework (SPF)only protects the RFC 5321.Mail From field, and does not protect against spoofing of the Display Name. Only Domain-based Message Authentication, Reporting & Conformance (“DMARC” – RFC 7489) protects against the Display Name field (RFC 5322.From Field). However, since very few organizations have implemented DMARC, then this simulated phishing attack is very effective.

Carrying out the phishing simulation is a straight-forward wizard in the documentation found (here). Basically you enter the email address that you want to spoof and the targeted users that you want to send the fake email to. You can pick from a few pre-built templates, then you can do some customization of the email that would be sent out. After running the campaign, you can monitor to see which users clicked on the link, and which users went a step further and gave away their credentials.

How does this compare to other Phishing Simulators?

Other phishing simulators such as KnowBe4 or PhishMe have been around a lot longer, obviously, but Attack Simulator is great for customers who maybe already own the E5 license and want to phish their users at no added cost. If you only have E3 then you could purchase “Threat Intelligence” as an add-on license on top of E3 in order to get the Attack Simulator feature. However, there is another recently added feature included in the Advanced Threat Protection (ATP) license called ATP Anti-Phishing Policies which you would also get in the E5 license and therefore I feel the best value is to get the E5 rather than trying to purchase separate add-ons. I wrote a little bit about the new Anti-Phishing solution in my recent post where I wrote about the top 15 things to do before and after a phishing attack in Office 365. Basically, the new Anti-Phishing Policy can send items to quarantine if any part of the email address has been modified to bypass DMARC. For example, while DMARC protects the exact spelling of an impersonated CEO, it does not protect against a slight variation of a CEO’s address. Like spelled with a zero instead of an alphabetic O, like In those cases, the new Anti-phishing policy can be configured to send those emails to quarantine, or redirect them to a security team, or other actions.


Need help?

Patriot Consulting provides assistance with deploying Microsoft Security solutions. We start with a free consultation to help you understand your current Microsoft licensing level, and we help you deploy the security solutions that you may already own inside your Microsoft licenses. Then we can help you pilot additional security solutions from Microsoft.

Why Patriot?

We are a Microsoft Gold EMS Competency Partner and have helped hundreds of companies deploy Microsoft security solutions. We focus 100% exclusively on Microsoft Cloud technologies and believe in “do one thing and do it well.” We participate in the Microsoft Partner Seller Program, and we are a Managed Microsoft Partner, which gives us access to the latest training and roadmap. As a member of the Microsoft Security Council, we have direct access to the Microsoft Product Group that develops the software.



15 Things to do before and after a phishing event in Office 365

Statistics indicate that 20% of corporate users will give away their username and password when asked to do so by a social engineer (for example through a phishing email).

Some of the more clever and convincing ones originate from a trusted person such as the CEO, HR Department, IT Department, or even Microsoft. The HR Department example might say “you have received an encrypted message from HR” and if you click on the link to view the message, it steals your O365 password. The attacker then logs into your account, forwards your email to them, and then send emails out to your customers or other colleagues to continue to propagate.

Here are a few tips on how to prepare for when this happens to you.

  1. Be prepared to Reset the affected user’s password right away. Note that if you reset the password on-premises, it can take a few minutes before that password change is synced to Office 365 (if you are using Password Hash Sync, it can take 3 to 4 minutes). If you are using ADFS then there is no delay.
  2. Document the steps to immediately revoke an active user’s session in Office 365, forcing them to try to logon with the new password. There are three supported methods
    “The first option is found in the Office 365 Admin Center under Home > Active Users. Select a user and expand the OneDrive Settings section for that user. Select “Initiate” to perform a one-time sign-out for that user that revokes active sessions across Office 365 services including Exchange Online.
    The second option to force logoff during an active user session in Office 365 to use Revoke-SPOUserSession cmdlet from the SharePoint Online PowerShell Module. This method is helpful for automating security incident response flows or when there is a need to revoke multiple users’ sessions.
    The third option to force a user sign-out extends beyond Office 365 services to all active user sessions in any Azure AD application. The Revoke-AzureADUserAllRefreshToken cmdlet is available in the AzureAD V2 PowerShell Module and expires a user’s refresh token by modifying the user’s token validity period”
  3. Deploy Multi Factor Authentication on targeted users, privileged users, and users who access sensitive information. Many people do not know that O365 includes free MFA without the need for additional licenses.. it comes built into all O365 plans.
  4. Check to see if mailbox forwarding was enabled, and if so to who (document the external addresses to verify the validity).
    Here is a great one-liner to run in Exchange Online Powershell:
    get-mailbox -resultsize unlimited |where {$_.ForwardingSmtpAddress -ne $null} | select displayname,forwardingsmtpaddress
  5. Check message trace logs in Exchange Online Admin center ( to see what items were sent to suspected unauthorized external accounts.
  6. Disable forwarding via Transport Rule, and create an alert in Security and Compliance Center when someone tries to create a forwarding inbox rule (Indicator of Compromise)



  7. Enable Mailbox Auditing in Exchange Online. Many people do not realize that Mailbox Auditing is disabled by default in Exchange Online, and that even if you enable it, the ‘Message Bind’ action cannot be logged, so it is not possible to know when an attacker viewed a particular message. You also won’t be able to determine whether the attacker downloaded a local copy or not. If you use RMS or Azure Information Protection, then additional logging is possible but if the identity is compromised for someone who is authorized to view the email or document, then that form of Encryption doesn’t help.
  8. Review Azure Reports on a frequent basis
    1. Risky Sign-Ins
      1. Sign-ins from anonymous IP addresses
      2. Impossible travels to atypical locations
      3. Sign-ins from infected devices
    2. Users flagged for risk

    Note: These reports are pretty basic but if you own Azure AD Premium P1, then you can drill into ‘why’ a user was flagged as a risk.

  9. Use Message Trace to see who received emails from the attacker’s email address.
  10. Use ATP URL Trace to view who clicked on the hyperlink sent from the attacker.
  11. Purge the email with powershell for any user who has not yet clicked on the email sent from the attacker.
  12. Cloud App Security is valuable for many reasons, but it extends the auditing to 180 days whereas the built-in audit logs in the Office 365 Security and Compliance Center only go back 90 days.
    Licensing: CAS is available in two forms, O365 E5 or EMS E5… the former protects mostly O365 and 750 other SaaS apps, whereas the later protects 15,000 SaaS apps and supports automatic log uploads from your on-premises firewalls.
  13. Office 365 Threat Intelligence (an E5 feature) can identify who your top targeted users are and alert you when there are active email campaigns going on so that you can alert your users of the threat.
  14. Consider Disabling User Consent to 3rd party applications in Azure Active Directory. This prevents users from granting consent to 3rd party apps that may be the next wave of ransomware, that encrypts mailboxes. A proof of concept was recently demonstrated on the internet.
  15. Deploy ATP Anti-Phishing – Just started rolling out on 2/5/2018. For more details:


  • Deploying MFA should be the first priority because if a user gives away their credentials, then the attacker cannot access the mailbox to do further damage.
  • Many people ask me how to view reports of who has or who has not been enabled for MFA. There are not GUI reports available for this in O365, so I wrote some powershell scripts at the bottom of this blog post to help you enumerate those scenarios.
    Hint: It is highly recommended to enable oAuth first (via PowerShell) so that users are not prompted to use ‘MFA App Passwords)
    oAuth is off by default in Exchange Online and Skype for Business Online. It is ON by default in SharePoint and OneDrive. For more info see:



  • Disabling mailbox forwarding is important because in the most recent incidents, the attacker will forward the mailbox to an outside email address and monitor for a while before initiating emails to customers or other employees.
  • Enabling auditing in Exchange Online is important, because by default auditing mailbox activity is disabled. But enabling it is not as easy as you would think – you have to be specific on what actions you want to audit, so I have included examples below.
  • Reviewing the Azure reports is important because they will indicate whether a user’s mailbox is being accessed by an unusual or distant IP address. This is often how you will find out that an account has been compromised.

Exchange Online Mailbox Auditing 101

get-mailbox | group-object AuditEnabled

This command will give you a quick and high level picture of how many accounts have Auditing enabled.

get-mailbox -resultsize unlimited | set-mailbox -AuditEnabled $true -AuditLogAgeLimit 180

This command will enable mailbox auditing on all accounts and increase the default audit level from 90 to 180

The following commands will show you the default auditing settings on a single mailbox user “Joe”

get-mailbox joe | select -ExpandProperty auditadmin

get-mailbox joe | select -ExpandProperty auditowner

get-mailbox joe | select -ExpandProperty auditdelegate


You’ll notice that the Mailbox Owner auditing only logs a single event by default: MailboxLogin

That’s unfortunate, because you might really want additional details. Therefore, to enable the maximum level of auditing that you can for a mailbox owner, here is the command:

get-mailbox -ResultSize unlimited | set-mailbox -AuditOwner @{Add=”create”,”HardDelete”,”MailboxLogin”,”Move”,”MoveToDeletedItems”,”SoftDelete”,”Update”,”UpdateFolderPermissions”}

Please note that I am not aware of any way to enable mailbox auditing by default on all new accounts, so make sure that your new hire onboarding scripts takes this into account (and have them enable MFA for all new accounts while you are at it :)



MFA Reporting

The MFA reporting in Office 365 is almost non-existent. You need to go to powershell to audit who has been enforced, enabled or is not yet enabled.

  1. Enabled (Means the user has been enabled but they have not yet completed MFA registration)

Get-MsolUser -All | where {$_.StrongAuthenticationRequirements.state -eq ‘Enabled’ } | Select-Object -Property UserPrincipalName,whencreated,islicensed,BlockCredential | export-csv enabled.csv -noTypeInformation



  1. Enforced (The user has completed MFA registration, so their account is not protected by MFA)

Get-MsolUser -All | where {$_.StrongAuthenticationRequirements.state -eq ‘Enforced’ } | Select-Object -Property UserPrincipalName,whencreated,islicensed,BlockCredential | export-csv enforced.csv -noTypeInformation


  1. Not Yet Enabled (These users have not yet been enabled for MFA)


Get-MsolUser -All | where {$_.StrongAuthenticationMethods.Count -eq 0 -and $_.UserType -ne ‘Guest’} | Select-Object -Property UserPrincipalName | export-csv non-enabled.csv -noTypeInformation


Need Help?

Patriot consulting offers many security services for Office 365 including deploying any of the security solutions you read about in this article. We can also do a full audit of your Office 365 environment and make recommendations to harden the security. We also offer incident response services after you get phished. Contact us at


PowerShell script to automatically heal non-deliverable emails (NDRs) as X500

One of the possible causes for a non-delivery report (NDR) is when a mail object in Exchange is moved or removed (Contact, Mailbox, MailUser, etc). When an object is moved, any internal user who had previously emailed that object will have a cached entry in their autocomplete cache that no longer matches up to what now exists. This happens because the autocomplete cache stores the value of the LegacyExchangeDN attribute of the original object before it is moved. When an object is moved, a new LegacyExchangeDN is created.

When sending a message, Outlook will check the Global Address List (GAL), and if it can’t find a match in the GAL, IMCEA encapsulation is used (Internet Mail Connector Encapsulated Addressing). An IMCEA encapsulated address looks like:

So when an email is sent to the original cached object, an NDR will be sent back to the user containing a construct of the LegacyExchangeDN that it failed to reach, in a slightly different format:

A trailing “EX” at the end of IMCEAEX indicates that a non-SMTP address was encapsulated.

While a quick fix is to have the sender clear their autocomplete cache and re-send the message, a more automated solution is desirable when there are hundreds of potential senders who cached the old object.

I discovered a nifty PowerShell script (here) written by Michael England on 2/11/2013 that searches the Message Tracking Logs for NDRs, and then reconstructs the LegacyExchangeDN from the IMCEA format and adds that as an X500 proxy alias on the target recipient object (if it can be found). This works great in a scenario where the contact object was replaced by a Mailbox or MailUser object.

One of the things I appreciate about how he wrote this script is that when you run the script it reports what would be modified, then when you are ready to modify you just add the -Autoheal parameter to the end of the script. This way you can get an idea of what would be modified before it happens.

I had to modify Michael’s script because Microsoft changed the behavior of the LegacyExchangeDN value in Exchange 2010 SP1 Rollup 6 (released 10/27/2011) to add 3 random hex characters for uniqueness at the end of the LegacyExchangeDN. So when I attempted to use Michael’s script, it was not finding the destination object to add the X500 to it because the 3 random characters threw the search off. So I have posted a very minor change to an otherwise awesome script to strip the 3 characters when searching for the object to place the proxy alias on.

            Write-Host “looking for: ” $user.Substring(0,$user.Length-3)

            $user = $user.Substring(0,$user.Length-3)


I also made three separate copies, depending on the recipient object type (Mailbox, MailContact or MailUser).

You can download my slightly modified version of Michael’s script from the Microsoft Technet Gallery here.

I left all original credit to Michael in the script, since I only slightly changed the code to strip the 3 random hex characters out, and made separate copies of the script to search for MailUser and MailContact since his script only searched for Mailboxes. I was stoked to discover this awesome script by Michael.

Also shout out to a different Michael, Michael de Rooij for an excellent blog article on this topic here:

Change Skype Audio Conferencing Number in Bulk using PowerShell

A client recently asked me if it was possible to change their default Skype Audio Conferencing phone number, because the one they had been assigned was in a different city than where their headquarters was. While it is possible to do a bulk change in the graphical user interface, doing it in PowerShell was better because we could target the change to only users using the old number. Here is the one-liner powershell syntax that we used:

get-CsOnlineDialInConferencingUser | Where-Object {$_.servicenumber -eq (old number)} | %{Set-CsOnlineDialInConferencingUser -identity $_.sipaddress.substring(4) -ServiceNumber (new number)}

To change it for an individual user:

individual user: Set-CsOnlineDialInConferencingUser -Identity (username) -ServiceNumber (new phone number)


Notes: Does not require meeting update service to run! old invites still work fine!

Office 365 Groups Expiration Review

This is a quick review of the new groups expiration feature in Office 365.

Pros: Very simple to configure – set a group expiration of 180, 365, or Custom.

Then enter an email address of someone to notify if a group does not have an owner.

Those two settings make perfect sense.

The third setting is why I am writing this blog post. The setting ‘Enable expiration for these Office 365 groups” [All] [Selected] or [None]

Let’s dissect this a bit…

ALL probably makes sense …

None might make sense…


But I’m having a hard time understanding when I would select certain groups for expiration. You see, by the very nature these Groups are very dynamic usually – by default, any user can create a group. So if today I pick a set of 15 groups that I want to expire, then tomorrow there could be 30 more created that will not expire. So then, I would have to continuously come back here and update that list if there were some groups that I did not want to have deleted. So my choice would then have to be revert to the None setting.

What’s really needed is an exclusion list, ex: Expire all groups EXCEPT for these 5 that I really really care about. All the others, let the owners decide if they want to keep them, but these 5, I keep important stuff in there, and I don’t want to sweat it about missing an email and potentially losing all that information.

So Microsoft, I hope you are listening, please add an Exclusion button. I posted this idea to the UserVoice site here if you want to vote on it!


End of Support for Office for Mac 2011

Microsoft will end support for Office for Mac 2011 on Oct. 10, 2017 (a date set two years ago). After that date, Microsoft will no longer provide patches for security vulnerabilities or fixes for other bugs, and halt both free and paid assisted support.

How to block legacy authentication in Azure AD Premium Conditional Access

Azure AD Premium’s Conditional Access feature requires Modern Authentication to function properly. This has led some to believe that legacy clients (ex: Outlook 2010 and older, or Activesync) can bypass Conditional Access Policies.

Based on my testing, this is only half true, as it depends upon the policy that you select. If you select a ‘Grant’ policy then the legacy clients will not be able to bypass your conditional access policy. However, if you select a Block policy, then the legacy clients will bypass it and connect to the service that you want to block.

So the most conservative thing to do is to use a Grant Policy, not a Block policy.