Monitor & Respond - Email User Submissions Queue

Role Requirements

Procedure Scope: Administrators

Required Group Membership: Admin.Security

Handbook Reference

Package: TBD

Domain: TBD

Modifies: TBD

Full Description

The Defender for Office 365 User Submission Queue serves as a critical tool for fostering user security awareness, allowing analysts to assess and respond to user-reported emails. This feature provides a direct way for users to flag emails they may find as spam, phishing, or clean, and have an analyst review them. tuning the email filter appropriately. The daily review of the user submission queue is important not only for prevention and email filter tuning, but also for establishing a security culture where users can be confident that they will receive accurate responses to their email concerns. Leveraging this feature builds on acknowledging the importance of users and their vigilance in ensuring the organizations email stays secure.

Security Importance: Monitoring and responding to the submission queue helps to refine threat protection, enabling users’ vigilance to be an extension of the security team in protecting the organization from threats.

Business Importance: Monitoring and responding to the submission queue helps to declutter mailboxes, as reports for spam and phishing will automatically be moved out of the user's main inbox and help cut down on unwanted emails.

When to Perform this Operation: Twice a day at key times, such as 8am and 2pm.

Included in this Operation:

Train

• Understand Users’ Options and Microsoft’s Analysis
• Understanding Email Security Information
• Understanding Email Detection Technologies
• Analyzing Email Security Context
• Recognize when to Mark as Clean
• Recognize when to Mark as Phish
• Recognize when to Mark as Spam

Monitor

• Review the Email User Submissions Queue

Respond

• Respond with Email Mark and Notify Spam Action
• Respond with Email Mark and Notify Phishing Action
• Respond with Email Mark and Notify No Threats Action

                                                                        Train

In the context of managing the email user submission queue, training goes beyond marking an email as what users have submitted it as. Analysts need the ability to accurately respond as a subject matter expert, which in some cases, may not align with their original reason for reporting. Understanding the security information tied to each entry, comprehending the detection methods in use, and identifying instances where users can be misled are core skills needed for this.

This skillset enables analysts to process the entries efficiently while maintaining accuracy and quality in decision-making. Effective training here isn’t always about agreeing with users — it’s having the judgment needed to provide feedback on users suspicious around emails they receive, confidently ensuring the queue is managed effectively and users receive accurate feedback for their involvement in contributing to a secure email security culture.

Understand Users’ Options and Microsoft’s Analysis

It can be helpful to know both what users can report and how Microsoft’s automatic analysis categorizes each submission. This insight lets you respond confidently and back up user feedback with solid threat intelligence.

User Submission Options
Users have three main choices for reporting emails, each showing how they see the email’s intent:

•  Junk: This means the email isn’t dangerous, just unwanted. It might be spam or something clogging up the inbox but not a security risk. The message is moved to the junk folder at the time of reporting.
• Phishing: Users pick this when they see an email that looks suspicious or like a threat—maybe something trying to impersonate someone it’s not, or it is simply more pressing than junk. These are flagged as dangerous. The message is moved to the deleted folder at the time of reporting.
• Not Junk: This is for legitimate emails that shouldn’t have been blocked in the first place. Users submit these when they want similar emails allowed in the future, or, to reverse the effects of accidentally reporting a legitimate email as spam or phishing. The message is moved to the inbox folder at the time of reporting.

Microsoft Submission Results
Once a user reports an email, Microsoft’s system automatically re-scans as if it was being delivered to the mailbox at the time of report, and categorizes it as one of these:

• Bulk: Likely part of a campaign which is not malicious or spam, but more bulk. Probable cause for why this email was delivered to the inbox in the first place can be attributed to newer campaigns being identified after delivery, and ZAP functionality not having the priority to proactively remove a “non-threat” from user inboxes.
• Threats Found: Confirmed to have a threat such as spam, phishing, or malware. Probable cause for why this email was delivered to the inbox can be attributed to newer spam campaigns or a change in URL or file determinations post-delivery. It is typically a spam determination, as ZAP functionality can proactively remove a phishing or malware threat from user inbox via action center approval, however, edge cases where user submissions time beats the continuous rescanning this service employs can occur.
• No Threats Found: Safe email, with nothing suspicious detected. Microsoft’s analysis shows it’s clear based on the threat analytics at the time of reporting and does not match known email threats.

 When automated response for user submissions is enabled, the Microsoft attempts to automate this behavior. However, real world testing has yielded erratic results such as those below, and do not provide consistent enough behavior for the scanning and response engine employed to be effective enough for the task at hand.

Understanding Email Security Information

When an email is clicked into, useful information becomes available. This can be used for the analysis of primary indicators.

Result details section:

• Result: Microsoft’s analysis of the email at the time of user report.
• Recommended Steps for Email Submissions: Extra steps that can be used to either allow, block, or further investigate the email based on the classification given by Microsoft.

Reported Message details section:

• Date Reported: The date/time when the message was reported by the user.
• Submission Name: The subject of the email.
• Reported Reason: The original choice made by the user via the Report Phishing button.
• Message Reported ID: The unique ID for the reported message, as it is used by Microsoft.
• Reported By: The user which has reported the message.
• Phish Simulation: If the email is noted to be from an internal Microsoft phishing simulation campaign.
• Converted to Admin Submission: If the email was submitted to Microsoft by an administrator.

Delivery details section:

• Original Threats: General detection of email threats using technologies such as phishing, spam, or malware, as they were identified at the time of initial receipt.
• Latest Threats: General detection of email threats using technologies such as phishing, spam, or malware, as they were identified at the time of user report.
• Original Location: Initial delivery or detection location, such as Inbox folder, Junk Email folder, Quarantine, or External.
• Latest Delivery Location: Post-action location of the message, updated after actions like ZAP or admin interventions; includes Inbox, Quarantine, Deleted Items, etc.
• Delivery Action: System actions based on detection, including Delivered, Junked, and Blocked.
• Detection Technologies: Technologies used to identify threats, including Advanced filter, Campaign, File detonation, Impersonation user, and URL detonation.
• Primary Override: Actions due to policy or user/admin override, with options like Allowed by organization policy, Blocked by user policy, and sources including Antispam policy settings and Tenant Allow/Block List.
• Additional Action: Actions that have occurred after delivery, as they relate to post-delivery remediation activities. In typical environments, this occurs as ZAP remediations, unless administrators initiate a manual remediation action for the email.

Email details section:

• Sender display name: Display name associated with the sender, portion of the from section that does not include the [@] domain portion.
• Sender address: Address associated with the sender, if the display name and the address do not match or follow a similar formatting this can be an indicator of something suspicious.
• SMTP Mail From address: Indicates the address used by the mail client or application.
• Sent on behalf of: Indicates that the mail was sent from another user, this will bring awareness to who initiated the message send.
• Return path: Used to verify the identity of where the email originated from, if this address differs from the SMTP Mail from Address, this could be an indicator of something suspicious.
• Sender IP: The IP address associated with sender.
• Location: The location based on the associated IP address.
• Recipients: The address associated with the recipient.
• Time received: Provides the Date and Time the message was received.
• Directionality: Specifies if the mail is inbound or outbound.
• Network message ID: A network message id is a unique identifier for a digital message, most commonly an email. It is generated by the sending mail server or host and has a specific format that is a subset of an email address. The message id can contain the name and location of the server that generated it and can be used to trace the message hops in the internet header.
• Internet message ID: A unique identifier that refers to a particular version of a particular message.
• Campaign ID: A unique number used to calculate the success and allows for reporting on groups of campaigns.
• DMARC: Domain-based Message Authentication Reporting and Conformance (DMARC) tells a receiving email server what to do given the results after checking SPF and DKIM. A domain's DMARC policy can be set in a variety of ways — it can instruct mail servers to quarantine emails that fail SPF or DKIM (or both), to reject such emails, or to deliver them. A success or fail will be applied to this section based on the result of the DMARC check.
• DKIM: DomainKeys Identified Mail (DKIM) enables domain owners to automatically "sign" emails from their domain. DKIM uses public key cryptography; a DKIM record stores the domain's public key, and mail servers receiving emails from the domain can check this record to obtain the public key, the private key is kept secret by the sender, who signs the email's header with this key, Mail servers receiving the email can verify that the sender's private key was used by applying the public key. A success or fail will be applied to this section based on the result of the DKIM check.
• SPF: Sender Policy Framework (SPF) is a way for a domain to list all the servers they send emails from. SPF records list all the IP addresses of all the servers that are allowed to send emails from the domain, mail servers that receive an email message can check it against the SPF record before passing it on to the recipient's inbox. A success or fail will be applied to this section based on the result of the SPF check.
• Composite authentication: Helps identify spoofed senders and phishing emails when senders do not use SPF, DKIM, and DMARC authentication.

URLs section:

• URL: Provides a list of all URLs that were housed within the email.
• Threat: Indicates if any of the housed URLs show signs of being malicious or compromised.

Attachments section:

• Attachment name: Provides the name of the attachment.
• Threat: Indicates if any of the provided attachments are malicious.
• Detection tech/Malware family: Provides additional information on the type of malware that might be present within the attachment.

Understanding Email Detection Technologies

Understanding “Spam”, “Phish”, or “Malware” is only the general determination. The delivery details section specifies the status of the emails. Use this section to determine the how the determination has changed since being reported and align the email security information and context activities to investigate this determination. For Microsoft’s verbatim definitions of these, view the knowledge article here.

Analyzing Email Security Context

For each analysis, the primary indicators should be analyzed to determine the technical legitimacy and any alignment with common email threats. Informed by the detection technology, investigation of these items will contribute to the overall legitimacy of user submissions and help make a final determination. Validation of these indicators against contextual data from headers and message content can be performed in the subsequent step with the aim to validate results found from this primary indicator analysis.

Subject Line:

• Urgency or fear-inducing language ("Immediate action required", "Alert", "Notice").
• Grammatical errors, odd phrasing, or unusual formatting.
• Relevance and consistency with expected communication themes.

Recipient Tag:

• If tagged as a "Priority account," extra scrutiny is needed since such accounts are often targets of focused attacks.

Sender Information:

• Compare the sender's display name and email address for mismatches or suspicious similarities to known brands (e.g., "Micros0ft" instead of "Microsoft").
• Check if the return path or sender email match the receiving organizations domain, indicating a phishing attempt or 3^rd party mailer utilizing the domain space.
• Check the return path and sender email address for discrepancies that could indicate masking or redirection.

Directionality:

• Identify is the directionality of Inbound, Intra-org, or Outbound align with the claims made by the sender and recipient, specifically to differentiate inbound phishing attempts from intra-org communications.

Campaign ID:

• Identify if the email is part of a larger campaigned, denoted by a campaign identifier.

Authentication Results:

• Verify DMARC, DKIM, and SPF results. Any failures here are significant red flags for domain spoofing or misuse.

URL Examination:

• Check for shortened URLs that might mask malicious sites.
• Look for URLs that misspell well-known domain names.
• Optional: Perform a URL analysis via a 3^rd party to assess any links flagged as suspicious to confirm if they are known for hosting malware or phishing schemes.

Attachment Review:

• Identify file types; be wary of executable files (.exe, .scr) or documents with macros (.docm, .xlsm).
• Check the detection technology or malware family associated with any attachments flagged as suspicious to understand the specific threat they might pose.

Recognize when to Mark as Clean

When it comes to marking an email as clean, it’s important to make this determination before marking as Spam or Phishing. This initial step is critical because it is where users are protected from accidental submissions impacting delivery of critical email. In these scenarios, the initial submission reason is key in determining these. This is because we need to account for users' intent, whether it be verifying their intuition, stopping accidental reporting of legitimate email, or undoing a previous submission. In essence, this first step ensures email that is reported into this queue does not allow a user to mistakenly cause impact to availability.

Marking Clean when Reported as Phishing or Spam

When the item has been reported as phishing or spam, it can be reasonably assumed the user noticed reason for a threat, or simply does not want the email in their inbox. While some users may interchangeably submit using Junk and Phishing, this typically indicates the user observed a reason to no longer receive these types of email. Overriding these should only be performed when the analyst can be exceptionally confident that the email is indeed clean, and the submission is user error. The situations where analysts may see this in play are as such:

• Internal Communications: Daily operations, meetings, or internal announcements from colleagues or departments. It is not uncommon for a user to occasionally report an email as phishing, despite it being from an internal sender.
• Business Deals: Details about negotiations, contracts, or agreements. These can be trickier to interpret, however, an understanding of the business and its common business partners can help in identification.

Marking Clean when Reported as Not Junk

When the item has been reported as not junk (clean), it means a user may have accidentally submitted it as Phishing or Junk in the past and is looking to undo the action. Additionally, the user could be looking for confirmation from the security team that an email does not exhibit malicious attributes. Confirming a clean action is different than an override of a different determination as clean, where the analyst knows specific conditions in which it occurs. Rather, the email should be treated more in line with verifying the security legitimacy and safety – not so much caring about whether it feels relevant or not. Taking into account security information and context is crucial. The situations where analysts may see this in play are as such, but not inclusive of all scenarios given how broad it could be interpreted, are as such:

• Internal Communications: Daily operations, meetings, or internal announcements from colleagues or departments.
• Business Deals: Details about negotiations, contracts, or agreements.
• Client Communications: Project updates, feedback, or requests from clients or stakeholders.
• Technical and IT Alerts: Notifications related to IT maintenance, system outages, or security alerts.
• HR Communications: Updates on policy changes, employee benefits, or payroll information.
• Legal Notices: Compliance requirements, litigation issues, or regulatory communications.
• Transactional Emails: Order confirmations, shipping notifications, or transaction receipts.
• Education and Training: Professional development opportunities, training sessions, or educational materials.

Recognize when to Mark as Phish

When it comes to marking an email as phish, it’s important to make this determination before marking as Spam. This secondary step is equally important as marking clean, as it is where users are protected potentially malicious emails. In these scenarios, the initial submission reason matters less because the security of the email is based on analysis, rather than the users opinion. Analysis of these emails should focus on identifying those emails which have malicious attributes and could lead to potential security risks such as malicious email attachments, URLs, or impersonation of users.

Marking Phish when Reported as Phish, Not Junk, or Junk

Differently than marking as clean, where the reported context can inform the outcome, malicious emails are universally malicious. As such, ensuring that primary indicators and contextual verification has been performed is especially important for this step. The conditions to mark as phish mirror those actions of preventing release that are observed in the quarantine queue.

• Phishing, Suspicious, or Malicious Emails: Emails that attempt to have users perform tasks through deceptive requests, encourage users to click or download contents, respond back with sensitive information, or masquerade as other companies or people. Often contain urgent, threatening, or suspicious language. May include mismatched/malicious links and malicious attachments. Typically combined with failed primary identifiers.

Recognize when to Mark as Spam

When it comes to marking as spam, marking to align with the users’ submission is usually appropriate, and the validation process is less intensive. At this point in the analysis process, the message has been confirmed to not have legitimate business need and does not contain malicious intent. As a result, any markings of spam are for inbox cleanliness and are not in scope to dramatically affect the organization. The situations where analysts may see this in play are as such:

• Technical and IT Alerts: Notifications related to IT maintenance, system outages, or security alerts.
• Transactional Emails: Order confirmations, shipping notifications, or transaction receipts.
• Education and Training: Professional development opportunities, training sessions, or educational materials.

                                                                      Monitor

In managing the Defender email user submission queue, monitoring focuses on providing users with prompt, accurate feedback on reported emails, whether flagged as potential threats or marked as clean.

By evaluating relevant security data alongside any available contextual information, analysts can deliver timely responses that reinforce users' trust in the system. This approach ensures users receive clear, constructive feedback, whether confirming safety or addressing potential risks.

Review the Defender for Email User Submissions Queue

Finding emails which have been submitted by users and not yet reviewed by an analyst is the primary goal of monitoring.

[Insert the embedded link here]

                                                                          Respond

In the context of email user submission management, a response action is always warranted.

This fact is due to the nature of the email user submission management system, wherein, an item in the queue indicates a user is awaiting response and grading. Although the core scope of this operation is focused on grading and notifying user submission, analysts can opt to use these submissions to tune the email filtering rule stack by additionally submitting these entries via the “Defender for Email Mark and Notify No Threats Action” article.

Respond with Email Mark and Notify No Threats Action

This action applies to email submission entries which have been confirmed by an analyst as not containing threats and will subsequently notify the user the message they reported was confirmed to be clean.

[Insert the embedded link here]

Optional Email Tuning: Defender for Email Submit and Allow Action

If an item has been identified as no threats (clean), the user training can be used to further tune the email filter. Submitting clean emails to the user submission queue automatically helps the Microsoft scanning engine, however, performing the submit and allow action ensures they are delivered to the user next time. While this step is not required to effectively manage the user submission queue, it is effective in fine tuning the email filter.

[Insert the embedded link here]

Respond with Email Mark and Notify Phishing Action

This action applies to email submission entries which have been confirmed by an analyst as phishing and will subsequently notify the user the message they reported was confirmed to be phishing.

[Insert the embedded link here]

Optional Email Tuning: Defender for Email Submit and Block Action

If an item has been identified as phishing, the user training can be used to further tune the email filter. Submitting phishing emails to the user submission queue automatically helps the Microsoft scanning engine, however, performing the submit and block action ensures they are prevented from reaching the user again. While this step is not required to effectively manage the user submission queue, it is effective in fine tuning the email filter.

[Insert the embedded link here]

Respond with Email Mark and Notify Spam Action

This action applies to email submission entries which have been confirmed by an analyst as spam and will subsequently notify the user the message they reported was confirmed to be spam.

[Insert the embedded link here]

Need Assistance?