Latest update from Microsoft:
Status
Service Restored
Impacted services
Exchange Online
Details
Title: Some users may have been unable to send Exchange Online email messages
User impact: Some users may have been unable to send or receive Exchange Online email messages.
More info: Affected users may have received NDRs or the following error "503 5.5.1 bad sequence of commands"
Final status: We’ve determined that a recent change caused a large influx of free/busy Control Flow Messages (CFM) to be sent to the Exchange online infrastructure, resulting in stale free-busy information. An interceptor rule was applied to discard these CFM messages to mitigate impact for the free/busy issue, which inadvertently caused impact to mail flow.
We've removed the interceptor rule and verified that the impact to the mail flow has now been mitigated. We'll continue to investigate the free/busy issue, which is being communicated under the SI EX648815.
Due to the nature of the issue, the users who had received the NDRs during this event will need to resend those emails.
Scope of impact: Impact was specific to some users who are served through the affected infrastructure.
Start time: Tuesday, July 18, 2023, at 5:00 AM UTC
End time: Tuesday, July 18, 2023, at 8:25 AM UTC
Root cause: We’ve determined that a recent change caused a large influx of free/busy Control Flow Messages (CFM) to be sent to the Exchange online infrastructure, resulting in stale free-busy information. An interceptor rule was applied to discard these CFM messages to mitigate impact for the free/busy issue, which inadvertently caused impact to mail flow.
Next steps:
- We're reviewing our interceptor rule implementation procedures to understand why mail flow was inadvertently affected, in order to help prevent similar impact from happening again in the future.