Data Warehouse Stale Due To Azure Issues
Incident Report for Washington University - IT
Resolved
Microsoft has resolved their underlying infrastructure issue. All WashU data and processes have returned to baseline.
Posted Jan 13, 2025 - 09:20 CST
Update
Microsoft continues to experience a service disruption in their Azure data center, per their status page at https://azure.status.microsoft/en-us/status. WashU IT has identified the resulting impact of delayed data on reporting. The most recent update from Microsoft states that their service has mostly recovered, but that some clients may still have delays or other issues until everything is completely recovered. WashU IT will continue to monitor our scheduled jobs and will provide an update ~10am on Monday morning 1/13.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions.
Posted Jan 10, 2025 - 17:31 CST
Monitoring
Microsoft continues to experience a service disruption in their Azure data center, per their status page at https://azure.status.microsoft/en-us/status. WashU IT has identified the resulting impact of delayed data on reporting. Although the vendor has not provided a formal resolution, all data marts currently up to date.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions.
Posted Jan 10, 2025 - 15:46 CST
Identified
Microsoft continues to experience a service disruption in their Azure data center, per their status page at https://azure.status.microsoft/en-us/status. WashU IT has identified the resulting impact of delayed data on reporting. Although the vendor has not provided a resolution formally, we have been able to manually run some jobs, with intermittent success pulling in some Workday data.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions.
Posted Jan 10, 2025 - 13:21 CST
Update
Microsoft continues to experience a service disruption in their Azure data center, per their status page at https://azure.status.microsoft/en-us/status. WashU IT has identified the resulting impact of delayed data on reporting. Although the vendor has not provided a resolution formally, we have been able to manually run some jobs.

Our current status is as follows:
RMS data is up to date
ILMS and Space data marts are updated to 1/8/25
Workday HCM and Finance data marts are updated to 1/9/25

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions.
Posted Jan 10, 2025 - 09:22 CST
Update
Microsoft continues to experience a service disruption in their Azure data center, per their status page at https://azure.status.microsoft/en-us/status. WashU IT has identified the resulting impact of delayed data on reporting. Although the vendor has not provided a resolution formally, we have been able to manually run some jobs, with another job scheduled to run on Friday morning. Additional updates will be posted ~10am on Friday morning (1/10/25) or when the status changes.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions.
Posted Jan 09, 2025 - 17:40 CST
Investigating
WashU IT is aware that reporting may be stale and reflecting yesterday's data.

The underlying cause is a large outage in Microsoft's Azure data center.

WashU IT is tracking the issue and will re-run jobs when Microsoft resolves the issue.

More information can be found at https://azure.status.microsoft/en-us/status
Posted Jan 09, 2025 - 09:27 CST
This incident affected: Administrative & Enterprise Systems (Cognos Reporting).