Washington University

Service Outage - Low Severity: Danforth Inbound E-Faxes

Update - WashU IT is investigating an issue with customers on the Danforth campus not getting inbound e-faxes. Outbound e-fax is working properly. Desktop Engineering and the Danforth Telecom team worked with the vendor and have another support call scheduled for tomorrow morning. The next update we send will be after 10am tomorrow.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 20, 16:17 CST

Update - WashU IT is investigating an issue with customers on the Danforth campus not getting inbound e-faxes. Outbound e-fax is working properly. Desktop Engineering is working with the Danforth Telecom team to engage vendors for support, and they are requesting a joint call to troubleshoot together.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 20, 13:11 CST

Investigating - WashU IT is investigating an issue with customers on the Danforth campus not getting inbound e-faxes. Outbound e-fax is working properly. Desktop Engineering is working with the Danforth Telecom team and our vendors to resolve the issue. At this time there has been one reported Incident related to this issue.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 20, 11:55 CST

Service Degradation (Vendor) - Low Severity: LAMPS

Update - Enterprise Applications is investigating an issue slow performance in the LAMPS platform. This is impacting various aspects of using the service.

The LAMPS site is currently experiencing loading and performance issues. The LAMPS vendor continues to troubleshoot and evaluate the environment, attempting to identify the root cause of our performance issues. We do not currently have an ETA on a resolution.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Dec 20, 10:12 CST

Update - Enterprise Applications is investigating an issue slow performance in the LAMPS platform. This is impacting various aspects of using the service. The LAMPS platform now appears to be completely offline. We assume this to be part of corrective actions by the vendor, but we will not be able to confirm with them until later today (Sunday 12/12).

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Dec 12, 07:51 CST

Update - Enterprise Applications is investigating an issue slow performance in the LAMPS platform. This is impacting various aspects of using the service. The vendor has scheduled an active monitoring session with the Enterprise Applications team for Monday morning, with the intention to observe system activity and try to further isolate the root cause of the degradation.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Dec 10, 13:37 CST

Update - Enterprise Applications is investigating an issue slow performance in the LAMPS platform. This is impacting various aspects of using the service. The vendor sent an update today stating that they have ruled out some possible root causes and continue to investigate the issue. There is a meeting between the vendor and Enterprise Applications team tomorrow (Thu 12/9) which will hopefully provide further direction on resolving this.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Dec 8, 23:09 CST

Update - Enterprise Applications is investigating an issue slow performance in the LAMPS platform. This is impacting various aspects of using the service. The vendor continues to investigate the situation, but we have not received an ETR. We anticipate sending another update after 10am tomorrow (Wednesday 12/8).

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Dec 7, 16:17 CST

Update - Enterprise Applications is investigating an issue slow performance in the LAMPS platform. This is impacting various aspects of using the service. The vendor is doing performance analytics and advised that they will be sending an update to us this afternoon.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Dec 7, 15:16 CST

Update - Enterprise Applications is investigating an issue slow performance in the LAMPS platform. This is impacting various aspects of using the service. The vendor is still investigating the issue and we have requested an update on their current status.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Dec 7, 10:39 CST

Update - Enterprise Applications is investigating an issue slow performance in the LAMPS platform. This is impacting various aspects of using the service. The vendor is still investigating the issue, but we do not anticipate further updates from them until after 10am on Tuesday 12/7.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Dec 6, 22:28 CST

Investigating - Enterprise Applications is investigating an issue slow performance in the LAMPS platform. This is impacting various aspects of using the service. This issue seems similar to recent events, but the vendor is currently investigating.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Dec 6, 10:38 CST

About This Site

WashU IT's Service Dashboard gives you real-time updates on system performance issues. If you require assistance, please call our Service Desk at 314-933-3333.

Uptime over the past 90 days. View historical uptime.

Administrative & Enterprise Systems Major Outage

90 days ago

100.0 % uptime

Today

Administrative Information System (AIS) Operational

Alumni and Development Information System (ADIS) Operational

Human Resources Management System (HRMS) Operational

Research Management System (RMS) Operational

Student Information System (SIS) Operational

Vended Applications Operational

Cognos Reporting Operational

Marketplace Operational

Vended Applications (Inc. LAMPS) Major Outage

Workday Operational

SUBS Awards Operational

Bear Bucks Operational

Back-end Processing (AutoSys) Operational

90 days ago

100.0 % uptime

Today

Clinical Operational

90 days ago

99.07 % uptime

Today

Enterprise Client Virtualization (ECV) Operational

EPIC Operational

Follow Me Desktop (FMD) Operational

Vended Applications Operational

Clinical Remote Access Operational

Clinical Websites Operational

Credit Card Systems Operational

90 days ago

99.07 % uptime

Today

Communications & Collaboration Operational

Email Operational

Office 365 Operational

Skype / Teams Operational

Maestro mass communication system Operational

Zoom Operational

Box File Storage System Operational

Computing and Printing Partial Outage

E-Fax Partial Outage

PaperCut Printing Operational

WashU Cloud Storage Operational

Azure Hosted Infrastructure Operational

WashU IT Private Cloud Computing Operational

IT Infrastructure Components Operational

JAMF Apple Device Management System Operational

File Services Operational

IT Management Operational

Software Licensing Operational

Network & Connectivity Operational

Internet Operational

Internet 2 Operational

Phone Operational

VPN Operational

WIFI Operational

Washington University Research Network (WURN) Operational

Vendor Services (Spectrum, AT&T, etc.) Operational

Network Services Operational

Research Operational

Compute Operational

Storage Operational

Security Operational

Identity Management System (IDM Wustl Key) Operational

Spam Filter Operational

Duo Two-Factor Authentication Operational

Teaching & Learning Operational

Canvas Operational

Learn@Work Operational

Kaltura Operational

Curriculum Management Applications Operational

EMS (Event Management System) Operational

University Websites Operational

WashU IT Hosted Websites Operational

Externally Hosted Websites Operational

BJC Devices and Services Operational

BJC Network Connectivity Operational

BJC Clinical Applications Operational

Pharmacy POS Machines Operational

BJC Phone System Operational

BJC Handheld Supply Scanners (ATPAR) Operational

Cloud Services Operational

Google Operational

Engineering School Operational

Operational

Degraded Performance

Partial Outage

Major Outage

Maintenance

Past Incidents

Jan 21, 2022

No incidents reported today.

Jan 20, 2022

Unresolved incident: Service Outage - Low Severity: Danforth Inbound E-Faxes.

Jan 19, 2022

No incidents reported.

Jan 18, 2022

Service Outage - High Severity: Papercut Printing Authentication Problem

Resolved - WashU IT Engineers have resolved the Papercut issue for the majority of affected printers and will handle any additional reported issues directly if they come up.

If you continue to experience further issues, please contact us at 314-933-3333. We appreciate your patience during this incident.
Jan 18, 19:33 CST

Update - WashU IT is investigating an issue with users being unable to release secure Papercut print jobs, including from web printing and from email to print. Direct print seems to be working ok but is not a practical workaround for all affected users. Technical teams are engaged in troubleshooting with an escalation engineer from the vendor and are making some progress in identifying and resolving the issue. Next update at 6pm.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 18, 17:01 CST

Update - WashU IT is investigating an issue with users being unable to release secure Papercut print jobs, including from web printing and from email to print. Direct print seems to be working ok but is not a practical workaround for all affected users. Technical teams are still engaged on a troubleshooting session and the vendor has joined that call, as well as escalating internally. Next update at 5pm.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 18, 16:01 CST

Update - WashU IT is investigating an issue with users being unable to release secure Papercut print jobs, including from web printing and from email to print. Direct print seems to be working ok but is not a practical workaround for all affected users. Technical teams are still engaged on a troubleshooting session and there is no ETR at this time. The next update will be around 4pm.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 18, 15:10 CST

Update - WashU IT is investigating an issue with users being unable to release secure Papercut print jobs, including from web printing and from email to print. Direct print seems to be working ok but is not a practical workaround for all affected users. Technical teams are engaged on a troubleshooting session and there is no ETR at this time. The next update will be around 3pm.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 18, 14:12 CST

Investigating - WashU IT is investigating an issue with Papercut secure printing. Users are unable to authenticate to Papercut devices such as multifunction printers. This also affects web printing and email to print. Direct print seems to be working ok but is not a practical workaround for all affected users.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 18, 12:32 CST

Service Outage - High Severity: Power Outage @ Rosedale Data Center

Resolved - WashU IT and Facilities teams have resolved the power outage and air conditioning issues at the Rosedale Data Center. We will continue to assist the small number of data center customers who are still working to restore their services, but we will not be sending additional updates. If you continue to experience further issues, please contact us at 314-933-3333.

We appreciate your patience during this incident.
Jan 18, 13:19 CST

Update - Power has been restored to the Rosedale Data Center and many data center customers have confirmed that their services are up and running. A small number of data center customers are still working on restoring their services. The next update will be around 1:30pm.

If you continue to experience further issues, please contact us at 314-933-3333. We appreciate your patience during this incident.
Jan 18, 12:05 CST

Update - Power has been restored to the Rosedale Data Center and many data center customers have confirmed that their services are up and running. Technical teams are monitoring the environment and the Major Incident will remain open while a small number of remaining customers work on restarting their equipment and restoring service. If you continue to experience further issues, please contact us at 314-933-3333.

We appreciate your patience during this incident.
Jan 18, 10:35 CST

Monitoring - Power has been restored to the Rosedale Data Center. Technical teams are monitoring the environment to ensure no further disruptions and that all services restarted correctly. If you continue to experience further issues, please contact us at 314-933-3333.

We appreciate your patience during this incident.
Jan 18, 09:09 CST

Update - WashU IT is investigating a power outage at the Rosedale Data Center after a server room air conditioning failure occurred on Monday night, 1/17. At this time power has been restored and we are evaluating if any services or servers need manual intervention to be restored.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 18, 08:27 CST

Investigating - WashU IT is investigating a power outage at the Rosedale Data Center.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.
Jan 18, 07:40 CST

Jan 17, 2022

No incidents reported.

Jan 16, 2022

No incidents reported.

Jan 15, 2022

No incidents reported.

Jan 14, 2022

No incidents reported.

Jan 13, 2022

Service Outage (Vendor) - Low Severity: Learn@Work Training Unavailable

Resolved - WashU IT identified an issue with certain content being unavailable within the Learn@Work system. This content is provided by Skillshare and was presumed to be unavailable due to a technical or configuration issue on their end. A support case was placed with Skillshare, and the content is once again available within Learn@Work.

Please call 314-933-3333 if you have any questions.
Jan 13, 11:00 CST

Jan 12, 2022

No incidents reported.

Jan 11, 2022

No incidents reported.

Jan 10, 2022

No incidents reported.

Jan 9, 2022

No incidents reported.

Jan 8, 2022

No incidents reported.

Jan 7, 2022

No incidents reported.

About This Site

Related