IT service updates
Checkbox survey access issue
Reduced service delivery during period
Currently there is an issue with Checkbox based surveys that require authentication. Using multiple browsers for authenticated surveys by the same user will cause an exception error.
ITS and the vendor are working on the issue. A temporary workaround is to clear the browser cookies or use the browser incognito mode / inPrivate window.
ITS and the vendor are working on the issue. A temporary workaround is to clear the browser cookies or use the browser incognito mode / inPrivate window.
Service affected: Checkbox
Start: 30 Oct 2024, 9:00 AM
End: Unknown
Login to request notification
UQ Search results may be incomplete
Reduced service delivery during period
UQ Search may be returning incorrect results, or partial results, or no results at all in some circumstances.
Microsoft and Bing search services are being looked at, and support staff are currently investigating.
Microsoft and Bing search services are being looked at, and support staff are currently investigating.
Update: 6 Sep 2024, 11:36 AM
Microsoft support engineers have been contacted and are looking into the issue.
Service affected: UQ Search
Start: 2 Sep 2024, 5:18 PM
End: Unknown
Login to request notification
Known issues with Microsoft Teams
Reduced service delivery during period
The following issue has been identified with Microsoft Teams:
Students are not found when using the New Chat button to start a new chat from the top of the Chat list in Teams. The issue has been reported to Microsoft. In the interim the following workaround can be used to initiate chats with students, using the Teams web client (Microsoft Teams) :
1) In the Search bar at the very top of the Teams window search for the students name or sNumber. If presented, click View all results
2) Select the People tab to view the search results.
3) Locate the students result. When hovering over the top right-hand corner of their record an icon with ...a diagonal arrow appears. This can be used to initiate a chat with the student.
After the student has been added to the chat, their chat will appear in the Chat list and now also in the results of the New Chat button
Students are not found when using the New Chat button to start a new chat from the top of the Chat list in Teams. The issue has been reported to Microsoft. In the interim the following workaround can be used to initiate chats with students, using the Teams web client (Microsoft Teams) :
1) In the Search bar at the very top of the Teams window search for the students name or sNumber. If presented, click View all results
2) Select the People tab to view the search results.
3) Locate the students result. When hovering over the top right-hand corner of their record an icon with ...a diagonal arrow appears. This can be used to initiate a chat with the student.
After the student has been added to the chat, their chat will appear in the Chat list and now also in the results of the New Chat button
Read more
Update: 19 Aug 2024, 3:15 PM
Reported issues with:
* Teams freezing when creating and renaming channels in succession
* Recent changes to the channels list not appearing
have been removed from the known issues list.
There have been no recent customer reports and no current issue can be reproduced. A temporary issue may have been resolved by Microsoft.
Update: 5 Aug 2024, 3:53 PM
The 401 Unauthorised error that was appearing when clicking on "Home Page" in some class Teams has been resolved by Microsoft. This issue has therefore been removed from the known issues list.
Service affected: Office 365 - Teams
Start: 8 Jul 2024, 8:57 AM
End: Unknown
Login to request notification
ECS - Prentice - Offline due to hardware failure.
No impact on service during period
ECS cloud pools on the Prentice ECS are not accessible from Isilon due to controller/PSU failure.
This will create errors for Isilion storage accessing Prentice ECS storage.
ITS are working with hardware vendors to resolve this asap.
This will create errors for Isilion storage accessing Prentice ECS storage.
ITS are working with hardware vendors to resolve this asap.
Update: 27 Sep 2024, 9:50 AM
Full data replication across CCR and Polaris ECS Cluster has been completed
Replacement ECS hardware at Prentice has been installed and XOR Data replication to the Prentice ECS cluster will begin the 1st week of October.
Update: 5 Jun 2024, 2:58 PM
ECS pools all online again and redundancy working
Dell Tec still need to replace PSU
Service no longer impacted
Service affected: ITS ECS S3 Object Store
Start: 5 Jun 2024, 10:00 AM
End: Unknown
Login to request notification
Some staff not showing in Academic Portal profiles
Reduced service delivery during period
A small number of academic staff are not loading correctly into the Profiles section of Academic Portal. This issue is preventing the affected staff from appearing on the Profiles page.
ITS staff are currently investigating.
ITS staff are currently investigating.
Service affected: Academic Portal
Start: 10 Apr 2024, 3:38 PM
End: Unknown
Login to request notification
Upgrade of Jira to version 9.12.13
Resolved
An upgrade of ITS on-premise version of Jira to version 9.12.13 will be conducted. This will provide a number of security and functionality improvements. Version 9.12.13 has been chosen as the upgrade target, as it is a modern LTS (long term support) Data Center version.
Resolution: 27 Oct 2024, 4:30 PM
Work has been completed successfully.
Jira is now accessible again.
Service affected: ITS Jira
Start: 27 Oct 2024, 9:00 AM
End: 27 Oct 2024, 4:30 PM
Login to request notification
DC1 Power Outage - RDM Services IMpacted
Resolved
Due to a generator failure, DC1 lost power at 8:25AM.
Power is restored, and RDM Services are being recovered. There are disk failures on Dell ME Storage being addressed.
No ETA on full service recovery yet.
Power is restored, and RDM Services are being recovered. There are disk failures on Dell ME Storage being addressed.
No ETA on full service recovery yet.
Resolution: 27 Oct 2024, 4:47 PM
RDM Virtual hosts migrated and restarted
Update: 27 Oct 2024, 2:41 PM
Access to data through UQ RDM has been restored. Cloud.rdm.uq.edu.au and other shares are available.
Minor service degradation still being work through under P2
Service affected: UQRDM Storage Service
Start: 27 Oct 2024, 8:25 AM
End: 27 Oct 2024, 4:47 PM
Login to request notification
Network Outage to UQ Remote sites and UQSchoolsNet sites
Resolved
A Data Centre power outage is currently affecting the network at some UQ Remote sites and UQ SchoolsNet sites.
Resolution: 27 Oct 2024, 2:39 PM
Power and Services restored
RDM Service degredation still, and a P2.
Seperate service status notice for this
Service affected: Networks
Start: 27 Oct 2024, 8:20 AM
End: 27 Oct 2024, 2:39 PM
Login to request notification
Network shutdown - St Lucia CAI Level 2
Resolved
There will no network service on CAI level 2 while the network is upgraded. For further information, please call the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 27 Oct 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 27 Oct 2024, 7:00 AM
End: 27 Oct 2024, 10:00 PM
Login to request notification
Exadata Cloud @ Customer Patching - Production databases
Resolved
NOTES FOR CAB: This CR has been entered as a significant change for the purposes of seeking pre-approval going forward.
Patch the PRN Exadata Cloud @ Customer with the latest quarterly Oracle Critical Patch Updates
Previous pre-approved change process for on-premises Exadata patching : 10437,10439,10440,10441 and 10442 will be redundant going forward due to changes in the patching processes.
Patch the PRN Exadata Cloud @ Customer with the latest quarterly Oracle Critical Patch Updates
Previous pre-approved change process for on-premises Exadata patching : 10437,10439,10440,10441 and 10442 will be redundant going forward due to changes in the patching processes.
Resolution: 27 Oct 2024, 7:00 PM
Automatic resolution.
Service affected: Exadata Cloud @ Customer Production
Start: 27 Oct 2024, 6:00 AM
End: 27 Oct 2024, 7:00 PM
Login to request notification
Various, intermittent failures with Microsoft Teams
Resolved
An unscheduled service functionality issue is currently impacting the M365 Teams service.
Impact may be occurring in the following scenarios:
- Sending messages
- Creating new group chats
- Creating new meetings
- Updating meetings
- Bots ability to post messages
- Any Microsoft Teams Graph API which supports messages
- Creating Virtual Events such as Webinars
- Escalating 1-1 calls
Next update: 1:00pm
Impact may be occurring in the following scenarios:
- Sending messages
- Creating new group chats
- Creating new meetings
- Updating meetings
- Bots ability to post messages
- Any Microsoft Teams Graph API which supports messages
- Creating Virtual Events such as Webinars
- Escalating 1-1 calls
Next update: 1:00pm
Resolution: 25 Oct 2024, 1:03 PM
Preliminary root cause:
A routine change initiated by Microsoft experienced a deployment issue, which led to this service impact.
Microsoft will provide UQ with a preliminary Post-Incident Report within two business days and a final Post-Incident Report within five business days.
Service affected: Office 365 - Teams
Start: 25 Oct 2024, 8:57 AM
End: 25 Oct 2024, 1:03 PM
Login to request notification
Replication issues with MySQL databases
Resolved
The DBA team are investigating reports of issues resulting from an unscheduled failover of MySQL services from the Prentice to CCR data centres which occured at 12:15AM this morning.
Transactions for VPNext and hostinfo have been replayed
Other DB are being checked for consistency
Transactions for VPNext and hostinfo have been replayed
Other DB are being checked for consistency
Resolution: 26 Oct 2024, 9:25 AM
Data and databases have been restored and recovered.
The mysqlprod service is now restored.
Update: 26 Oct 2024, 6:09 AM
Data has been restored and recovered.
ITS are awaiting confirmation / validation from stakeholders before closing this incident
Update: 25 Oct 2024, 12:11 PM
Data restoration activities are underway.
No data loss is expected once that is complete
Service affected: MySQL General Purpose and Web Database Servers
Start: 25 Oct 2024, 12:15 AM
End: 26 Oct 2024, 9:25 AM
Login to request notification
Weekly Production Migration for SI-net
Resolved
Weekly Migration to Production.
Refer to page below for a full list of changes or contact eaps@uq.edu.au for more information.
https://confluence.its.uq.edu.au/confluence/display/ESS/2024-10-23+Weekly+Production+Changes+-+EA+Change+and+Release
Refer to page below for a full list of changes or contact eaps@uq.edu.au for more information.
https://confluence.its.uq.edu.au/confluence/display/ESS/2024-10-23+Weekly+Production+Changes+-+EA+Change+and+Release
Resolution: 24 Oct 2024, 7:30 AM
Automatic resolution.
Service affected: SI-net
Start: 24 Oct 2024, 7:00 AM
End: 24 Oct 2024, 7:30 AM
Login to request notification
Hazardous Work - Replacing APIC L2 servers with L4 servers at Prentice NGDC
Resolved
Replacing 3 x APIC L2 servers with 3 x APIC L4 servers at Prentice NGDC.
And upgrade APIC L4 Leaf connectivity to 25Gig
Note: This is a management appliance change, not a networking switch change.
And upgrade APIC L4 Leaf connectivity to 25Gig
Note: This is a management appliance change, not a networking switch change.
Resolution: 20 Oct 2024, 5:00 PM
Automatic resolution.
Service affected: NGDC Serverfarm
Start: 20 Oct 2024, 9:00 AM
End: 20 Oct 2024, 5:00 PM
Login to request notification
Connection Issues with SInet
Resolved
Please be advised that ITS are investigation connection issues in SInet resulting in logouts or connection errors.
Resolution: 18 Oct 2024, 1:12 PM
Issue resolved at 1:12 PM.
Service affected: SI-net
Start: 18 Oct 2024, 11:39 AM
End: 18 Oct 2024, 1:12 PM
Login to request notification
Users are Intermittently Being Kicked Out of SI-net
Resolved
Users may be able to logon to SI-net, but get kicked out while accessing the application. They may be able to re-logon again.
Resolution: 18 Oct 2024, 1:12 PM
Automatic resolution.
Update: 18 Oct 2024, 1:12 PM
Issue resolved at 1:12 PM.
Service affected: SI-net
Start: 18 Oct 2024, 9:00 AM
End: 18 Oct 2024, 1:12 PM
Login to request notification
ePAD and MyProgress currently unavailable
Resolved
From approximately 11 AM on 16 October 2024, ePAD and MyProgress became unavailable due to an invalid certificate. The vendor has been notified and is working to resolve the issue. Access will be restored once the certificate is updated.
Resolution: 16 Oct 2024, 9:15 PM
The vendor has fixed the invalid certificate and service has been restored.
Service affected: MyKnowledgeMap-MyProgress
Start: 16 Oct 2024, 11:00 AM
End: 16 Oct 2024, 9:15 PM
Login to request notification
Network shutdown - St Lucia CAI Level 4
Resolved
There will no network service on level 4 of the CAI (0057) building during a network upgrade. For more information, please contact the Project Manager, Keryn Masters on 0458 585 405
Resolution: 13 Oct 2024, 6:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 12 Oct 2024, 7:00 AM
End: 13 Oct 2024, 6:00 PM
Login to request notification
Upgrade of Confluence to version 8.5.16 (Patch Critical Vulnerabilities)
Resolved
As per the Atlassian September 2024 Security Bulletin, an urgent update needs to be applied to patch these critical vulnerabilities.
Resolution: 11 Oct 2024, 7:00 PM
Automatic resolution.
Service affected: ITS Confluence
Start: 11 Oct 2024, 6:00 PM
End: 11 Oct 2024, 7:00 PM
Login to request notification
Bunya HPC storage disruption
Resolved
Storage for the Bunya HPC system is currently experiencing an unscheduled disruption. This is preventing new users from logging in and queued jobs from starting. UQ RCC is working to resolve this issue as a matter of priority.
Resolution: 11 Oct 2024, 10:37 AM
Services have now been restored.
New users can now log back in.
Current jobs are still running and queued jobs will be restored.
Service affected: Bunya HPC
Start: 11 Oct 2024, 10:09 AM
End: 11 Oct 2024, 10:37 AM
Login to request notification
Errors when accessing ePortfolio links in courses
Resolved
There is currently an issue impacting users ability to access ePortfolio through LTI links in courses, with issues starting around 8:30 AM, 11 October 2024. The vendor has been notified and is investigating the issue. In the meantime, users can access ePortfolio directly by navigating there.
For staff please refer to https://userguide.chalkandwire.com/m/Assessor/l/203107-log-in-to-your-account
For students please refer to https://elearning.uq.edu.au/student-guides-original/eportfolio-original/create-my-eportfolio-original
For staff please refer to https://userguide.chalkandwire.com/m/Assessor/l/203107-log-in-to-your-account
For students please refer to https://elearning.uq.edu.au/student-guides-original/eportfolio-original/create-my-eportfolio-original
Resolution: 11 Oct 2024, 11:50 PM
The vendor implemented a fix during Friday night at 23:50
Service affected: Chalk and Wire ePortfolio
Start: 11 Oct 2024, 8:30 AM
End: 11 Oct 2024, 11:50 PM
Login to request notification
Chalk and Wire (ePortfolio) service disruption
Resolved
We are currently experiencing an issue with accessing ePortfolio through LTI links within courses, which may also impact grade syncing with the grade centre. The vendor has been notified and is investigating the issue. In the meantime, users can access ePortfolio directly.
For staff please refer to https://userguide.chalkandwire.com/m/Assessor/l/203107-log-in-to-your-account
For students please refer to
https://elearning.uq.edu.au/student-guides-original/eportfolio-original/create-my-eportfolio-original
For staff please refer to https://userguide.chalkandwire.com/m/Assessor/l/203107-log-in-to-your-account
For students please refer to
https://elearning.uq.edu.au/student-guides-original/eportfolio-original/create-my-eportfolio-original
Resolution: 10 Oct 2024, 8:39 AM
The LTI vendor has advised that a fix has been put in place to resolve the cause of this issue with stakeholder confirmation carried out.
Service affected: Chalk and Wire ePortfolio
Start: 9 Oct 2024, 1:00 PM
End: 10 Oct 2024, 8:39 AM
Login to request notification
Users are unable to complete setup for iOS devices
Resolved
There is currently an issue with Microsoft Intune where users are unable to complete the setup process for newly purchased Apple iOS devices.
ITS engineers are investigating the issue.
ITS engineers are investigating the issue.
Resolution: 8 Oct 2024, 11:05 AM
MDM Push Certificate for Intune had expired.
Service affected: Intune
Start: 8 Oct 2024, 8:54 AM
End: 8 Oct 2024, 11:05 AM
Login to request notification
Storage Upgrade
Resolved
Resolution: 6 Oct 2024, 9:05 AM
Automatic resolution.
Service affected: UQRDM Storage Service
Start: 5 Oct 2024, 8:05 AM
End: 6 Oct 2024, 9:05 AM
Login to request notification
Network Upgrade - CAI Level 5
Resolved
There will be no network service on CAI level 5 during the upgrade. For further information, please call the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 5 Oct 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 5 Oct 2024, 7:00 AM
End: 5 Oct 2024, 10:00 PM
Login to request notification
SInet Login Error
Resolved
We are investigation login issues currently being experienced with SInet. After login users may be taken to an error page.
Resolution: 3 Oct 2024, 10:59 AM
The login error issue experienced has now been resolved. Apologies for the inconvenience.
Resolution: 3 Oct 2024, 10:59 AM
The login issue experienced has now been resolved. Apologies for the inconvenience.
Resolution: 3 Oct 2024, 10:59 AM
The login issue experienced has now been resolved.
Service affected: SI-net
Start: 3 Oct 2024, 9:41 AM
End: 3 Oct 2024, 10:59 AM
Login to request notification
Unscheduled outage - SCCM
Resolved
There is currently an unscheduled outage for SCCM.
Clients will be unable to download software from the Software Center during this time.
ITS technicians are currently investigating the issue.
Clients will be unable to download software from the Software Center during this time.
ITS technicians are currently investigating the issue.
Resolution: 3 Oct 2024, 8:42 AM
There was some connectivity issues between SCCM and the SQL databases which has now been resolved.
Service affected: System Center Configuration Manager
Start: 3 Oct 2024, 8:42 AM
End: 3 Oct 2024, 8:42 AM
Login to request notification
ACI Upgrade - APIC and Leaf Switch Upgrade/Replacement in the CCR Next Gen DC - Stage 1
Resolved
Multi-day change for a software and hardware refresh for the CCR ACI DataCentre.
Additional work will be required for the final software update to complete the change. That is under a separate CR.
Milestones Summary-
2nd Wednesday - 9pm to 3am Thursday
MILESTONE 0.1 - Upgrade APIC to 4.2(7v)
MILESTONE 0.2 - Switch Upgrade to 14.2(7v)
MILESTONE 0.3 - Change Post Checks
4th Friday - 9pm to 3am Saturday
MILESTONE 1.1 - Upgrade APIC to 5.2(7f)
MILESTONE 1.2 - Switch Upgrade to 15.2(7f)
MILESTONE 1.3 - Change Post Checks
5th Saturday - 2pm to 12am
MILESTONE 1.4 - Switch Pair ...Replacement (CCRDC-A9-1 and CCRDC-A9-2)
MILESTONE 1.5 - Switch Pair Replacement (CCRDC-A10-1 and CCRDC-A10-2)
MILESTONE 1.6 - Switch Pair Replacement (CCRDC-A11-1 and CCRDC-A11-2)
MILESTONE 1.7 - Switch Pair Replacement (CCRDC-B11-1 and CCRDC-B11-2)
6th Sunday - 12pm to 11pm
MILESTONE 1.8 - Spine Replacement (CCRDC-A12-9)
MILESTONE 1.9 - Spine Replacement (CCRDC-B12-9)
MILESTONE 2.0 - Border Leaf Pair Replacement (CCRDC-A12-2 and CCRDC-B12-2)
MILESTONE 2.1 - Network Leaf Pair Replacement (CCRDC-A12-1 and CCRDC-B12-1)
Overall ACI Upgrade Project Scope & Schedule - https://confluence.its.uq.edu.au/confluence/display/ITSNDC/ACI+Upgrade+Project+2023-2024
Additional work will be required for the final software update to complete the change. That is under a separate CR.
Milestones Summary-
2nd Wednesday - 9pm to 3am Thursday
MILESTONE 0.1 - Upgrade APIC to 4.2(7v)
MILESTONE 0.2 - Switch Upgrade to 14.2(7v)
MILESTONE 0.3 - Change Post Checks
4th Friday - 9pm to 3am Saturday
MILESTONE 1.1 - Upgrade APIC to 5.2(7f)
MILESTONE 1.2 - Switch Upgrade to 15.2(7f)
MILESTONE 1.3 - Change Post Checks
5th Saturday - 2pm to 12am
MILESTONE 1.4 - Switch Pair ...Replacement (CCRDC-A9-1 and CCRDC-A9-2)
MILESTONE 1.5 - Switch Pair Replacement (CCRDC-A10-1 and CCRDC-A10-2)
MILESTONE 1.6 - Switch Pair Replacement (CCRDC-A11-1 and CCRDC-A11-2)
MILESTONE 1.7 - Switch Pair Replacement (CCRDC-B11-1 and CCRDC-B11-2)
6th Sunday - 12pm to 11pm
MILESTONE 1.8 - Spine Replacement (CCRDC-A12-9)
MILESTONE 1.9 - Spine Replacement (CCRDC-B12-9)
MILESTONE 2.0 - Border Leaf Pair Replacement (CCRDC-A12-2 and CCRDC-B12-2)
MILESTONE 2.1 - Network Leaf Pair Replacement (CCRDC-A12-1 and CCRDC-B12-1)
Overall ACI Upgrade Project Scope & Schedule - https://confluence.its.uq.edu.au/confluence/display/ITSNDC/ACI+Upgrade+Project+2023-2024
Read more
Resolution: 6 Oct 2024, 11:00 PM
Automatic resolution.
Service affected: NGDC Serverfarm
Start: 2 Oct 2024, 9:00 PM
End: 6 Oct 2024, 11:00 PM
Login to request notification
Access Project: Gehrmann levels 6 and 7
Resolved
Network upgrade and outage, please avoid the area if possible. For further information, please contact the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 29 Sep 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 29 Sep 2024, 7:00 AM
End: 29 Sep 2024, 10:00 PM
Login to request notification
Aussie Broadband users experiencing authentication issues connecting to UQ
Resolved
Aussie Broadband RSP users are experiencing authentication issues connecting to UQ Services including VPN.
Other service providers are not impacted at this time
Other service providers are not impacted at this time
Resolution: 27 Sep 2024, 9:45 AM
This has been resolved by the ISP.
Service affected: UQ Authenticate
Start: 27 Sep 2024, 7:00 AM
End: 27 Sep 2024, 9:45 AM
Login to request notification
UniTask access (role assignments) incomplete
Resolved
An attempt to switch over to Workday data from our data provider was not fully successful. As a result, we are not receiving all role assignments for staff members.
We have requested a rollback of the switch and restoration of roles base on Aurion data.
We expect this to be completed before COB today.
We have requested a rollback of the switch and restoration of roles base on Aurion data.
We expect this to be completed before COB today.
Resolution: 26 Sep 2024, 5:26 PM
Aurion-based staff roles and access has been restored.
Service affected: Unitask
Start: 26 Sep 2024, 1:00 PM
End: 26 Sep 2024, 5:26 PM
Login to request notification
Aurion not available
Resolved
Aurion is currently unavailable due to overrun of the overnight processing.
Resolution: 27 Sep 2024, 7:15 AM
Aurion overnight completed and services were restored at approximately 3.30 PM yesterday
Service affected: Aurion
Start: 26 Sep 2024, 5:00 AM
End: 27 Sep 2024, 7:15 AM
Login to request notification
Network upgrade - CAI level 2 - selected devices
Resolved
A number of CAI devices/instruments will be offline during this upgrade. For further information, please contact the Project Manager on 0458 585 405.
Resolution: 22 Sep 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 22 Sep 2024, 7:00 AM
End: 22 Sep 2024, 10:00 PM
Login to request notification
Scratch filesystem on bunya unavailable - job failures
Resolved
RCC are aware of an issue with the high performance storage sub-system that provides access to /home /scratch and /sw and $TMPDIR. Clients may experience: * very slow response to commands, * stale filehandle error messages * job failures in certain circumstances RCC staff are working with the backline vendor to resolve the issue as soon as possible and return all nodes to regular operations. Once normal operations resume, clients should be encouraged to check the status of their jobs and re-submit if required.
Resolution: 16 Sep 2024, 7:11 PM
Issues with Scratch storage have been resolved and the system is now operational.
Update: 16 Sep 2024, 3:50 PM
RCC is continuing to work with the vendor to address the underlying cause of the outage.
An estimated return to service time is not yet available.
Service affected: Bunya HPC
Start: 16 Sep 2024, 1:02 PM
End: 16 Sep 2024, 7:11 PM
Login to request notification
Network upgrade - Gehrmann and CAI
Resolved
Network upgrade and outage in Gehrmann 523A, 227B and CAI 107, 425 and 527. For further information, please call the Project Manager on 0458 585 405.
Resolution: 15 Sep 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 15 Sep 2024, 7:00 AM
End: 15 Sep 2024, 10:00 PM
Login to request notification
Intermittent timeouts and slow response
Resolved
The UQ Org website is intermittently timing out and is generally slow to respond.
Resolution: 13 Sep 2024, 2:30 PM
Performance limit reached on a shared file system cache due to load on the system. Increased the capacity of EFS and throughput has improved considerably.
Service affected: UQ Organisational Database
Start: 13 Sep 2024, 11:00 AM
End: 13 Sep 2024, 2:30 PM
Login to request notification
Intermittent timeouts and slow response
Resolved
The Programs & Courses website is intermittently timing out and is generally slow to respond.
Resolution: 16 Sep 2024, 10:30 AM
Response times improved by adding additional memory and by moving the location of the cache to local disk storage.
Service affected: Programs & Courses
Start: 13 Sep 2024, 10:00 AM
End: 16 Sep 2024, 10:30 AM
Login to request notification
SharePoint breadcrumbs site navigation temporarily unavailable
Resolved
A key rotation in a recent update is taking longer than expected to propogate causing requests to become unauthorised.
The breadcrumb, page hierarchy, and side navigation components will contain unauthorised errors during this time.
The breadcrumb, page hierarchy, and side navigation components will contain unauthorised errors during this time.
Resolution: 10 Sep 2024, 2:56 AM
Authentication issue with identity provider resolved
Service affected: Office 365 - SharePoint
Start: 10 Sep 2024, 1:56 PM
End: 10 Sep 2024, 2:56 AM
Login to request notification
Si-Net is currently not accessible
Resolved
Si-Net is currently not accessible.
Issue is under investigation.
Issue is under investigation.
Resolution: 9 Sep 2024, 3:00 PM
Services restarted and working as expected.
Investigation into the root cause is ongoing and a PIR will be scheduled.
Resolution: 9 Sep 2024, 3:00 PM
Service affected: SI-net
Start: 9 Sep 2024, 9:30 AM
End: 9 Sep 2024, 3:00 PM
Login to request notification
Network upgrade and outage - Gehrmann lvls 2,3 and 8
Resolved
There will be no network services on level 2, part of level 3 and level 8 during this upgrade. For further information, please contact the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 8 Sep 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 8 Sep 2024, 10:00 AM
End: 8 Sep 2024, 10:00 PM
Login to request notification
QBP (Building 80) - Wired/Wireless Network Outage
Resolved
A power outage in the QBP building is causing a wireless/wired network outage.
Resolution: 5 Sep 2024, 2:07 PM
Service affected: Building Networks
Start: 3 Sep 2024, 6:00 AM
End: 5 Sep 2024, 2:07 PM
Login to request notification
SharePoint breadcrumbs site navigation unavailable
Resolved
The certificate for the SharePoint intranet based site breadcrumbs application has expired. This is affecting the site navigation via the breadcrumb web part. ITS is currently working on a replacement certificate.
Resolution: 2 Sep 2024, 4:05 PM
Manual update of backend certificates has been performed across the affected environments. Documentation and a recurring renewal schedule is being prepared to prevent this in future.
Service affected: Office 365 - SharePoint
Start: 29 Aug 2024, 1:23 PM
End: 2 Sep 2024, 4:05 PM
Login to request notification
UQ Extend intermittent errors accessing content
Resolved
ITS is aware of reports of unexpected errors for users when accessing content on UQ Extend. Efforts are currently underway to address the cause of this issue.
Resolution: 29 Aug 2024, 8:41 AM
Vendor has advised that a bug has been identified in the software causing issues with stale user session data. A patch to address this is being developed to submit upstream whilst mitigations are in place to prevent further occurrence.
Resolution: 29 Aug 2024, 8:41 AM
Automatic resolution.
Update: 29 Aug 2024, 8:41 AM
Our Vendor has advised that a fix is now in place and our testing has confirmed this. Additional monitoring & review is now underway to guard against further recurrence.
Update: 28 Aug 2024, 4:30 PM
Engineers from the support vendor have narrowed down the root cause of the issue, and are continuing to work on a resolution.
Service affected: UQ Extend Learning Activity Management System
Start: 27 Aug 2024, 5:42 PM
End: 29 Aug 2024, 8:41 AM
Login to request notification
504 errors across multiple websites
Resolved
Several websites are reporting 504 errors on some pages.
ITS are working with the service provider to resolve.
ITS are working with the service provider to resolve.
Resolution: 28 Aug 2024, 5:30 AM
The service provider has resolved the issue.
Resolution: 28 Aug 2024, 5:30 AM
Automatic resolution.
Update: 27 Aug 2024, 5:18 PM
Also impacted: Course profiles are not being updated, and course profile previews within JAC are not functioning.
ITS is continuing to work with the vendor to resolve this issue.
Service affected: CMS - ITS Drupal
Start: 27 Aug 2024, 10:15 AM
End: 28 Aug 2024, 5:30 AM
Login to request notification
Migrate HUBPROD Production database to new hardware
Resolved
Migrate HUBPROD Production database to new Exatada hardware.
Resolution: 27 Aug 2024, 9:00 AM
Automatic resolution.
Service affected: UQ Data Hub
Start: 27 Aug 2024, 6:00 AM
End: 27 Aug 2024, 9:00 AM
Login to request notification
Internet access problems for some wired devices
Resolved
Some wired devices are reported as being unable to access the internet. Wireless is not impacted.
WORKAROUND for this is to enable VPN
This is under investigation by ITS.
WORKAROUND for this is to enable VPN
This is under investigation by ITS.
Resolution: 27 Aug 2024, 11:01 AM
This has been resolved.
Update: 27 Aug 2024, 10:12 AM
This has been resolved
Service affected: Internet Management Service
Start: 26 Aug 2024, 3:49 PM
End: 27 Aug 2024, 11:01 AM
Login to request notification
Course Publishing Failing
Resolved
Jac course publishing are currently failing. We are currently investigation.
Resolution: 27 Aug 2024, 8:14 AM
This has been resolved. Network connectivity issue has been bypassed.
Service affected: Jac
Start: 26 Aug 2024, 12:05 PM
End: 27 Aug 2024, 8:14 AM
Login to request notification
Prentice DC Load Balancer intermittent Issues
Resolved
There is currently an issue affecting the Prentice Load balancer services.
These services have been failed across to the CCR Datacentre while ITS and Vendor engineers actively investigate intermittent issues in the Prentice DC.
These services have been failed across to the CCR Datacentre while ITS and Vendor engineers actively investigate intermittent issues in the Prentice DC.
Resolution: 26 Aug 2024, 9:00 PM
Service affected: NGDC Serverfarm Load Balancer
Start: 26 Aug 2024, 11:30 AM
End: 26 Aug 2024, 9:00 PM
Login to request notification
Archive ECP Students is offline
Resolved
The Archived Electronic Course Profile Systems Student interface is offline.
ITS Staff are investigating.
As a note, the course profiles shifted from ECP to Drupal & JAC for Semester 2, 2024, and unaffected. This is for archived ones only
ITS Staff are investigating.
As a note, the course profiles shifted from ECP to Drupal & JAC for Semester 2, 2024, and unaffected. This is for archived ones only
Resolution: 24 Aug 2024, 8:02 PM
ECP Students is now back up and running.
Service affected: Electronic Course Profile System - Student Interface
Start: 24 Aug 2024, 12:04 PM
End: 24 Aug 2024, 8:02 PM
Login to request notification
APIC Upgrade and Leaf Switch Upgrade/Replacement in the Prentice Next Gen DC - Stage 2
Resolved
The Prentice NGDC Server Farm will be undergoing Scheduled maintenance,
This is Stage 2 - ACI Network leaf and Spine replacements and APIC/Switch upgrade to final version.
Scheduled start: Sat, 24 Aug 2024 12PM (midday)
Scheduled end: Sun, 25 Aug 2024 8PM
Please note the below Impact Statement:
SATURDAY 24/8/2024 9PM - For up to one minute, there will a disruption of network traffic inbound and outbound of the Prentice NGDC Data Centre. Mitigations will be in place to maintain core I.T. services, however this will impact access to a large number of service hosted in the Data Centre.
Schedule Overview -
Saturday 2...4/8/2024 - 12:00PM to 10:00PM
Sunday, 25/8/2024 - 10:00am to 5:00pm
Project Scope/Schedule - https://confluence.its.uq.edu.au/confluence/display/ITSNDC/ACI+Upgrade+Project+2023-2024
https://confluence.its.uq.edu.au/confluence/pages/viewpage.action?pageId=367465465
This is Stage 2 - ACI Network leaf and Spine replacements and APIC/Switch upgrade to final version.
Scheduled start: Sat, 24 Aug 2024 12PM (midday)
Scheduled end: Sun, 25 Aug 2024 8PM
Please note the below Impact Statement:
SATURDAY 24/8/2024 9PM - For up to one minute, there will a disruption of network traffic inbound and outbound of the Prentice NGDC Data Centre. Mitigations will be in place to maintain core I.T. services, however this will impact access to a large number of service hosted in the Data Centre.
Schedule Overview -
Saturday 2...4/8/2024 - 12:00PM to 10:00PM
Sunday, 25/8/2024 - 10:00am to 5:00pm
Project Scope/Schedule - https://confluence.its.uq.edu.au/confluence/display/ITSNDC/ACI+Upgrade+Project+2023-2024
https://confluence.its.uq.edu.au/confluence/pages/viewpage.action?pageId=367465465
Read more
Resolution: 25 Aug 2024, 8:00 PM
Automatic resolution.
Update: 24 Aug 2024, 11:56 PM
Issues have been identified with F5 loadbalanced services in Prentice DC.
Services have been failed over to CCR DC whilst root cause is invesitgated.
Update: 23 Aug 2024, 2:31 PM
The Prentice NGDC Server Farm will be undergoing Scheduled maintenance,
This is Stage 2 - ACI Network leaf and Spine replacements and APIC/Switch upgrade to final version.
Scheduled start: Sat, 24 Aug 2024 12PM (midday)
Scheduled end: Sun, 25 Aug 2024 8PM
Please note the below Impact Statement:
SATURDAY 24/8/2024 9PM - For up to one minute, there will a disruption of network traffic inbound and outbound of the Prentice NGDC Data Centre. Mitigations will be in place to maintain core I.T. services, however this will impact access to a large number of service hosted in the Data Centre.
Schedule Overview -
Saturday 24/8/2024 - 12:00PM to 10:00PM
Sunday, 25/8/2024 - 10:00am to 5:00pm
Service affected: NGDC Serverfarm
Start: 24 Aug 2024, 12:00 PM
End: 25 Aug 2024, 8:00 PM
Login to request notification
Migrate SSSDMP Production databases to new hardware
Resolved
Migrate SSSDMP Production databases to new Exatada hardware.
----------------------------
This has been moved ahead to 8:00 am on 24th Aug (after discussion with stakeholders) because of another change - Prentice ACI Upgrade - Stage 2 - 23rd/24th August CR14043
----------------------------
This has been moved ahead to 8:00 am on 24th Aug (after discussion with stakeholders) because of another change - Prentice ACI Upgrade - Stage 2 - 23rd/24th August CR14043
Resolution: 24 Aug 2024, 10:00 AM
Automatic resolution.
Service affected: Student Services Student Management System
Start: 24 Aug 2024, 8:00 AM
End: 24 Aug 2024, 10:00 AM
Login to request notification
Portal My.UQ is offline
Resolved
Portal My.UQ is offline.
ITS Staff are investigating.
ITS Staff are investigating.
Resolution: 23 Aug 2024, 7:21 PM
Portal My.UQ has been recovered, and the website is back online.
Service affected: Portal.MyUQ
Start: 23 Aug 2024, 5:46 PM
End: 23 Aug 2024, 7:21 PM
Login to request notification
Migrate SPOKE and SYNC Production databases to new hardware
Resolved
Migrate SPOKE and SYNC Production databases to new Exatada hardware.
Resolution: 22 Aug 2024, 9:00 AM
Automatic resolution.
Service affected: Oracle Database Administration
Start: 22 Aug 2024, 6:00 AM
End: 22 Aug 2024, 9:00 AM
Login to request notification
Linux Routine OS Patching - connection manager hosts & OMS1
Resolved
Linux Routine OS Patching - cloud-conman2 , conman2 and OMS1
(Please refer Impact Section for further info)
(Please refer Impact Section for further info)
Resolution: 21 Aug 2024, 9:20 AM
Automatic resolution.
Service affected: Oracle Development DB Hosting
Start: 21 Aug 2024, 9:00 AM
End: 21 Aug 2024, 9:20 AM
Login to request notification
Restart listener on magpiedb01
Resolved
Restart LISTENER_SCAN1 on magpiedb01
Resolution: 19 Aug 2024, 5:10 PM
Automatic resolution.
Service affected: Oracle Database Administration
Start: 19 Aug 2024, 5:00 PM
End: 19 Aug 2024, 5:10 PM
Login to request notification
Weekly Production Migration for SI-net
Resolved
Weekly Migration to Production.
Resolution: 19 Aug 2024, 9:00 AM
Automatic resolution.
Service affected: SI-net
Start: 19 Aug 2024, 8:45 AM
End: 19 Aug 2024, 9:00 AM
Login to request notification
MyAurion intermittent
Resolved
There is an issue on MyAurion which is causing it to be intermittently accessible.
Resolution: 19 Aug 2024, 12:16 PM
Service affected: MyAurion
Start: 19 Aug 2024, 7:00 AM
End: 19 Aug 2024, 12:16 PM
Login to request notification
Network Upgrade - Gehrmann levels 4 and 2
Resolved
Network unavailable during the upgrade. For further information please call the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 18 Aug 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 18 Aug 2024, 7:00 AM
End: 18 Aug 2024, 10:00 PM
Login to request notification
Migrate UniFi Production databases to new hardware
Resolved
Migrate UniFi Production databases to new Exatada hardware
Resolution: 17 Aug 2024, 11:00 AM
Automatic resolution.
Service affected: Oracle Production DB Hosting
Start: 17 Aug 2024, 5:00 AM
End: 17 Aug 2024, 11:00 AM
Login to request notification
UQ Extend Scheduled Maintenance
Resolved
UQ Extend will be unavailable for a period of up to 1 hour between this window while technicians perform scheduled technical maintenance.
Resolution: 16 Aug 2024, 11:59 PM
Automatic resolution.
Service affected: UQ Extend Learning Activity Management System
Start: 16 Aug 2024, 9:00 PM
End: 16 Aug 2024, 11:59 PM
Login to request notification
APIC Upgrade and Leaf Switch Upgrade/Replacement in the Prentice Next Gen DC - Stage 1
Resolved
Stage 1 - APIC Upgrade and Leaf Switch replacements Server Racks and Border Rack
Friday 16th Aug 6PM to 12PM
a) APIC Software Upgrade will start on 06:00 PM on Fri 16th Aug and expected to be completed by
9:00PM
b) Gen2 Leaf switch upgrade will commence on 9PM on Fri 16th Aug. These will be done in two groups
of leaf switches, which may likely take 2-3 hours.
Saturday 17th Aug after 4pm to 12AM (Starts after payrun is completed)
c) 2 RACKS - Pair of Gen 1 Leaf Switches are to be replaced with pair of new Gen 3 switches
Sunday 18th Aug 10AM to 10PM
d) 2 RACKS - ...Pair of Gen 1 Leaf Switches are to be replaced with pair of new Gen 3 switches
e) 1 Border Rack - Pair of Gen 1 Border Leaf Switches are to be replaced with pair of new Gen 3 switches
Project Scope/Schedule - https://confluence.its.uq.edu.au/confluence/display/ITSNDC/ACI+Upgrade+Project+2023-2024
Friday 16th Aug 6PM to 12PM
a) APIC Software Upgrade will start on 06:00 PM on Fri 16th Aug and expected to be completed by
9:00PM
b) Gen2 Leaf switch upgrade will commence on 9PM on Fri 16th Aug. These will be done in two groups
of leaf switches, which may likely take 2-3 hours.
Saturday 17th Aug after 4pm to 12AM (Starts after payrun is completed)
c) 2 RACKS - Pair of Gen 1 Leaf Switches are to be replaced with pair of new Gen 3 switches
Sunday 18th Aug 10AM to 10PM
d) 2 RACKS - ...Pair of Gen 1 Leaf Switches are to be replaced with pair of new Gen 3 switches
e) 1 Border Rack - Pair of Gen 1 Border Leaf Switches are to be replaced with pair of new Gen 3 switches
Project Scope/Schedule - https://confluence.its.uq.edu.au/confluence/display/ITSNDC/ACI+Upgrade+Project+2023-2024
Read more
Resolution: 18 Aug 2024, 11:00 PM
Automatic resolution.
Service affected: NGDC Serverfarm
Start: 16 Aug 2024, 6:00 PM
End: 18 Aug 2024, 11:00 PM
Login to request notification
Learn.UQ: LTI External Users unavailable
Resolved
The External Users LTI integration within Blackboard Learn.UQ is currently experiencing technical issues. Technicians have been alerted and a review is underway.
Resolution: 19 Aug 2024, 12:28 PM
An issue has been identified whereby specific courses whereby the course's name contains invalid non-ascii characters experiences errors accessing this LTI tool.
Service affected: LMS LTI - External User
Start: 16 Aug 2024, 9:48 AM
End: 19 Aug 2024, 12:28 PM
Login to request notification
Weekly Production Migration for SI-net
Resolved
Weekly Migration to Production.
Refer: https://confluence.its.uq.edu.au/confluence/pages/viewpage.action?pageId=175726386 for a full list of changes, or contact eaps@uq.edu.au for more information.
Refer: https://confluence.its.uq.edu.au/confluence/pages/viewpage.action?pageId=175726386 for a full list of changes, or contact eaps@uq.edu.au for more information.
Resolution: 15 Aug 2024, 7:30 AM
Automatic resolution.
Service affected: SI-net
Start: 15 Aug 2024, 7:00 AM
End: 15 Aug 2024, 7:30 AM
Login to request notification
Network Upgrade - Gatton 8106, 8114 and 8307
Resolved
8106 and 8114 will affect AV only, 8307 will affect WiFi in houses 8307 to 8329. For further information, please contact the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 11 Aug 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 11 Aug 2024, 10:00 AM
End: 11 Aug 2024, 10:00 PM
Login to request notification
Migrate Control-M Production databases to new hardware
Resolved
Migrate Control-M Production databases to new Exatada hardware
Resolution: 10 Aug 2024, 6:00 PM
Automatic resolution.
Service affected: Control-M
Start: 10 Aug 2024, 12:00 PM
End: 10 Aug 2024, 6:00 PM
Login to request notification
Blackboard Learn Instability with integrations
Resolved
The Blackboard Learn.UQ environment is currently experiencing issues with stability. ITS has engaged with the Vendor for further investigation.
Resolution: 9 Aug 2024, 2:36 PM
An issue has been identified with technicians having implemented solutions aimed at alleviating further disruption to service. The Blackboard Learn.UQ platform is now in stable condition.
Resolution: 9 Aug 2024, 2:36 PM
An issue has been identified with technicians having implemented solutions aimed at alleviating further disruption to service. The Blackboard Learn.UQ platform is now in stable condition.
Service affected: Blackboard Learn
Start: 9 Aug 2024, 2:30 PM
End: 9 Aug 2024, 2:36 PM
Login to request notification
Migrate Collation Production databases to new hardware
Resolved
Migrate Collation Production databases to new Exatada hardware.
Resolution: 9 Aug 2024, 10:30 AM
Automatic resolution.
Service affected: Collation Database
Start: 9 Aug 2024, 9:00 AM
End: 9 Aug 2024, 10:30 AM
Login to request notification
Heron Island Network Outaqe - Telsra Mobile network also offline
Resolved
Heron Island Research Station communications network to the mainland is offline.
Carrier services are investigating.
Carrier services are investigating.
Resolution: 10 Aug 2024, 1:40 PM
Power restored to Heron Island Carrier equipment
Update: 13 Aug 2024, 9:11 AM
Power restored and network back online
Update: 9 Aug 2024, 9:57 AM
The communciations systems power is offline due to water ingress in the Telstra Hut on Heron Island causing the breakers to trip. Staff are waiting for the site to dry out before attempting to power the equipment back up.
Service affected: Building Networks
Start: 9 Aug 2024, 4:32 AM
End: 10 Aug 2024, 1:40 PM
Login to request notification
Weekly Production Migration for SI-net
Resolved
Weekly Migration to Production.
Refer: https://confluence.its.uq.edu.au/confluence/pages/viewpage.action?pageId=175726386 for a full list of changes, or contact eaps@uq.edu.au for more information.
Refer: https://confluence.its.uq.edu.au/confluence/pages/viewpage.action?pageId=175726386 for a full list of changes, or contact eaps@uq.edu.au for more information.
Resolution: 8 Aug 2024, 7:30 AM
Automatic resolution.
Service affected: SI-net
Start: 8 Aug 2024, 7:00 AM
End: 8 Aug 2024, 7:30 AM
Login to request notification
Workday NOT Available During Maintenance
Resolved
Workday NOT Available During Maintenance
Resolution: 8 Aug 2024, 7:00 AM
Automatic resolution.
Service affected: Workday - Human Capital Management System
Start: 7 Aug 2024, 5:00 PM
End: 8 Aug 2024, 7:00 AM
Login to request notification
AV Room Control
Resolved
Issue impacting projection in AV Rooms. Projection is muted after short period and does not restore without intervention.
Resolution: 7 Aug 2024, 8:15 AM
Rollback of software version is now complete and functionality has been restored.
Update: 7 Aug 2024, 10:12 AM
Teaching spaces have all now been resolved, with remaining meeting and other spaces still being rolled back. Full roll back expected within the hour.
Update: 7 Aug 2024, 9:15 AM
Majority of rooms have been rolled back to previous version rectifying issue. Rollbacks to remaining rooms impacted are underway.
Service affected: AV Room Control
Start: 7 Aug 2024, 7:45 AM
End: 7 Aug 2024, 8:15 AM
Login to request notification
SI-net Weekly Migrations
Resolved
SI-net Weekly Migration / Security Maintenance
Resolution: 7 Aug 2024, 6:55 AM
Automatic resolution.
Service affected: SI-net
Start: 7 Aug 2024, 6:40 AM
End: 7 Aug 2024, 6:55 AM
Login to request notification
Gatton Solar Firewall Migration
Resolved
Migrating physical connections from EoL Palo pa500 to Palo pa820
Resolution: 6 Aug 2024, 2:30 PM
Automatic resolution.
Service affected: P&F Gatton Solar Plant
Start: 6 Aug 2024, 2:00 PM
End: 6 Aug 2024, 2:30 PM
Login to request notification
Network upgrade - Gehrman level 2 and 3
Resolved
There will be no network service on Gehrman level 2 and 3 during this upgrade. For further information, please call the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 4 Aug 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 4 Aug 2024, 7:00 AM
End: 4 Aug 2024, 10:00 PM
Login to request notification
Blackboard Learn: Echo360 embedded video issues
Resolved
An update released by the vendor over night has resulted in embedded Echo360 video content not functioning as expected. Users will encounter an error message instead. The Vendor is currently working to resolve this issue.
Resolution: 1 Aug 2024, 11:14 AM
The issue with embedded Echo360 content has now been resolved.
Resolution: 1 Aug 2024, 11:14 AM
Automatic resolution.
Update: 1 Aug 2024, 11:46 AM
The issue with embedded Echo360 content has now been resolved.
Service affected: Blackboard Learn
Start: 1 Aug 2024, 9:30 AM
End: 1 Aug 2024, 11:14 AM
Login to request notification
Intermittent data access issues on Network file storage
Resolved
User are currently experience issues accessing some data from Network file storage.
This is impacting Standard share drive and home drives and also potentially RDM data.
ITS staff and Vendors are currently investigating
This is impacting Standard share drive and home drives and also potentially RDM data.
ITS staff and Vendors are currently investigating
Resolution: 1 Aug 2024, 10:53 AM
Scope of the problem was found to be limited to just large restores onto one node that had a misconfiguration on the load balancer. This has now been rectified and confirmed working.
Update: 1 Aug 2024, 4:37 PM
Issue summary:
Intermittent access issue to ECS due to misconfigured node on F5.
Some nodes could not talk to ECS as result, and trying to restore files on one node errored.
Restoring using the whole cluster fixed (to avoid the Unavailable nodes)
Issue permanently fixed and confirmed all nodes now working
Update: 1 Aug 2024, 11:12 AM
ITS Staff and Dell Techs have confirmed the impact is limited to recovery of any snapshot data being retrieved from ECS.
No impact to RDM data
No impact to existing data on shared drives on NAS01 and NAS02
ITS and Dell Techs are doing further investigation into issues in recovery snapshots
Update: 1 Aug 2024, 10:54 AM
Testing is now showing RDM is not impacted, and issues are limited to NAS02, and NAS01 files on ECS can be retrieved ok
Service affected: nas01 NFS File Storage
Start: 1 Aug 2024, 8:07 AM
End: 1 Aug 2024, 10:53 AM
Login to request notification
SYNCPROD Database Oracle 19c upgrade
Resolved
SYNCPROD Oracle database upgrade from 18c to 19c
Resolution: 1 Aug 2024, 12:00 PM
Automatic resolution.
Service affected: Oracle Database Administration
Start: 1 Aug 2024, 6:00 AM
End: 1 Aug 2024, 12:00 PM
Login to request notification
UQRDM Cloud is currently unavailable
Resolved
The UQ RDM Cloud portal is currently offline while ITS are working to resolve database issues.
Data is still currently accessable via shares.
Data is still currently accessable via shares.
Resolution: 31 Jul 2024, 4:39 PM
database functions restored
Service affected: UQRDM Cloud
Start: 31 Jul 2024, 3:53 PM
End: 31 Jul 2024, 4:39 PM
Login to request notification
HUBPROD Database Oracle 19c upgrade
Resolved
HUBPROD Oracle database upgrade from 12c to 19c
Resolution: 30 Jul 2024, 2:00 PM
Automatic resolution.
Resolution: 30 Jul 2024, 10:00 AM
Automatic resolution.
Resolution: 30 Jul 2024, 10:00 AM
Automatic resolution.
Update: 30 Jul 2024, 12:01 PM
The Upgrade activity has overrun and the new completion time is 14:00, if not earlier.
A new update will be sent once the upgraded environment is released.
Resolution: 30 Jul 2024, 10:00 AM
Automatic resolution.
Service affected: UQ Data Hub
Start: 30 Jul 2024, 6:00 AM
End: 30 Jul 2024, 2:00 PM
Login to request notification
HUBPROD Database Oracle 19c upgrade
Resolved
HUBPROD Oracle database upgrade from 12c to 19c
Resolution: 30 Jul 2024, 2:00 PM
Automatic resolution.
Service affected: UQ Data Hub
Start: 30 Jul 2024, 6:00 AM
End: 30 Jul 2024, 2:00 PM
Login to request notification
UQ Wireless network software upgrade
Resolved
Scheduled maintenance for the wireless network is set for Sunday morning, which will result in two brief 5-minute outages. The first outage at 8:00 am will affect all UQ sites except the Colleges, and the second at 10:30 am will impact all sites outside of St Lucia.
Resolution: 28 Jul 2024, 11:00 AM
Automatic resolution.
Service affected: Wireless Network
Start: 28 Jul 2024, 8:00 AM
End: 28 Jul 2024, 11:00 AM
Login to request notification
Network Upgrade - Gatton 8114 Levels 1 and 2
Resolved
There will be no network on levels 1 and 2 during this upgrade. For further information, please contact the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 28 Jul 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 28 Jul 2024, 7:00 AM
End: 28 Jul 2024, 10:00 PM
Login to request notification
Migrate SI-net Production databases to new hardware
Resolved
Migrate SI-net Production databases to new Exatada hardware
Resolution: 28 Jul 2024, 11:00 AM
Automatic resolution.
Service affected: Oracle Production DB Hosting
Start: 28 Jul 2024, 5:00 AM
End: 28 Jul 2024, 11:00 AM
Login to request notification
Exams Management System Upgrade
Resolved
The Exams Management System (Databee) will be upgraded to the latest version.
Resolution: 27 Jul 2024, 1:00 PM
Automatic resolution.
Service affected: Examinations Management System
Start: 27 Jul 2024, 9:00 AM
End: 27 Jul 2024, 1:00 PM
Login to request notification
UQRDM Service - Significant service impact
Resolved
The UQRDM service is currently being impacted by a significant servcie disruption.
ITS/RCC Staff are currently investigating and working with vendors to restore services.
ITS/RCC Staff are currently investigating and working with vendors to restore services.
Resolution: 26 Jul 2024, 1:00 PM
Mitigated then resolved by RCC
Update: 26 Jul 2024, 11:49 AM
The ITS & RCC team are finalising a permanent resolution for the issue, but a workaround has been implemented which has enabled access to UQRDM to be partially restored. Whilst working through the backlog of data some recalls from Q Collections will continue to be impacted and users may experience additional latency issues until this is mitigated.
The incident has now been downgraded to a P2.
Service affected: UQRDM Cloud
Start: 26 Jul 2024, 8:00 AM
End: 26 Jul 2024, 1:00 PM
Login to request notification
Gatton Power Outages - Multiple buildings affected
Resolved
Their are multiple buildings impacted by power outages. No network services will be available for those impacted buildings. P&F aware, working with Energex to resolve.
Network services to Gatton site are not impaced
Network services to Gatton site are not impaced
Resolution: 25 Jul 2024, 12:30 PM
Power restored to buildings
All network switches and services back on line
Service affected: Networks
Start: 25 Jul 2024, 5:12 AM
End: 25 Jul 2024, 12:30 PM
Login to request notification
Upgrade of Jira to version 9.4.24 (Patch Critical Vulnerabilities)
Resolved
As per the Atlassian July 2024 Security Bulletin, an urgent update needs to be applied to patch these critical vulnerabilities.
Resolution: 23 Jul 2024, 7:30 PM
Automatic resolution.
Service affected: ITS Jira
Start: 23 Jul 2024, 6:00 PM
End: 23 Jul 2024, 7:30 PM
Login to request notification
Email in Service CRM not being delivered to student facing queuess
Resolved
Email to Oracle Service Cloud (Service CRM) not being delivered to CRM queues on the uqcurrent interface (Student facing queues)
Email is not being retrieved from mailboxes for the student interface.
All other email is being delivered from exchange
Email is not being retrieved from mailboxes for the student interface.
All other email is being delivered from exchange
Resolution: 22 Jul 2024, 4:00 PM
Oracle restarted the techmail utility and email started processing again.
Root Cause still unknown. Oracle investigating monitoring options.
Service affected: Oracle Service Cloud a.k.a Service CRM
Start: 22 Jul 2024, 1:38 PM
End: 22 Jul 2024, 4:00 PM
Login to request notification
MFA service unavailable
Resolved
Users attempting to authenticate to MFA protected systems are being redirected in a loop. ITS staff are investigating.
Resolution: 21 Jul 2024, 10:10 AM
Traffic between UQ Authentication and MFA services have been restored.
Update: 21 Jul 2024, 10:28 AM
Traffic between UQ Authentication and MFA services have been restored.
Service affected: UQ Authenticate
Start: 21 Jul 2024, 7:30 AM
End: 21 Jul 2024, 10:10 AM
Login to request notification
Unable to copy files in OneDrive and SharePoint
Resolved
Users are currently unable to copy files within OneDrive and SharePoint. Microsoft are aware of the issue (Microsoft advisory notice SP820584) and working on a resolution.
Resolution: 19 Jul 2024, 4:45 PM
Preliminary root cause: A configuration change in a portion of Microsoft's Azure backend workloads, caused interruption between storage and compute resources which resulted in connectivity failures that affected downstream Microsoft 365 services dependent on these connections.
Update: 19 Jul 2024, 2:45 PM
As per Microsoft, as at 2.45pm:
We remain focused on redirecting the impacted traffic to healthy systems as we investigate the root cause. Your organization may experience relief as our mitigation efforts progress. We understand the impact that this issue may have on your organization and we're continuing to treat this event with the highest priority.
Additionally, we are shifting our focus to developing a mitigation that will help determine and resolve the underlying source of the problem.
Service affected: Office 365 - OneDrive
Start: 19 Jul 2024, 4:19 AM
End: 19 Jul 2024, 4:45 PM
Login to request notification
Upgrade of Confluence to version 8.5.12 (Patch Critical Vulnerabilities)
Resolved
As per the Atlassian July 2024 Security Bulletin, an urgent update needs to be applied to patch these critical vulnerabilities.
Resolution: 18 Jul 2024, 7:00 PM
Automatic resolution.
Service affected: ITS Confluence
Start: 18 Jul 2024, 6:00 PM
End: 18 Jul 2024, 7:00 PM
Login to request notification
Issues recovering deleted items from users Recycle Bin in OneDrive and SharePoint Online
Resolved
Some users are reporting issues when attempting to recover previously deleted items from their Recycle Bin in the M365 OneDrive and M365 SharePoint Online services.
ITS staff are currently working with Microsoft to troublehsoot and rectify this issue.
ITS staff are currently working with Microsoft to troublehsoot and rectify this issue.
Resolution: 19 Jul 2024, 4:45 PM
Preliminary root cause: A configuration change in a portion of Microsoft's Azure backend workloads, caused interruption between storage and compute resources which resulted in connectivity failures that affected downstream Microsoft 365 services dependent on these connections.
Resolution: 19 Jul 2024, 4:44 PM
Preliminary root cause: A configuration change in a portion of Microsoft's Azure backend workloads, caused interruption between storage and compute resources which resulted in connectivity failures that affected downstream Microsoft 365 services dependent on these connections.
Update: 19 Jul 2024, 4:41 PM
It appears that this issue has now been resolved by Microsoft. We will keep this Service Status entry opened until the start of next week / until MS advise it is resolved.
Service affected: Office 365 - OneDrive
Start: 17 Jul 2024, 4:00 PM
End: 19 Jul 2024, 4:45 PM
Login to request notification
UQ Extend access issues
Resolved
Though access to UQ Extend is working, ITS is aware of reports for some content not working properly on UQ Extend since the recent upgrade. ITS is troubleshooting with the vendor to resolve these issues.
Resolution: 11 Oct 2024, 2:25 PM
Service affected: UQ Extend Learning Activity Management System
Start: 15 Jul 2024, 11:40 PM
End: 11 Oct 2024, 2:25 PM
Login to request notification
UQ Extend service issues
Resolved
ITS is investigating access issues to https://extend.uq.edu.au (UQ Extend/AIM). Some are reporting access issues, but for others access is ok.
Resolution: 15 Jul 2024, 11:40 PM
Full access/normal service resumed on UQ Extend/AIM at 23:40 Monday night.
Update: 15 Jul 2024, 10:57 AM
ITS is troubleshooting with the vendor and working on the issue affecting UQ Extend/AIM. Some of the course content is not working properly. Staff and students may encounter error messages.
Service affected: UQ Extend Learning Activity Management System
Start: 15 Jul 2024, 7:20 AM
End: 15 Jul 2024, 11:40 PM
Login to request notification
Network upgrade - Gatton 8106
Resolved
There will be no network services in 8106 during this upgrade. For further information, please call the Project Manager, Keryn Masters, 0458 585 405.
Resolution: 14 Jul 2024, 10:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 14 Jul 2024, 7:00 AM
End: 14 Jul 2024, 10:00 PM
Login to request notification
Control-M production database upgrade
Resolved
The Control-M enterprise scheduler will be unavailable whilst the database is upgraded to the latest version of Oracle.
Resolution: 13 Jul 2024, 4:00 PM
Automatic resolution.
Service affected: Control-M
Start: 13 Jul 2024, 12:00 PM
End: 13 Jul 2024, 4:00 PM
Login to request notification
UQ Extend scheduled upgrade
Resolved
The UQ Extend platform will be unavailable as a result of upgrades occurring during this time.
Resolution: 13 Jul 2024, 12:00 AM
Automatic resolution.
Service affected: UQ Extend Learning Activity Management System
Start: 12 Jul 2024, 4:00 PM
End: 13 Jul 2024, 12:00 AM
Login to request notification
Headshots Production Database Oracle 19c upgrade
Resolved
Headshots Production Database (ITSHSP) upgrade from 18c to 19c and conversion to multitenant
Resolution: 11 Jul 2024, 2:00 PM
Automatic resolution.
Service affected: Headshot Database
Start: 11 Jul 2024, 10:00 AM
End: 11 Jul 2024, 2:00 PM
Login to request notification
UQ StudentHub (CareerHub) version upgrade
Resolved
The UQ StudentHub (Careerhub) application will be upgraded by the vendor (Symplicity) to version 5.13.
This new version will introduce a number of bug fixes and feature updates, including:
- Job Alert emailing
- Organisational logos
- Timezone updates for online/off campus
- Event management improvements
- Experiences and Skills improvements
- Improvements to the application permissions model
- New API endpoints
This new version will introduce a number of bug fixes and feature updates, including:
- Job Alert emailing
- Organisational logos
- Timezone updates for online/off campus
- Event management improvements
- Experiences and Skills improvements
- Improvements to the application permissions model
- New API endpoints
Resolution: 10 Jul 2024, 8:30 PM
Automatic resolution.
Service affected: CareerHub
Start: 10 Jul 2024, 8:00 PM
End: 10 Jul 2024, 8:30 PM
Login to request notification
COLLPROD Database Oracle 19c upgrade
Resolved
COLLATION PROD Oracle database upgrade from 12c to 19c
Resolution: 10 Jul 2024, 2:00 PM
Automatic resolution.
Service affected: Collation Database
Start: 10 Jul 2024, 10:00 AM
End: 10 Jul 2024, 2:00 PM
Login to request notification
UQ Book It Internal Server Error
Resolved
The system is returning an internal server error on login. The issue has been escalated to the vendor.
Resolution: 9 Jul 2024, 4:00 PM
Service affected: UQ Bookit
Start: 9 Jul 2024, 3:00 PM
End: 9 Jul 2024, 4:00 PM
Login to request notification
OLA Form Validation Issues Encountered
Resolved
We have received request in regards to form validation errors. It is believed this is only impacting certain pages. We are currently investigating the issue and hope to have a resolution as soon as possible.
Resolution: 9 Jul 2024, 2:49 PM
A patch has been applied to resolve the form validation issue. This issue is believed to be resolved, we will continue to monitor for the rest of the day.
Service affected: Online Applications - OLA
Start: 9 Jul 2024, 11:39 AM
End: 9 Jul 2024, 2:49 PM
Login to request notification
Blackboard Learning Pathway upgrade maintenance
Resolved
The Blackboard Learn Learning Pathways integrations will be unavailable for a period of 3 hours while the vendor performs an upgrade to add new functionality.
Resolution: 9 Jul 2024, 12:00 AM
Automatic resolution.
Service affected: Learning Pathway
Start: 8 Jul 2024, 9:00 PM
End: 9 Jul 2024, 12:00 AM
Login to request notification
Service CRM Mailboxes not delivering all email
Resolved
We are receiving reports of:
1. Email to Service CRM not being delivered as marked as Junk in the mailbox
2. Email to Service CRM not being delivered (not marked as junk) and blocking further delivery for that mailbox
1. Email to Service CRM not being delivered as marked as Junk in the mailbox
2. Email to Service CRM not being delivered (not marked as junk) and blocking further delivery for that mailbox
Resolution: 9 Jul 2024, 12:08 PM
Ongoing investigation and monitoring
Service affected: Exchange Mail and Calendaring
Start: 8 Jul 2024, 3:00 PM
End: 9 Jul 2024, 12:08 PM
Login to request notification
Deploy Class Teams Provisioning for Staff and Students
Resolved
This new process will create Microsoft Teams for Learn.uq.edu.au courses using the Microsoft School Data Sync service.
Once deployed, this process will run ongoing indefinitely.
Learn.uq.edu.au course instructors and teacher assistants will have access to their relevant Teams as Team Owners.
Other staff and students will have no visibility or access to the course Team unless the Team is activated by the staff Team Owners.
Once deployed, this process will run ongoing indefinitely.
Learn.uq.edu.au course instructors and teacher assistants will have access to their relevant Teams as Team Owners.
Other staff and students will have no visibility or access to the course Team unless the Team is activated by the staff Team Owners.
Resolution: 8 Jul 2024, 4:00 PM
Automatic resolution.
Service affected: Microsoft School Data Sync
Start: 8 Jul 2024, 10:00 AM
End: 8 Jul 2024, 4:00 PM
Login to request notification