IT service updates
Cycad network upgrade and outage
Service NOT available during period
There will be no network service in Cycad, She-Oak and Tuckeroo while an upgrade is performed. Please avoid the buildings where possible. For more information, please contact the Project Manager, Keryn Masters on 0458 585 405.
Service affected: Building Networks
Start: 2 Apr 2023, 10:00 AM
End: 2 Apr 2023, 9:00 PM
Login to request notification
VM Provisioning Unavailable
Service NOT available during period
Vendor security changes are causing some VM provisioning processes to fail. ITS Engineers are investigating.
Service affected: VPNext
Start: 31 Mar 2023, 7:28 AM
End: Unknown
Login to request notification
HAZARD AWARENESS - DC1 UPS Failure - On Mains Power Only
No impact on service during period
DC1 has lost UPS protection due to significant failure on all battery strings. There is NO impact at this stage to services, however the DataCentre is now on mains power only.
UPS Service Tech is on the way to site.
UPS Service Tech is on the way to site.
Service affected: Data Centre Network
Start: 10 Mar 2023, 9:30 AM
End: Unknown
Login to request notification
Call transfers failing
Reduced service delivery during period
Some phone calls are losing audio when being transferred. The customer will end up on a dead line.
Agents are advised to try warm transfer for the time being.
Agents are advised to try warm transfer for the time being.
Update: 2 Feb 2023, 11:30 AM
Phone carrier has released a patch to try resolve the issue.
Update: The issue continues to persist after the patch. Carrier investigating.
Service affected: Amazon Connect
Start: 23 Jan 2023, 1:00 PM
End: Unknown
Login to request notification
UQ Book It Booking Request Notification Issue
Reduced service delivery during period
Bookers are not receiving booking request notifications from the UQ Book It system. Other email notifications are still working such as accepted, rejected and cancelled.
Service affected: UQ Bookit
Start: 18 Jan 2023, 10:26 AM
End: Unknown
Login to request notification
Access to UQAPS Citrix only accessible via UQ Network or VPN
Reduced service delivery during period
Access to UQAPS / Citrix has been changed to be only accessible from the UQ Network / VPN for the time being.
At this stage there is no ETA when this will be re-enabled.
At this stage there is no ETA when this will be re-enabled.
Service affected: UQAPS Citrix
Start: 13 Jan 2023, 3:42 PM
End: Unknown
Login to request notification
Prentice DC2 Rack B3 Power redundancy failure
No impact on service during period
A power rail has tripped RCD protection in Prentice DC2 Rack B3. All Equipment in this rack is running without power redundancy. The issue is being investigated by ITS engineers.
Update: 30 Sep 2022, 11:45 AM
All services are online from remaining power supply. ITS DC Engineers are investigating
Service affected: Data Centre
Start: 30 Sep 2022, 11:02 AM
End: Unknown
Login to request notification
Gerhamn Building 60 - Network redundancy reduced
No impact on service during period
Due to an un-resolved fault within the Network Access Layer, all access switches in building 60 are connected to a single Agg, therefore reducing network redundancy.
This will be in place until the fault can be found and rectified
This will be in place until the fault can be found and rectified
Service affected: Building Networks
Start: 16 Sep 2022, 2:12 PM
End: Unknown
Login to request notification
WebPrism Slow Performance
Reduced service delivery during period
Intermittent issues will be experienced
Service affected: PrISM Identity Management
Start: 15 Aug 2022, 10:30 PM
End: Unknown
Login to request notification
Building 89f Boat Shed - No network services - No Power
Service NOT available during period
Due to flooding network services are currently not available
Service affected: Building Networks
Start: 1 Mar 2022, 11:34 AM
End: Unknown
Login to request notification
Apply ESXi 6.7 patches to PRN Nextgen Nutanix cluster
Resolved
Apply latest ESXi 6.7 patches to PRN Nextgen vSphere Datacenter - Prentice Nutanix cluster.
Resolution: 2 Apr 2023, 5:00 PM
Automatic resolution.
Service affected: VMware NextGEN PRN DC
Start: 2 Apr 2023, 9:00 AM
End: 2 Apr 2023, 5:00 PM
Login to request notification
Address Critical Vulnerability in the ITS Confluence Service
Resolved
A critical service vulnerability (Confluence Security Advisory 2022-06-02) has been identified to exist in the ITS Confluence Service. We need to apply the mitigation that Atlassian has provided.
More information about this vulnerability can be found at: https://confluence.atlassian.com/doc/confluence-security-advisory-2022-06-02-1130377146.html
This maintenance work is required to address and resolve this critical vulnerability.
More information about this vulnerability can be found at: https://confluence.atlassian.com/doc/confluence-security-advisory-2022-06-02-1130377146.html
This maintenance work is required to address and resolve this critical vulnerability.
Resolution: 31 Mar 2023, 6:00 PM
Automatic resolution.
Service affected: ITS Confluence
Start: 31 Mar 2023, 5:00 PM
End: 31 Mar 2023, 6:00 PM
Login to request notification
Disabling virtual machine logging (For Nutanix CVM) - Prn Nutanix cluster.
Resolved
Nutanix clusters on VMware ESXi 6.7U3i may experience stability issues
Products Affected:
Nutanix clusters running VMware ESXi 6.7U3 - 7.0U3
Severity: Critical
Products Affected:
Nutanix clusters running VMware ESXi 6.7U3 - 7.0U3
Severity: Critical
Resolution: 31 Mar 2023, 7:30 PM
Automatic resolution.
Service affected: VMware NextGEN PRN DC
Start: 31 Mar 2023, 5:00 PM
End: 31 Mar 2023, 7:30 PM
Login to request notification
Emergency reboot of Host in PRN NextGen DC. Refer to Description for VM Hosts require shutdown
Resolved
Reboot of ntnx-78qzfl2-a.inf.dc.uq.edu.au is required to resolve an issue causing the host to become unresponsive in vCenter server.
VM Hosts will need to be shutdown for the reboot, and service owners have been advised.
VMs which will be impacted -
DNS Name
corpusmate.compute.dc.uq.edu.au
fom-imu01.compute.dc.uq.edu.au
pf-arc-stl16.compute.dc.uq.edu.au
pf-ir.compute.dc.uq.edu.au
uq-fleetrmm-mgt.compute.dc.uq.edu.au
uq-mspapp03.compute.dc.uq.edu.au
auscontrol.compute.dc.uq.edu.au
auslegacy.compute.dc.uq.edu.au
auswekan.compute.dc.uq.edu.au
its-ds-sql16.compute.dc.uq.edu.au
auraps01....compute.dc.uq.edu.au
aurwcp01.compute.dc.uq.edu.au
im-duo-01.compute.dc.uq.edu.au
salt.compute.dc.uq.edu.au
uq-adstl07.uq.edu.au
uq-azmgmt01.test.ad.uq.edu.au
uq-azmgmt1.compute.dc.uq.edu.au
clag.compute.dc.uq.edu.au
crayola.compute.dc.uq.edu.au
gravitational.compute.dc.uq.edu.au
insist.compute.dc.uq.edu.au
kermit.compute.dc.uq.edu.au
ping.compute.dc.uq.edu.au
psyduck.compute.dc.uq.edu.au
qm-server3.compute.dc.uq.edu.au
rmuir-testbox.compute.dc.uq.edu.au
roof.compute.dc.uq.edu.au
roomba.compute.dc.uq.edu.au
ohna-db.compute.dc.uq.edu.au
baldonnell.inf.dc.uq.edu.au
ntnx-78qzfl2-a-cvm
ca-bahamas.compute.dc.uq.edu.au
cses-prod.compute.dc.uq.edu.au
ctmmfthub01.compute.dc.uq.edu.au
its-testlab3.compute.dc.uq.edu.au
ctmhubpd01.compute.dc.uq.edu.au
ajc.maths.uq.edu.au
uq-pcut-test.compute.dc.uq.edu.au
frontloader.compute.dc.uq.edu.au
smtp1.compute.dc.uq.edu.au
ise-pan
its-un-ucesql.compute.dc.uq.edu.au
sauron.compute.dc.uq.edu.au
VM Hosts will need to be shutdown for the reboot, and service owners have been advised.
VMs which will be impacted -
DNS Name
corpusmate.compute.dc.uq.edu.au
fom-imu01.compute.dc.uq.edu.au
pf-arc-stl16.compute.dc.uq.edu.au
pf-ir.compute.dc.uq.edu.au
uq-fleetrmm-mgt.compute.dc.uq.edu.au
uq-mspapp03.compute.dc.uq.edu.au
auscontrol.compute.dc.uq.edu.au
auslegacy.compute.dc.uq.edu.au
auswekan.compute.dc.uq.edu.au
its-ds-sql16.compute.dc.uq.edu.au
auraps01....compute.dc.uq.edu.au
aurwcp01.compute.dc.uq.edu.au
im-duo-01.compute.dc.uq.edu.au
salt.compute.dc.uq.edu.au
uq-adstl07.uq.edu.au
uq-azmgmt01.test.ad.uq.edu.au
uq-azmgmt1.compute.dc.uq.edu.au
clag.compute.dc.uq.edu.au
crayola.compute.dc.uq.edu.au
gravitational.compute.dc.uq.edu.au
insist.compute.dc.uq.edu.au
kermit.compute.dc.uq.edu.au
ping.compute.dc.uq.edu.au
psyduck.compute.dc.uq.edu.au
qm-server3.compute.dc.uq.edu.au
rmuir-testbox.compute.dc.uq.edu.au
roof.compute.dc.uq.edu.au
roomba.compute.dc.uq.edu.au
ohna-db.compute.dc.uq.edu.au
baldonnell.inf.dc.uq.edu.au
ntnx-78qzfl2-a-cvm
ca-bahamas.compute.dc.uq.edu.au
cses-prod.compute.dc.uq.edu.au
ctmmfthub01.compute.dc.uq.edu.au
its-testlab3.compute.dc.uq.edu.au
ctmhubpd01.compute.dc.uq.edu.au
ajc.maths.uq.edu.au
uq-pcut-test.compute.dc.uq.edu.au
frontloader.compute.dc.uq.edu.au
smtp1.compute.dc.uq.edu.au
ise-pan
its-un-ucesql.compute.dc.uq.edu.au
sauron.compute.dc.uq.edu.au
Read more
Resolution: 31 Mar 2023, 3:45 PM
Automatic resolution.
Service affected: VMware NextGEN PRN DC
Start: 31 Mar 2023, 3:00 PM
End: 31 Mar 2023, 3:45 PM
Login to request notification
Reboot its-ds-sql16 - For Emergency Change Request #12351
Resolved
IS will reboot Reboot ntnx-78qzfl2-a.inf.dc.uq.edu.au to resolve an issue causing the host to become unresponsive in vCenter server. - Emergency Change Request #12351
Affected VM VM its-ds-sql16.compute.dc.uq.edu.au will be shutdown before the scheduled reboot at 3 pm.
Affected VM VM its-ds-sql16.compute.dc.uq.edu.au will be shutdown before the scheduled reboot at 3 pm.
Resolution: 31 Mar 2023, 3:15 PM
Automatic resolution.
Service affected: SQL 2017 Clustered Database Hosting
Start: 31 Mar 2023, 2:55 PM
End: 31 Mar 2023, 3:15 PM
Login to request notification
SI-net - Online Search Function Unavailable
Resolved
Search functionality from the banner at the top of the page will be unavailable.
Lookup search from withing individual pages will be available.
Lookup search from withing individual pages will be available.
Resolution: 31 Mar 2023, 3:26 PM
services restarted as expected. Search functionality has been restored
Service affected: SI-net
Start: 31 Mar 2023, 2:55 PM
End: 31 Mar 2023, 3:26 PM
Login to request notification
Awareness Only - PRN NextGen resilance in reduced state
Resolved
The PRN NextGen environment is in a reduced state of resilance on the Nutanix cluster due to VMware bug affecting one host.
Currently in the process of shutting down VMs on the affected host in order to reboot and restore resilancy.
Currently in the process of shutting down VMs on the affected host in order to reboot and restore resilancy.
Resolution: 31 Mar 2023, 3:57 PM
Impacted host has been reloaded
Service affected: VMware NextGEN PRN DC
Start: 31 Mar 2023, 8:00 AM
End: 31 Mar 2023, 3:57 PM
Login to request notification
Q collection data service issue
Resolved
The storage infrastructure that provides back end delivery of older, larger files for UQ RDM and QRISCloud Q collections has been disrupted.
Access to older, less frequently accessed data may be impacted by this issue.
Technical staff are working with the storage vendor to restore full functionality.
Access to older, less frequently accessed data may be impacted by this issue.
Technical staff are working with the storage vendor to restore full functionality.
Resolution: 30 Mar 2023, 1:15 PM
Aarnet fixed an issue on the link and storage services have been restored.
Update: 30 Mar 2023, 12:40 PM
The upstream network provider (AARNET) had determined that there is an active network fault in their services. This has caused cascading effects upon UQ infrastructure.
AARNET NoC are working to correct the issues on their infrastructure.
To resolve the cascading effects, UQ RCC staff are now bringing services back into production and a resolution time will be forthcoming.
Service affected: Giovanni SpectrumScale Cluster
Start: 30 Mar 2023, 10:59 AM
End: 30 Mar 2023, 1:15 PM
Login to request notification
Test failover CCR DC firewall
Resolved
Test failover CCR DC firewall
Resolution: 29 Mar 2023, 11:00 PM
Automatic resolution.
Service affected: NGDC Serverfarm Firewall
Start: 29 Mar 2023, 9:00 PM
End: 29 Mar 2023, 11:00 PM
Login to request notification
Herston CCR and HIRF network upgrade
Resolved
Full network outage in Herston CCR building and HIRF. Nurse call, PWD,
Resolution: 26 Mar 2023, 9:00 PM
Service affected: Building Networks
Start: 26 Mar 2023, 9:00 AM
End: 26 Mar 2023, 9:00 PM
Login to request notification
INFO ONLY - Internet Services to switch over to our secondary link for urgent maintenance
Resolved
INFO ONLY - Urgent maintenance is required on the GW10 internet connection to address port errors on our primary Aarnet Internet link.
Internet services shall be moved over to our GW20 Aarnet link to allow this work to be proceed.
There will be no service impact during this change. Redundant internet services will not be available during this change period.
Internet services shall be moved over to our GW20 Aarnet link to allow this work to be proceed.
There will be no service impact during this change. Redundant internet services will not be available during this change period.
Resolution: 25 Mar 2023, 8:00 PM
Automatic resolution.
Service affected: Internet Services UQ
Start: 25 Mar 2023, 7:00 PM
End: 25 Mar 2023, 8:00 PM
Login to request notification
Gatton Campus Power Outage - Network services impacted
Resolved
Network services have been impacted at Gatton campus due to a power loss in across the campus.
Outage started 4:03PM, network devices impacted came back online around 4:22PM.
P&F Have been advised.
Outage started 4:03PM, network devices impacted came back online around 4:22PM.
P&F Have been advised.
Resolution: 24 Mar 2023, 4:22 PM
Automatic resolution.
Service affected: Building Networks
Start: 24 Mar 2023, 4:03 PM
End: 24 Mar 2023, 4:22 PM
Login to request notification
Weekly Production Migration for SI-net
Resolved
ENTSS Migrator will be migrating new codes changes to Production for SI-net applications
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-21+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrat...e the changes into Production.
This service view will be marked to complete as soon as the migration completes.
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-21+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrat...e the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Read more
Resolution: 23 Mar 2023, 7:30 AM
Automatic resolution.
Service affected: SI-net
Start: 23 Mar 2023, 7:00 AM
End: 23 Mar 2023, 7:30 AM
Login to request notification
Weekly Production Migration for UniFi
Resolved
ENTSS Migrator will be migrating new codes changed to Production for both SI-net and UniFi applications. Refer to the latest weekly migration minutes for the list approved changes:
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-21+Weekly+Production+Changes
In order to seek for approval to migrate changes to Production, it is required that the required member of the team that made the changes to be present at the weekly migration meeting to present their changes. The attendees ( consists of ENTSS staff and EAPS staff ) will review these before making the decision to approve or pre-approval ( conditional to completion of UAT testing ). Refer to the ab...ove hyperlink for the attendees who would reviewed and approved the changes to production.
As such this service view will not have any Peer Reviews nor Approval as that would have been completed as outlined above.
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Additional information:
SI-net CAB Pre-Approved Changes (#7317)
https://sharepoint.uq.edu.au/its/ITGovernance/Lists/PreApproved%20Change%20List/DispForm.aspx?ID=45&Source=https%3A%2F%2Fsharepoint%2Euq%2Eedu%2Eau%2Fits%2FITGovernance%2FLists%2FPreApproved%2520Change%2520List%2FAllItems%2Easpx%23InplviewHashc5ffa001%2D2eb0%2D4be2%2D987f%2Dc20e677aea89%3D&ContentTypeId=0x0100D236C02FEDCD4E48B359619FB9CFDF4E
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-21+Weekly+Production+Changes
In order to seek for approval to migrate changes to Production, it is required that the required member of the team that made the changes to be present at the weekly migration meeting to present their changes. The attendees ( consists of ENTSS staff and EAPS staff ) will review these before making the decision to approve or pre-approval ( conditional to completion of UAT testing ). Refer to the ab...ove hyperlink for the attendees who would reviewed and approved the changes to production.
As such this service view will not have any Peer Reviews nor Approval as that would have been completed as outlined above.
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Additional information:
SI-net CAB Pre-Approved Changes (#7317)
https://sharepoint.uq.edu.au/its/ITGovernance/Lists/PreApproved%20Change%20List/DispForm.aspx?ID=45&Source=https%3A%2F%2Fsharepoint%2Euq%2Eedu%2Eau%2Fits%2FITGovernance%2FLists%2FPreApproved%2520Change%2520List%2FAllItems%2Easpx%23InplviewHashc5ffa001%2D2eb0%2D4be2%2D987f%2Dc20e677aea89%3D&ContentTypeId=0x0100D236C02FEDCD4E48B359619FB9CFDF4E
Read more
Resolution: 23 Mar 2023, 7:30 AM
Automatic resolution.
Service affected: UniFi
Start: 23 Mar 2023, 7:00 AM
End: 23 Mar 2023, 7:30 AM
Login to request notification
Large number of failing authentications
Resolved
Active Directory is experiencing a temporary increase in intermittent failed authentications.
Engineers are currently investigating.
Engineers are currently investigating.
Resolution: 21 Mar 2023, 3:45 PM
Three distinct periods of increased failed authentications occurred. Issues can be attributed to mismatch of new krb tickets with stale cached krb tickets of which the timeout period has now elapsed.
Service affected: Active Directory
Start: 21 Mar 2023, 1:00 PM
End: 21 Mar 2023, 3:45 PM
Login to request notification
New samba sessions may have intermittent issues connecting to file shares on nas01
Resolved
Resolution: 21 Mar 2023, 1:59 PM
AD connections re-established
Service affected: nas01 NFS File Storage
Start: 21 Mar 2023, 12:30 PM
End: 21 Mar 2023, 1:59 PM
Login to request notification
On Premise Exchange Mail Services - Exchange Patching
Resolved
Apply Security Updates for Microsoft Exchange Server 2013 (March 2023).
Resolution: 20 Mar 2023, 5:00 PM
Automatic resolution.
Service affected: Exchange Mail and Calendaring
Start: 20 Mar 2023, 4:00 PM
End: 20 Mar 2023, 5:00 PM
Login to request notification
Library PaperCut update to 22.0.9
Resolved
The Library PaperCut service will be updates to 22.0.9 to patch against some known vulnerabilies.
Releasing print jobs from Library printers will be unavailable while the upgrade it performed.
EAIT students should still be able to print, but charging may be delayed.
Releasing print jobs from Library printers will be unavailable while the upgrade it performed.
EAIT students should still be able to print, but charging may be delayed.
Resolution: 19 Mar 2023, 9:00 PM
Automatic resolution.
Service affected: Library PaperCut
Start: 19 Mar 2023, 8:00 PM
End: 19 Mar 2023, 9:00 PM
Login to request notification
Banksia network upgrade and outage
Resolved
Installation of building aggregation routers. Ful network outage throughout the building. For more information please contact the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 19 Mar 2023, 5:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 19 Mar 2023, 9:00 AM
End: 19 Mar 2023, 5:00 PM
Login to request notification
Apply Oracle PeopleTools and PUM critical patch updates (CPU) on UniFi (January)
Resolved
To ensure our systems remain secure and compliant, UQ must apply PeopleTools and PUM critical patches on UniFi.
Oracle has released their quarterly security vulnerability patches: https://www.oracle.com/security-alerts/cpujan2023.html.
Oracle has released their quarterly security vulnerability patches: https://www.oracle.com/security-alerts/cpujan2023.html.
Resolution: 18 Mar 2023, 5:00 PM
Automatic resolution.
Service affected: UniFi
Start: 18 Mar 2023, 7:00 AM
End: 18 Mar 2023, 5:00 PM
Login to request notification
Disabling virtual machine logging (For Nutanix CVM) - CCR Nutanix cluster.
Resolved
HAZARD ONLY
Emergency change to disable CVM logging as per Nutanix workaround to avoid CVM reloads that can cause loss of services.
CVM reload due to this bug has occurred on 4 occasions so far, one causing a major outage, therefore this is urgent to have applied.
Emergency change to disable CVM logging as per Nutanix workaround to avoid CVM reloads that can cause loss of services.
CVM reload due to this bug has occurred on 4 occasions so far, one causing a major outage, therefore this is urgent to have applied.
Resolution: 17 Mar 2023, 10:00 PM
Automatic resolution.
Service affected: VMware NextGEN CCR DC
Start: 17 Mar 2023, 8:00 PM
End: 17 Mar 2023, 10:00 PM
Login to request notification
Papercut vulnerability patching to 21.2.11
Resolved
ITS will be patching servers to fix the vulnerabilities specified by PaperCut.
Vulnerability: ZDI-CAN-18987 / PO-1216
Vulnerability: ZDI-CAN-19226 / PO-1219
Vulnerability: ZDI-CAN-18987 / PO-1216
Vulnerability: ZDI-CAN-19226 / PO-1219
Resolution: 17 Mar 2023, 6:45 PM
Automatic resolution.
Service affected: Papercut Printing Services
Start: 17 Mar 2023, 6:00 PM
End: 17 Mar 2023, 6:45 PM
Login to request notification
Oracle Client Patching - conman1, cloud-conman
Resolved
January 2023 RU Oracle Client Patching - conman1, cloud-conman
For any individual connections during this window, would need to access via : conman2.compute.dc.uq.edu.au
For any individual connections during this window, would need to access via : conman2.compute.dc.uq.edu.au
Resolution: 17 Mar 2023, 4:00 PM
Automatic resolution.
Service affected: Oracle Development DB Hosting
Start: 17 Mar 2023, 3:00 PM
End: 17 Mar 2023, 4:00 PM
Login to request notification
Connectivity Issues for some clients across various sites
Resolved
Some clients are experience network connection issues. This is random across various sites
ITS is investigating potential DHCP issues
ITS is investigating potential DHCP issues
Resolution: 16 Mar 2023, 11:18 AM
The problem was traced back to several DHCP servers in CCR not working properly. These have been restarted and working ok now.
Update: 16 Mar 2023, 11:20 AM
ITS are actively investigating this issue.
Service affected: Networks
Start: 16 Mar 2023, 10:47 AM
End: 16 Mar 2023, 11:18 AM
Login to request notification
VMs are not accessible in CCR DC
Resolved
VM Hosts offline due to vCentre being down.
VMs recovered, leaving open as some may still require reloading. Apps is working with IS to complete.
VMs recovered, leaving open as some may still require reloading. Apps is working with IS to complete.
Resolution: 16 Mar 2023, 7:00 PM
Every VM in CCR has been briefly inspected and those that required resetting to resolve file system issues have been done.
Update: 16 Mar 2023, 5:49 PM
While VMWare services have been restored, there are some vm's which have locked file systems and require a reboot.
Application owners have been advised to do this, or contact ITS if they are having issues and need assistance.
ITS is also checking every VM for confirm it is not locked, and reseting where that is required.
Will keep this status active for now.
Resolution: 16 Mar 2023, 1:30 AM
Automatic resolution.
Service affected: VMware NextGEN CCR DC
Start: 15 Mar 2023, 10:20 PM
End: 16 Mar 2023, 7:00 PM
Login to request notification
Test failover CCR DC firewall
Resolved
Failover test CCR FW to analyse root cause of Incident 269
Resolution: 15 Mar 2023, 11:00 PM
Automatic resolution.
Service affected: NGDC Serverfarm Firewall
Start: 15 Mar 2023, 9:00 PM
End: 15 Mar 2023, 11:00 PM
Login to request notification
Weekly Production Migration for SI-net
Resolved
ENTSS Migrator will be migrating new codes changes to Production for SI-net applications
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-14+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrat...e the changes into Production.
This service view will be marked to complete as soon as the migration completes.
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-14+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrat...e the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Read more
Resolution: 15 Mar 2023, 5:30 PM
Automatic resolution.
Service affected: SI-net
Start: 15 Mar 2023, 5:00 PM
End: 15 Mar 2023, 5:30 PM
Login to request notification
Weekly Production Migration for UniFi
Resolved
ENTSS Migrator will be migrating new codes changed to Production for both SI-net and UniFi applications. Refer to the latest weekly migration minutes for the list approved changes:
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-14+Weekly+Production+Changes
In order to seek for approval to migrate changes to Production, it is required that the required member of the team that made the changes to be present at the weekly migration meeting to present their changes. The attendees ( consists of ENTSS staff and EAPS staff ) will review these before making the decision to approve or pre-approval ( conditional to completion of UAT testing ). Refer to the above hyp...erlink for the attendees who would reviewed and approved the changes to production.
As such this service view will not have any Peer Reviews nor Approval as that would have been completed as outlined above.
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Additional information:
SI-net CAB Pre-Approved Changes (#7317)
https://sharepoint.uq.edu.au/its/ITGovernance/Lists/PreApproved%20Change%20List/DispForm.aspx?ID=45&Source=https%3A%2F%2Fsharepoint%2Euq%2Eedu%2Eau%2Fits%2FITGovernance%2FLists%2FPreApproved%2520Change%2520List%2FAllItems%2Easpx%23InplviewHashc5ffa001%2D2eb0%2D4be2%2D987f%2Dc20e677aea89%3D&ContentTypeId=0x0100D236C02FEDCD4E48B359619FB9CFDF4E
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-14+Weekly+Production+Changes
In order to seek for approval to migrate changes to Production, it is required that the required member of the team that made the changes to be present at the weekly migration meeting to present their changes. The attendees ( consists of ENTSS staff and EAPS staff ) will review these before making the decision to approve or pre-approval ( conditional to completion of UAT testing ). Refer to the above hyp...erlink for the attendees who would reviewed and approved the changes to production.
As such this service view will not have any Peer Reviews nor Approval as that would have been completed as outlined above.
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Additional information:
SI-net CAB Pre-Approved Changes (#7317)
https://sharepoint.uq.edu.au/its/ITGovernance/Lists/PreApproved%20Change%20List/DispForm.aspx?ID=45&Source=https%3A%2F%2Fsharepoint%2Euq%2Eedu%2Eau%2Fits%2FITGovernance%2FLists%2FPreApproved%2520Change%2520List%2FAllItems%2Easpx%23InplviewHashc5ffa001%2D2eb0%2D4be2%2D987f%2Dc20e677aea89%3D&ContentTypeId=0x0100D236C02FEDCD4E48B359619FB9CFDF4E
Read more
Resolution: 15 Mar 2023, 5:30 PM
Automatic resolution.
Service affected: UniFi
Start: 15 Mar 2023, 5:00 PM
End: 15 Mar 2023, 5:30 PM
Login to request notification
On Premise Exchange Mail Services - Windows OS Patching
Resolved
As part of ongoing and proactive OS patching maintenance, some Exchange services will experience brief service interruptions (mail flow may be queued for a minute whilst an active cluster node performs a service failover to the passive node). There may be a slight mail delay (usually less than a minute). However users should not notice or be affected by the patching.
Resolution: 15 Mar 2023, 5:00 PM
Automatic resolution.
Service affected: Exchange Mail and Calendaring
Start: 15 Mar 2023, 4:00 PM
End: 15 Mar 2023, 5:00 PM
Login to request notification
Oracle Client Patching - conman2, cloud-conman2
Resolved
January 2023 RU Oracle Client Patching - conman2, cloud-conman2
For any individual connections during this window, would need to access via : conman1.compute.dc.uq.edu.au
For any individual connections during this window, would need to access via : conman1.compute.dc.uq.edu.au
Resolution: 15 Mar 2023, 4:00 PM
Automatic resolution.
Service affected: Oracle Development DB Hosting
Start: 15 Mar 2023, 3:00 PM
End: 15 Mar 2023, 4:00 PM
Login to request notification
RCC/QRISCloud service disruption
Resolved
Multiple services are currently having connectivity issues. This is known to be impacting access to RCC HPC resources and QRIScloud storage (RDM Q collections).
RCC staff are working to resolve the issue.
RCC staff are working to resolve the issue.
Resolution: 15 Mar 2023, 4:01 PM
All services restored.
Service affected: RCC/QCIF GPFS Storage
Start: 15 Mar 2023, 2:47 PM
End: 15 Mar 2023, 4:01 PM
Login to request notification
Mailflow Change for ITS Domain
Resolved
Changing outbound mailflow for ITS.uq.edu.au mail domain.
The time and date have been chosen to ensure that the staff required for the change are available.
There is significant mail flow to monitor and test the changes as they are made.
Any issues that may arise can be captured and corrected or the change backed out in a timely fashion.
This initial change has been restricted to the ITS domain to limit any impact.
The time and date have been chosen to ensure that the staff required for the change are available.
There is significant mail flow to monitor and test the changes as they are made.
Any issues that may arise can be captured and corrected or the change backed out in a timely fashion.
This initial change has been restricted to the ITS domain to limit any impact.
Resolution: 15 Mar 2023, 3:00 PM
The Scheduled Change went according to plan and within time.
Update: 15 Mar 2023, 1:56 PM
The scheduled change went according to plan and within time.
Service affected: Mail Routing
Start: 15 Mar 2023, 12:00 PM
End: 15 Mar 2023, 3:00 PM
Login to request notification
Upgrade ESXi version from 6.7 u3 to 7.0 u3 for CCR Nextgen Nutanix cluster
Resolved
Upgrade ESXi version from 6.7 u3 to 7.0 u3 for CCR Nextgen Nutanix cluster - vSphere Datacenter - CCR.
Remaining ESXi hosts as part of the Nutanix cluster: NTNX-CCR02.
The original change 12067 was postponed part way through due to a host failure due to a faulty DIMM.
which as now been replaced
Due to the number of hosts and the time required for each, this change occurs over 2 days to complete the remaining hosts. Four will be upgraded on the first day (Tuesday 14th Feb)and the remaining four on the second (Wednesday 15th March). The hosts are listed in the change scope section for each day.
Remaining ESXi hosts as part of the Nutanix cluster: NTNX-CCR02.
The original change 12067 was postponed part way through due to a host failure due to a faulty DIMM.
which as now been replaced
Due to the number of hosts and the time required for each, this change occurs over 2 days to complete the remaining hosts. Four will be upgraded on the first day (Tuesday 14th Feb)and the remaining four on the second (Wednesday 15th March). The hosts are listed in the change scope section for each day.
Resolution: 16 Mar 2023, 1:00 AM
Automatic resolution.
Service affected: VMware NextGEN CCR DC
Start: 14 Mar 2023, 5:00 PM
End: 16 Mar 2023, 1:00 AM
Login to request notification
IMB PABX outage due to electrical storm
Resolved
Faults reported for TDM extensions at IMB. Isolated to physical handsets (Anlog & Digital) on IMB PABX only. Retstarted PIMs to restore services. Appears to have been caused by the electrical storm 12/3/23
Resolution: 13 Mar 2023, 11:00 AM
Automatic resolution.
Service affected: PABX Peripheral Support
Start: 13 Mar 2023, 10:00 AM
End: 13 Mar 2023, 11:00 AM
Login to request notification
Wireless Service outage - Wireless router reload
Resolved
Wireless services will be affected for 30 mins due to a reload required for the wireless router Megatron, to implement configuration changes
Resolution: 12 Mar 2023, 6:30 AM
Automatic resolution.
Service affected: Wireless Network
Start: 12 Mar 2023, 6:00 AM
End: 12 Mar 2023, 6:30 AM
Login to request notification
Switchover Some Databases Back to CCR
Resolved
Switchover Databases from magpie-cluster (PRN) to plover-cluster (CCR) .
approx. 5 minute minor disruption per database during switchover from PRN to CCR
approx. 5 minute minor disruption per database during switchover from PRN to CCR
Resolution: 12 Mar 2023, 8:00 AM
Automatic resolution.
Service affected: Oracle Production DB Hosting
Start: 12 Mar 2023, 5:30 AM
End: 12 Mar 2023, 8:00 AM
Login to request notification
Workday 2023R1
Resolved
Workday will be unavailable while the first bi-annual release for 2023 is applied.
Resolution: 11 Mar 2023, 8:00 PM
update completed
Resolution: 11 Mar 2023, 8:00 PM
update completed
Resolution: 11 Mar 2023, 8:00 PM
update completed
Service affected: Workday - Human Capital Management System
Start: 11 Mar 2023, 5:00 PM
End: 11 Mar 2023, 8:00 PM
Login to request notification
Exadata Platinum patching - plover-cluster
Resolved
Patch the CCR Exadata with the latest quarterly Oracle Critical Patch Updates (JAN 2023 CPU).
Resolution: 11 Mar 2023, 8:00 PM
Automatic resolution.
Service affected: Oracle Production DB Hosting
Start: 11 Mar 2023, 8:00 AM
End: 11 Mar 2023, 8:00 PM
Login to request notification
Switchover Databases to PRN and apply latest oracle patch
Resolved
Switchover Databases from plover-cluster (CCR) to magpie-cluster (PRN).
approx. 5 minute minor disruption per database during switchover from CCR to PRN.
approx. 5 minute minor disruption per database during switchover from CCR to PRN.
Resolution: 11 Mar 2023, 8:00 AM
Automatic resolution.
Service affected: Oracle Production DB Hosting
Start: 11 Mar 2023, 5:30 AM
End: 11 Mar 2023, 8:00 AM
Login to request notification
On Premise Exchange Mail Services - Exchange Patching
Resolved
Apply Security Updates for Microsoft Exchange Server 2013 (February 2023).
Resolution: 9 Mar 2023, 5:00 PM
Automatic resolution.
Service affected: Exchange Mail and Calendaring
Start: 9 Mar 2023, 4:00 PM
End: 9 Mar 2023, 5:00 PM
Login to request notification
User directory sync issues
Resolved
User sync issues have been found in the UQ Digital Workspace environment after scheduled maintenance.
A re-sync of all users is occurring to re-sync user permissions for both desktops and applications.
ITS will continue to monitor the service for further interruptions.
A service request with the vendor has been raised to investigate.
A re-sync of all users is occurring to re-sync user permissions for both desktops and applications.
ITS will continue to monitor the service for further interruptions.
A service request with the vendor has been raised to investigate.
Resolution: 10 Mar 2023, 3:57 PM
VMware resolved the issue.
Service affected: UQ Digital Workspace
Start: 9 Mar 2023, 10:00 AM
End: 10 Mar 2023, 3:57 PM
Login to request notification
Flexnet/Portal Down
Resolved
Flexnet Webpage is unavailable. Vendor is investigating.
Resolution: 9 Mar 2023, 9:24 AM
An issue with the SQL Cluster fail over for the FNMSCompliance database has been corrected.
Service affected: FlexNet
Start: 9 Mar 2023, 9:24 AM
End: 9 Mar 2023, 9:24 AM
Login to request notification
Workspace One Access connector upgrade
Resolved
Workspace One Access / Identity Manager upgrade is occurring due to The 19.03.0.1VMware Identity Manager connector reaching the end of extended support on August 31, 2022.
From the 15th March Workspace ONE Access tenants with 19.03.0.1 and earlier Connectors no longer will be able to create, edit, or delete directories.
Workspace ONE Access SaaS customers must upgrade to a supported version of the Workspace ONE Access connector to ensure the continued functionality of all features. Workspace ONE Access customers are strongly encouraged to migrate to the latest Connector (version 22.09.1.0) as soon as possible.
From the 15th March Workspace ONE Access tenants with 19.03.0.1 and earlier Connectors no longer will be able to create, edit, or delete directories.
Workspace ONE Access SaaS customers must upgrade to a supported version of the Workspace ONE Access connector to ensure the continued functionality of all features. Workspace ONE Access customers are strongly encouraged to migrate to the latest Connector (version 22.09.1.0) as soon as possible.
Resolution: 8 Mar 2023, 11:00 PM
Automatic resolution.
Service affected: UQ Digital Workspace
Start: 8 Mar 2023, 8:30 PM
End: 8 Mar 2023, 11:00 PM
Login to request notification
Weekly Production Migration for SI-net
Resolved
ENTSS Migrator will be migrating new codes changes to Production for SI-net applications
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-07+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrat...e the changes into Production.
This service view will be marked to complete as soon as the migration completes.
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-03-07+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrat...e the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Read more
Resolution: 8 Mar 2023, 5:30 PM
Automatic resolution.
Service affected: SI-net
Start: 8 Mar 2023, 5:00 PM
End: 8 Mar 2023, 5:30 PM
Login to request notification
Nutanix Host Failure, VM Hosts reloaded
Resolved
The Nutanix host - ntnx-55vt8x2-a.inf.dc.uq.edu.au failed due to a memory fault. VMs reloaded onto other hosts.
Update: 9 Mar 2023, 7:48 AM
VM's affected -
uq-admgmt02 is disconnected Information 03/08/2023, 4:58:40 PM uq-admgmt02 vim.event.VmDisconnectedEvent
atlas-ghranite is disconnected Information 03/08/2023, 4:58:40 PM atlas-ghranite vim.event.VmDisconnectedEvent
uq-m365-mig-2 is disconnected Information 03/08/2023, 4:58:40 PM uq-m365-mig-2 vim.event.VmDisconnectedEvent
PF-DIR-HER02 is disconnected Information 03/08/2023, 4:58:40 PM PF-DIR-HER02 vim.event.VmDisconnectedEvent
hamachi is disconnected Information 03/08/2023, 4:58:40 PM hamachi vim.event.VmDisconnectedEvent
curlew is disconnected Information 03/08/2023, 4:58:40 PM curlew vim.event.VmDisconnectedEvent
ca-ems1 is disconnected Information 03/08/2023, 4:58:40 PM ca-ems1 vim.event.VmDisconnectedEvent
ldap-prod-b02 is disconnected Information 03/08/2023, 4:58:40 PM ldap-prod-b02 vim.event.VmDisconnectedEvent
sa-pum03 is disconnected Information 03/08/2023, 4:58:40 PM sa-pum03 vim.event.VmDisconnectedEvent
NTNX-55WR8X2-A-CVM is disconnected Information 03/08/2023, 4:58:40 PM NTNX-55WR8X2-A-CVM vim.event.VmDisconnectedEvent
qaafi-hygain01 is disconnected Information 03/08/2023, 4:58:40 PM qaafi-hygain01 vim.event.VmDisconnectedEvent
uqp-ps-prod is disconnected Information 03/08/2023, 4:58:40 PM uqp-ps-prod vim.event.VmDisconnectedEvent
mis-transfer is disconnected Information 03/08/2023, 4:58:40 PM mis-transfer vim.event.VmDisconnectedEvent
uqp-booknet is disconnected Information 03/08/2023, 4:58:40 PM uqp-booknet vim.event.VmDisconnectedEvent
uq-license2 is disconnected Information 03/08/2023, 4:58:40 PM uq-license2 vim.event.VmDisconnectedEvent
pf-historian21 is disconnected Information 03/08/2023, 4:58:40 PM pf-historian21 vim.event.VmDisconnectedEvent
ise-2.3.0.298.SPA.x86_64-template is disconnected Information 03/08/2023, 4:58:40 PM ise-2.3.0.298.SPA.x86_64-template vim.event.VmDisconnectedEvent
its-spctlnk-ftp is disconnected Information 03/08/2023, 4:58:40 PM its-spctlnk-ftp vim.event.VmDisconnectedEvent
mandarin is disconnected Information 03/08/2023, 4:58:40 PM mandarin vim.event.VmDisconnectedEvent
its-ss-jenkins is disconnected Information 03/08/2023, 4:58:40 PM its-ss-jenkins vim.event.VmDisconnectedEvent
uq-adcsweb02 is disconnected Information 03/08/2023, 4:58:40 PM uq-adcsweb02 vim.event.VmDisconnectedEvent
its-nc-mgmt03 is disconnected Information 03/08/2023, 4:58:40 PM its-nc-mgmt03 vim.event.VmDisconnectedEvent
uq-m365-mig-16 is disconnected Information 03/08/2023, 4:58:40 PM uq-m365-mig-16 vim.event.VmDisconnectedEvent
copperhead is disconnected Information 03/08/2023, 4:58:40 PM copperhead vim.event.VmDisconnectedEvent
aurwhh02 is disconnected Information 03/08/2023, 4:58:40 PM aurwhh02 vim.event.VmDisconnectedEvent
pomelo is disconnected Information 03/08/2023, 4:58:40 PM pomelo vim.event.VmDisconnectedEvent
burleighheads is disconnected Information 03/08/2023, 4:58:40 PM burleighheads vim.event.VmDisconnectedEvent
sa-pum05 is disconnected Information 03/08/2023, 4:58:40 PM sa-pum05 vim.event.VmDisconnectedEvent
uq-chkbox01 is disconnected Information 03/08/2023, 4:58:40 PM uq-chkbox01 vim.event.VmDisconnectedEvent
its-gas1 is disconnected Information 03/08/2023, 4:58:40 PM its-gas1 vim.event.VmDisconnectedEvent
svs-acarcinom is disconnected Information 03/08/2023, 4:58:40 PM svs-acarcinom vim.event.VmDisconnectedEvent
pl-8-bkp is disconnected Information 03/08/2023, 4:58:40 PM pl-8-bkp vim.event.VmDisconnectedEvent
mintslice is disconnected Information 03/08/2023, 4:58:40 PM mintslice vim.event.VmDisconnectedEvent
mis-vs-dev1 is disconnected Information 03/08/2023, 4:58:40 PM mis-vs-dev1 vim.event.VmDisconnectedEvent
intelmqprod is disconnected Information 03/08/2023, 4:58:40 PM intelmqprod vim.event.VmDisconnectedEvent
gamora is disconnected Information 03/08/2023, 4:58:40 PM gamora vim.event.VmDisconnectedEvent
sasd02 is disconnected Information 03/08/2023, 4:58:40 PM sasd02 vim.event.VmDisconnectedEvent
bloodhound is disconnected Information 03/08/2023, 4:58:40 PM bloodhound vim.event.VmDisconnectedEvent
fswd02 is disconnected Information 03/08/2023, 4:58:40 PM fswd02 vim.event.VmDisconnectedEvent
jupyter is disconnected Information 03/08/2023, 4:58:40 PM jupyter vim.event.VmDisconnectedEvent
fsas02 is disconnected Information 03/08/2023, 4:58:40 PM fsas02 vim.event.VmDisconnectedEvent
uq-adfsp01-t is disconnected Information 03/08/2023, 4:58:40 PM uq-adfsp01-t vim.event.VmDisconnectedEvent
garter is disconnected Information 03/08/2023, 4:58:40 PM garter vim.event.VmDisconnectedEvent
hsu-test-dp8 is disconnected Information 03/08/2023, 4:58:40 PM hsu-test-dp8 vim.event.VmDisconnectedEvent
smi-digdb01 is disconnected Information 03/08/2023, 4:58:40 PM smi-digdb01 vim.event.VmDisconnectedEvent
bdds-dev is disconnected Information 03/08/2023, 4:58:40 PM bdds-dev vim.event.VmDisconnectedEvent
fn-auto is disconnected Information 03/08/2023, 4:58:40 PM fn-auto vim.event.VmDisconnectedEvent
ca-london is disconnected Information 03/08/2023, 4:58:40 PM ca-london vim.event.VmDisconnectedEvent
cubone is disconnected Information 03/08/2023, 4:58:40 PM cubone vim.event.VmDisconnectedEvent
ushas-win is disconnected Information 03/08/2023, 4:58:40 PM ushas-win vim.event.VmDisconnectedEvent
pf-parkguide2 is disconnected Information 03/08/2023, 4:58:40 PM pf-parkguide2 vim.event.VmDisconnectedEvent
tea is disconnected Information 03/08/2023, 4:58:40 PM tea vim.event.VmDisconnectedEvent
londinium is disconnected Information 03/08/2023, 4:58:40 PM londinium vim.event.VmDisconnectedEvent
dragonfruit4 is disconnected Information 03/08/2023, 4:58:40 PM dragonfruit4 vim.event.VmDisconnectedEvent
doi-tern1 is disconnected Information 03/08/2023, 4:58:40 PM doi-tern1 vim.event.VmDisconnectedEvent
krbs-prod-b01 is disconnected
uqp-dsfwebprod is disconnected Information 03/08/2023, 4:58:40 PM uqp-dsfwebprod vim.event.VmDisconnectedEvent
Doing updates
dapp-ng is disconnected Information 03/08/2023, 4:58:40 PM dapp-ng vim.event.VmDisconnectedEvent
Dapp.compute ???
smi-digdb01 is disconnected Information 03/08/2023, 4:58:40 PM smi-digdb01 vim.event.VmDisconnectedEven
Resolution: 8 Mar 2023, 5:30 PM
Automatic resolution.
Service affected: VMware NextGEN CCR DC
Start: 8 Mar 2023, 4:48 PM
End: 8 Mar 2023, 5:30 PM
Login to request notification
Aarnet Service outage to GPN DC - RCC Tape Libaries impacted
Resolved
Fibre leadin for Aarnet to GPN was partially damaged. THis affected RCC FC tape libaries (old and new). All other active service failed over to Prentice side.
Servcie restored now after repatching Aarnet leadin from GPN to Building 80 to a diferent fibre pair.
Servcie restored now after repatching Aarnet leadin from GPN to Building 80 to a diferent fibre pair.
Resolution: 8 Mar 2023, 11:47 AM
Automatic resolution.
Service affected: Networks
Start: 8 Mar 2023, 10:58 AM
End: 8 Mar 2023, 11:47 AM
Login to request notification
Upgrade ESXi version from 6.7 u3 to 7.0 u3 for CCR Nextgen Nutanix cluster
Resolved
Upgrade ESXi version from 6.7 u3 to 7.0 u3 for CCR Nextgen Nutanix cluster - vSphere Datacenter - CCR.
All ESXi hosts as part of the Nutanix cluster: NTNX-CCR02
Due to the number of hosts and the time required for each, this change occurs over 3 days. Three will be upgraded on the first day (Tuesday 7th Feb), four on the second (Wednesday 8th March) and four on the third day Thursday 9th March. The hosts are listed in the change scope section for each day.
All ESXi hosts as part of the Nutanix cluster: NTNX-CCR02
Due to the number of hosts and the time required for each, this change occurs over 3 days. Three will be upgraded on the first day (Tuesday 7th Feb), four on the second (Wednesday 8th March) and four on the third day Thursday 9th March. The hosts are listed in the change scope section for each day.
Resolution: 9 Mar 2023, 5:00 PM
Automatic resolution.
Service affected: VMware NextGEN CCR DC
Start: 7 Mar 2023, 7:30 AM
End: 9 Mar 2023, 5:00 PM
Login to request notification
Expand vSAN Cluster - vSAN-Prentice01
Resolved
This CR will add 2 nodes to the current 3 node VMWare vSAN cluster servicing NGDC PRN VMware farm vSAN-Prentice01.
Resolution: 6 Mar 2023, 3:00 PM
Automatic resolution.
Service affected: VMware NextGEN PRN DC
Start: 6 Mar 2023, 2:00 PM
End: 6 Mar 2023, 3:00 PM
Login to request notification
Expand vSAN Cluster - vSAN-CCR01
Resolved
This CR will add 2 nodes to the current 3 node VMWare vSAN cluster servicing NGDC CCR VMware farm - vSAN-CCR.
Resolution: 6 Mar 2023, 2:00 PM
Automatic resolution.
Service affected: VMware NextGEN CCR DC
Start: 6 Mar 2023, 1:00 PM
End: 6 Mar 2023, 2:00 PM
Login to request notification
Apply vSphere Appliance critical security updates - vCenter Server 7.0.3.01300
Resolved
Apply vCenter Server Appliance 7.0 Update 3 v7.0.3.01300 update to the vCenter server (With embedded PSC) for the following sites:
VMware NextGEN PRN DC (Infrastructure)
VMware NextGEN CCR DC (Infrastructure)
VMware Polaris DC (Infrastructure)
VMware GPN DC (Infrastructure)
itsosfpvc.inf.dc.uq.edu.au (Infrastructure)
itsresvc.research.dc.uq.edu.au
VMware NextGEN PRN DC (Infrastructure)
VMware NextGEN CCR DC (Infrastructure)
VMware Polaris DC (Infrastructure)
VMware GPN DC (Infrastructure)
itsosfpvc.inf.dc.uq.edu.au (Infrastructure)
itsresvc.research.dc.uq.edu.au
Resolution: 6 Mar 2023, 10:00 AM
Automatic resolution.
Service affected: VMware NextGEN CCR DC
Start: 6 Mar 2023, 7:00 AM
End: 6 Mar 2023, 10:00 AM
Login to request notification
Network services impacted due to loss of VLANS
Resolved
Network services have been impacted for AV, EAIT, and commercials groups due to vlan changes affecting switches teh changes were not meant for.
Changes were rolled back
Changes were rolled back
Resolution: 6 Mar 2023, 2:41 PM
Automatic resolution.
Service affected: Networks
Start: 5 Mar 2023, 3:00 PM
End: 6 Mar 2023, 2:41 PM
Login to request notification
Exadata Platinum patching - magpie-cluster
Resolved
Patch the Prentice Exadata with the latest quarterly Oracle Critical Patch Updates (JAN 2023 CPU).
Resolution: 5 Mar 2023, 5:00 PM
Automatic resolution.
Service affected: Oracle Production DB Hosting
Start: 5 Mar 2023, 8:00 AM
End: 5 Mar 2023, 5:00 PM
Login to request notification
Switchover Databases to CCR
Resolved
Switchover Databases from magpie-cluster (PRN) to plover-cluster (CCR).
approx. 5 minute minor disruption per database during switchover from PRN to CCR.
approx. 5 minute minor disruption per database during switchover from PRN to CCR.
Resolution: 5 Mar 2023, 8:00 AM
Automatic resolution.
Service affected: Oracle Production DB Hosting
Start: 5 Mar 2023, 5:30 AM
End: 5 Mar 2023, 8:00 AM
Login to request notification
SharePoint Cumulative Updates
Resolved
The latest SharePoint Cumulative Updates will be installed in the on-prem SharePoint environment.
Resolution: 4 Mar 2023, 3:00 PM
Automatic resolution.
Service affected: SharePoint
Start: 4 Mar 2023, 7:00 AM
End: 4 Mar 2023, 3:00 PM
Login to request notification
SCCM 2211 Upgrade
Resolved
Scheduled SCCM upgrade from version 2207 to 2211.
SCCM services and Software Centre will be unavailable during this time.
SCCM services and Software Centre will be unavailable during this time.
Resolution: 3 Mar 2023, 9:00 PM
Automatic resolution.
Service affected: System Center Configuration Manager
Start: 3 Mar 2023, 6:00 PM
End: 3 Mar 2023, 9:00 PM
Login to request notification
Deployment of Location API
Resolved
In Unitask, Singapore is not being returned in the country search results. We are deploying temporarily fix until UQMaps fixes the issue.
Location API( which interfaces with UQMaps) will be deployed in production. This will have minimal impact to the users.
Location API( which interfaces with UQMaps) will be deployed in production. This will have minimal impact to the users.
Resolution: 3 Mar 2023, 5:15 PM
Automatic resolution.
Service affected: Unitask
Start: 3 Mar 2023, 5:00 PM
End: 3 Mar 2023, 5:15 PM
Login to request notification
UQ Longpocket Site Power outage
Resolved
The UQ Longpocket Site is experiencing an unscheduled Power outage. P&F investigating.
Resolution: 3 Mar 2023, 2:24 PM
Site power restored
Service affected: Networks
Start: 3 Mar 2023, 1:37 PM
End: 3 Mar 2023, 2:24 PM
Login to request notification
Apply ESXi patches for Prn Nextgen vSAN cluster
Resolved
Apply ESXi 7.0 u3 patches for Prn Nextgen vSAN cluster - vSphere Datacenter - Prentice.
Resolution: 3 Mar 2023, 4:00 PM
Automatic resolution.
Service affected: VMware NextGEN PRN DC
Start: 3 Mar 2023, 12:00 PM
End: 3 Mar 2023, 4:00 PM
Login to request notification
Apply ESXi patches for CCR Nextgen vSAN cluster
Resolved
Apply ESXi 7.0 u3 patches for CCR Nextgen vSAN cluster - vSphere Datacenter - CCR.
Resolution: 3 Mar 2023, 12:00 PM
Automatic resolution.
Service affected: VMware NextGEN CCR DC
Start: 3 Mar 2023, 8:00 AM
End: 3 Mar 2023, 12:00 PM
Login to request notification
Checkbox Website Migration
Resolved
The Checkbox survey websites will be migrated to a new server.
Resolution: 3 Mar 2023, 7:45 AM
Automatic resolution.
Service affected: Checkbox
Start: 3 Mar 2023, 7:30 AM
End: 3 Mar 2023, 7:45 AM
Login to request notification
On Premise Exchange Mail Services - Windows OS Patching
Resolved
As part of ongoing and proactive OS patching maintenance, some Exchange services will experience brief service interruptions (mail flow may be queued for a minute whilst an active cluster node performs a service failover to the passive node). There may be a slight mail delay (usually less than a minute). However users should not notice or be affected by the patching.
Resolution: 2 Mar 2023, 5:00 PM
Automatic resolution.
Service affected: Exchange Mail and Calendaring
Start: 2 Mar 2023, 4:00 PM
End: 2 Mar 2023, 5:00 PM
Login to request notification
Increase default SharePoint Online site / Teams quota to 100GB
Resolved
The M365 SharePoint Online site storage quota will be increased from 50GB to 100GB.
This updated configuration will provide any new SharePoint Online site, any new Microsoft Team, any new Microsoft Team Private or Shared channel, and any new Yammer community with 100GB of storage quota.
Existing SharePoint Online sites, Microsoft Teams and Yammer communities will have their existing storage quotas increased at a later date.
This updated configuration will provide any new SharePoint Online site, any new Microsoft Team, any new Microsoft Team Private or Shared channel, and any new Yammer community with 100GB of storage quota.
Existing SharePoint Online sites, Microsoft Teams and Yammer communities will have their existing storage quotas increased at a later date.
Resolution: 2 Mar 2023, 9:01 AM
Automatic resolution.
Service affected: Office 365 - SharePoint
Start: 2 Mar 2023, 9:00 AM
End: 2 Mar 2023, 9:01 AM
Login to request notification
Data Quality for bulk mailing segments
Resolved
There are approximately 15,000 additional contacts coming through in the Program Enrolled segment for students. Additional students may receive broadcast emails if using segments to create audiences for bulk mailings using Service CRM.
This is being investigated by ITS
This is being investigated by ITS
Resolution: 6 Mar 2023, 10:06 AM
Student data backed-up.
Affected student tables purged.
Data feed disabled.
Relevant student data imported over weekend and numbers checked against expected segments from SI-net.
Resolution: 6 Mar 2023, 10:06 AM
All relevant student data imported over weekend and numbers checked against expected segments from SI-net.
Service affected: Oracle Service Cloud a.k.a Service CRM
Start: 1 Mar 2023, 9:43 AM
End: 6 Mar 2023, 10:06 AM
Login to request notification
Planned disruption for OHS course content on Learn.UQ (Blackboard)
Resolved
Access to the Blackboard/Learn.UQ suite of courses which sit under the heading:NOT FOR STAFF OR HDR - UQ Workplace Inductions and Training OHSB0000N_0000_00000 will be offline during Tuesday night while ITS is updating the OHS course content. Access to other Learn.UQ (Blackboard) courses will be normal during this time.
Examples of what will be affected are: Annual Fire Safety Awareness (OHSB09), Undergraduate Student Laboratory Safety Induction (UGRD01) and all other OHS training modules.
Examples of what will be affected are: Annual Fire Safety Awareness (OHSB09), Undergraduate Student Laboratory Safety Induction (UGRD01) and all other OHS training modules.
Resolution: 28 Feb 2023, 11:00 PM
Automatic resolution.
Service affected: Blackboard Learn
Start: 28 Feb 2023, 9:00 PM
End: 28 Feb 2023, 11:00 PM
Login to request notification
Emergency upgrade for Wireless router - Megatron
Resolved
Wireless services will be unavailable for 1 hr during the change window while the wireless routers are being upgraded. This is an emergency change to address capacity issues affecting services with existing routers due to high wireless client numbers being experienced this semester.
Resolution: 28 Feb 2023, 11:59 PM
Automatic resolution.
Service affected: Wireless Network
Start: 28 Feb 2023, 9:00 PM
End: 28 Feb 2023, 11:59 PM
Login to request notification
Connectivity Issues if not already connected
Resolved
Wireless clients may experience internet issues after connecting to WiFi. This is a suspected capacity issue seen in wireless routers due to number of clients now connecting. Existing,working clients are not impacted.
ITS is configuring higher capapcity router to be installed this morning to take the high loads.
ITS is configuring higher capapcity router to be installed this morning to take the high loads.
Resolution: 28 Feb 2023, 11:33 AM
Automatic resolution.
Service affected: Wireless Network
Start: 28 Feb 2023, 10:15 AM
End: 28 Feb 2023, 11:33 AM
Login to request notification
Emergency reload of wireless routers
Resolved
As per Cisco TAC recommendations, the wireless controllers - Megatron - shall be reloaded to clear suspect memory leak affecting services during peak periods. Wireless service will not be available across all sites for up to 20 minutes between 9:15PM and 11:15PM during the change period.
Resolution: 27 Feb 2023, 11:15 PM
Automatic resolution.
Service affected: Wireless Network
Start: 27 Feb 2023, 9:00 PM
End: 27 Feb 2023, 11:15 PM
Login to request notification
No Internet access for some wireless clients
Resolved
Wireless services are currently being impacted with new devices connecting unable to access the internet.
This is currently under investigation by ITS
This is currently under investigation by ITS
Resolution: 27 Feb 2023, 7:46 PM
All wireless clients now have internet access.
Wireless router memory is now at 12k free space
Service affected: Wireless Network
Start: 27 Feb 2023, 12:00 PM
End: 27 Feb 2023, 7:46 PM
Login to request notification
MyAurion Having Issues
Resolved
The MyAurion service is having issues and will be restarted.
Resolution: 27 Feb 2023, 9:15 AM
Automatic resolution.
Service affected: MyAurion
Start: 27 Feb 2023, 8:45 AM
End: 27 Feb 2023, 9:15 AM
Login to request notification
Aggregation Upgrade project - Herston router line card installation
Resolved
New line cards will be installed in Herston routers. No impact planned. For further information, please call the Project Manager, Keryn Masters 0458 585 405.
Resolution: 26 Feb 2023, 3:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 26 Feb 2023, 10:30 AM
End: 26 Feb 2023, 3:00 PM
Login to request notification
MFA rollout - SI-net
Resolved
This change is to enable Multifactor Authentication for all mySI-net users.
Resolution: 26 Feb 2023, 6:00 PM
Automatic resolution.
Service affected: SI-net
Start: 25 Feb 2023, 6:00 AM
End: 26 Feb 2023, 6:00 PM
Login to request notification
Network outage for (7) Parnell
Resolved
ITS are investigating the issue
Resolution: 22 Feb 2023, 8:45 AM
Power to the communication room was not restored until 8:30am from a failed RCD test that uncovered a fault in the power system.
Service affected: Building Networks
Start: 22 Feb 2023, 7:00 AM
End: 22 Feb 2023, 8:45 AM
Login to request notification
SDA Project - Prentice level 3 pilot
Resolved
Full network outage on level 3 of Prentice while switches are reloaded for SDA pilot. Please avoid the area. For further information, please contact the project manager, Keryn Masters on 0458 585 405.
Resolution: 12 Feb 2023, 5:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 11 Feb 2023, 9:00 AM
End: 12 Feb 2023, 5:00 PM
Login to request notification
United Care Hospital Eduroam Wireless Scheduled Outage
Resolved
Uniting Care QLD will be performing a generator test that will impact the wireless network
Resolution: 10 Feb 2023, 9:45 PM
Automatic resolution.
Service affected: Wireless Network
Start: 10 Feb 2023, 9:00 PM
End: 10 Feb 2023, 9:45 PM
Login to request notification
Weekly Production Migration for SI-net
Resolved
ENTSS Migrator will be migrating new codes changes to Production for SI-net applications
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-02-07+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to... migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-02-07+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to... migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Read more
Resolution: 8 Feb 2023, 5:30 PM
Automatic resolution.
Service affected: SI-net
Start: 8 Feb 2023, 5:00 PM
End: 8 Feb 2023, 5:30 PM
Login to request notification
Weekly Production Migration for UniFi
Resolved
ENTSS Migrator will be migrating new codes changed to Production for both SI-net and UniFi applications. Refer to the latest weekly migration minutes for the list approved changes:
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-02-07+Weekly+Production+Changes
In order to seek for approval to migrate changes to Production, it is required that the required member of the team that made the changes to be present at the weekly migration meeting to present their changes. The attendees ( consists of ENTSS staff and EAPS staff ) will review these before making the decision to approve or pre-approval ( conditional to completion of UAT testing ). Refer to the above hyp...erlink for the attendees who would reviewed and approved the changes to production.
As such this service view will not have any Peer Reviews nor Approval as that would have been completed as outlined above.
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Additional information:
SI-net CAB Pre-Approved Changes (#7317)
https://sharepoint.uq.edu.au/its/ITGovernance/Lists/PreApproved%20Change%20List/DispForm.aspx?ID=45&Source=https%3A%2F%2Fsharepoint%2Euq%2Eedu%2Eau%2Fits%2FITGovernance%2FLists%2FPreApproved%2520Change%2520List%2FAllItems%2Easpx%23InplviewHashc5ffa001%2D2eb0%2D4be2%2D987f%2Dc20e677aea89%3D&ContentTypeId=0x0100D236C02FEDCD4E48B359619FB9CFDF4E
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-02-07+Weekly+Production+Changes
In order to seek for approval to migrate changes to Production, it is required that the required member of the team that made the changes to be present at the weekly migration meeting to present their changes. The attendees ( consists of ENTSS staff and EAPS staff ) will review these before making the decision to approve or pre-approval ( conditional to completion of UAT testing ). Refer to the above hyp...erlink for the attendees who would reviewed and approved the changes to production.
As such this service view will not have any Peer Reviews nor Approval as that would have been completed as outlined above.
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Additional information:
SI-net CAB Pre-Approved Changes (#7317)
https://sharepoint.uq.edu.au/its/ITGovernance/Lists/PreApproved%20Change%20List/DispForm.aspx?ID=45&Source=https%3A%2F%2Fsharepoint%2Euq%2Eedu%2Eau%2Fits%2FITGovernance%2FLists%2FPreApproved%2520Change%2520List%2FAllItems%2Easpx%23InplviewHashc5ffa001%2D2eb0%2D4be2%2D987f%2Dc20e677aea89%3D&ContentTypeId=0x0100D236C02FEDCD4E48B359619FB9CFDF4E
Read more
Resolution: 8 Feb 2023, 5:30 PM
Automatic resolution.
Service affected: UniFi
Start: 8 Feb 2023, 5:00 PM
End: 8 Feb 2023, 5:30 PM
Login to request notification
Flexnet Offline
Resolved
Issue with the Flexnet VM host has caused a service outage
Resolution: 6 Feb 2023, 9:39 AM
Host restored
Service affected: FlexNet
Start: 6 Feb 2023, 9:15 AM
End: 6 Feb 2023, 9:39 AM
Login to request notification
its-isilon02 (nas02) - OS Upgrade + Patches + drive and node firmware
Resolved
Upgrade of cluster OS (OneFS) from 9.2.1.11 to 9.4.0.x (latest roll up patch currently 9.4.0.11)
Resolution: 5 Feb 2023, 5:00 PM
Change completed.
Service affected: nas02 Group & Home File Shares
Start: 5 Feb 2023, 7:00 AM
End: 5 Feb 2023, 5:00 PM
Login to request notification
HPCU Internet Traffic - potential service impact
Resolved
HPCU Internet traffic may be seeing some impact due to a faulty interface between GW20 and GW10.
ITS is onsite and investigating the fault now.
ITS is onsite and investigating the fault now.
Resolution: 5 Feb 2023, 8:35 AM
HPCU Traffic affected as been failed across to another link while the faulty interface between GW10 and GW20 is fixed.
Service affected: Internet Services UQ
Start: 5 Feb 2023, 6:40 AM
End: 5 Feb 2023, 8:35 AM
Login to request notification
St Lucia Campus - Network outage to selected areas in some buildings
Resolved
UPDATE - All restored apart from -
B47, Level - Power off to various rooms, archibus logged
B12 Duhig North Level 4
These building will be impacted with loss of network services in part or all due to failed network access switches -
39, 60, 23, 01, 47, 45, 12, 76, 79, 64, 74, 23, 09, 50
Works are underway to fix/replace the failed switches.
B47, Level - Power off to various rooms, archibus logged
B12 Duhig North Level 4
These building will be impacted with loss of network services in part or all due to failed network access switches -
39, 60, 23, 01, 47, 45, 12, 76, 79, 64, 74, 23, 09, 50
Works are underway to fix/replace the failed switches.
Resolution: 14 Feb 2023, 12:18 PM
Resolved
Update: 2 Feb 2023, 11:50 AM
Remaining buildings/comms room affected -
B47, Level - Power off to various rooms, archibus logged
B12 Duhig North Level 4
Service affected: Networks
Start: 2 Feb 2023, 6:00 AM
End: 14 Feb 2023, 12:18 PM
Login to request notification
Weekly Production Migration for SI-net
Resolved
ENTSS Migrator will be migrating new codes changes to Production for SI-net applications
Details of the issue and the resolution is recorded in phire CS001838, JIRA, and confluence.
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-01-31+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/...confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Details of the issue and the resolution is recorded in phire CS001838, JIRA, and confluence.
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-01-31+Weekly+Production+Changes
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then seek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/...confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Read more
Resolution: 2 Feb 2023, 5:30 AM
Automatic resolution.
Service affected: SI-net
Start: 2 Feb 2023, 5:00 AM
End: 2 Feb 2023, 5:30 AM
Login to request notification
Weekly Production Migration for SI-net
Resolved
ENTSS Migrator will be migrating new codes changes to Production for SI-net applications
Details of the issue and the resolution is recorded in phire CS001838, JIRA, and confluence.
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-01-31+Weekly+Production+Changes
https://confluence.its.uq.edu.au/confluence/pages/viewpage.action?pageId=246707312
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then se...ek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Details of the issue and the resolution is recorded in phire CS001838, JIRA, and confluence.
https://confluence.its.uq.edu.au/confluence/display/ESS/2023-01-31+Weekly+Production+Changes
https://confluence.its.uq.edu.au/confluence/pages/viewpage.action?pageId=246707312
If the change is simple enough then email approval will be sufficient. The email distribution should include Tech Masters, Functional Masters, Change Owner ( Func and Tech) and Manager to assess the impact before granting the Pre/approval.
If the change does not require an outage then se...ek the line manager for approval.
Refer to https://confluence.its.uq.edu.au/confluence/x/p220Dg
The Team Change Manager willl contain the Migrator name to migrate the changes into Production.
This service view will be marked to complete as soon as the migration completes.
Read more
Resolution: 1 Feb 2023, 6:00 PM
Automatic resolution.
Service affected: SI-net
Start: 1 Feb 2023, 5:00 PM
End: 1 Feb 2023, 6:00 PM
Login to request notification
Firewalling HPCU Traffic
Resolved
The University of Queensland (UQ) hosts a diverse cohort with complex information technology (IT) infrastructure. With the Higher Education sector being a prime target for cyber-criminal activity it is essential that the cybersecurity mechanisms in place at UQ are efficient and relevant to addressing these cyber threats. It is important for the cyber security and data integrity of UQ that the university appropriately manages the cyber risks associated with its networks and assets. This proposal targets improvements across several core infrastructure capabilities to further support the Universitys IT Security goals.
For this CR we plan to move HPCU networks behind a firewa...ll
Read more
Resolution: 29 Jan 2023, 3:00 PM
Change completed.
Service affected: RCC High Speed Data Centre Network Spine
Start: 29 Jan 2023, 9:00 AM
End: 29 Jan 2023, 3:00 PM
Login to request notification
its-isilon01 (nas01) - OS Upgrade + Patches + drive and node firmware
Resolved
Upgrade of cluster OS (OneFS) from 9.2.1.11 to 9.4.0.x (latest roll up patch currently 9.4.0.11)
Resolution: 29 Jan 2023, 5:00 PM
Completed Successfully
Service affected: nas01 NFS File Storage
Start: 29 Jan 2023, 7:00 AM
End: 29 Jan 2023, 5:00 PM
Login to request notification
UQ Authenticate failing for M365 logins
Resolved
UQ Authenticate is currently unable to authenticate users in to M365 (including Office Mail).
Existing M365 sessions are unaffected.
ITS staff are working to resolve the problem.
Existing M365 sessions are unaffected.
ITS staff are working to resolve the problem.
Resolution: 27 Jan 2023, 7:25 AM
This issue has been resolved.
Resolution: 27 Jan 2023, 7:00 AM
Service affected: UQ Authenticate
Start: 27 Jan 2023, 4:00 AM
End: 27 Jan 2023, 7:25 AM
Login to request notification
Users may intermittently be unable to access multiple Microsoft 365 services
Resolved
Microsoft have identified a potential networking issue and are reviewing service telemetry to determine the next troubleshooting steps.
Microsoft have received reports that the following services are impacted:
-Microsoft Teams
-Exchange Online
-Outlook
-SharePoint Online
-OneDrive
Next update by: Wednesday, January 25, 2023, 7:00 PM
Microsoft have received reports that the following services are impacted:
-Microsoft Teams
-Exchange Online
-Outlook
-SharePoint Online
-OneDrive
Next update by: Wednesday, January 25, 2023, 7:00 PM
Resolution: 25 Jan 2023, 9:00 PM
Microsoft has continued to take additional steps to expedite the recovery process, and their telemetry indicates that the majority of the affected M365 services have now returned to normal.
Update: 25 Jan 2023, 8:26 PM
Microsoft are continuing to monitor the recovery across the service and customers are also reporting mitigation. Microsoft are also connecting the service to additional infrastructure to expedite the recovery process.
Update: 25 Jan 2023, 6:49 PM
Impact is occurring to the following services but is not limited to them:
-Microsoft Teams
-Exchange Online
-Outlook
-SharePoint Online
-OneDrive for Business
-Microsoft Graph
-PowerBi
Current status: Microsoft have isolated the problem to a networking configuration issue, and are analyzing the best mitigation strategy to address it without causing additional impact. Microsoft will provide more information once they have additional information.
Scope of impact: Any user serviced by the affected infrastructure may be unable to access multiple Microsoft 365 services.
Service affected: Office 365 - Tenant Administration
Start: 25 Jan 2023, 5:26 PM
End: 25 Jan 2023, 9:00 PM
Login to request notification
Weekly system maintenance
Resolved
System maintenance
Resolution: 24 Jan 2023, 4:55 PM
Automatic resolution.
Service affected: UniFi
Start: 24 Jan 2023, 4:50 PM
End: 24 Jan 2023, 4:55 PM
Login to request notification
Weekly system maintenance
Resolved
System maintenance
Resolution: 24 Jan 2023, 4:45 PM
Automatic resolution.
Service affected: SI-net
Start: 24 Jan 2023, 4:40 PM
End: 24 Jan 2023, 4:45 PM
Login to request notification
Update IMS configuration to use AES256 for Active Directory Kerberos encryption
Resolved
Update the IMS trust object and the keytab used by IMS hosts, so that the IMS is forced to use AES256 for its Kerberos ticket encryption.
Additionally, the IMS account password will be reset.
Additionally, the IMS account password will be reset.
Resolution: 24 Jan 2023, 9:00 AM
Automatic resolution.
Service affected: Active Directory
Start: 24 Jan 2023, 8:00 AM
End: 24 Jan 2023, 9:00 AM
Login to request notification
Domain-wide policy (GPO) to define support of secure kerberos encryption types
Resolved
This change will deploy a domain-wide policy which will set Windows Server and Desktop Operating Systems to use secure kerberos encryption types when requesting kerberos tickets from the domain controllers.
Secure encryption types that will be allowed by the policy includes AES 256.
Insecure encryption types will not be allowed by the policy applied include RC4.
This GPO will affect all Windows domain-joined computers (including Servers and Workstations) that support secure encryption types. Secure encryption types are not supported in Windows 2000 Server, Windows XP, or Windows Server 2003.
When the windows clients process the policy, they... will update their own Active Directory msDS-SupportedEncryptionTypes attribute to indicate that they only support stronger kerberos encryption.
Secure encryption types that will be allowed by the policy includes AES 256.
Insecure encryption types will not be allowed by the policy applied include RC4.
This GPO will affect all Windows domain-joined computers (including Servers and Workstations) that support secure encryption types. Secure encryption types are not supported in Windows 2000 Server, Windows XP, or Windows Server 2003.
When the windows clients process the policy, they... will update their own Active Directory msDS-SupportedEncryptionTypes attribute to indicate that they only support stronger kerberos encryption.
Read more
Resolution: 24 Jan 2023, 4:00 PM
Automatic resolution.
Service affected: Active Directory
Start: 24 Jan 2023, 8:00 AM
End: 24 Jan 2023, 4:00 PM
Login to request notification
Windows devices experiencing connection problems to UQ SSID
Resolved
Some devices, potentailly only windows based, are unable to connect to the UQ SSID.
As a workaround EDUROAM does work for those devices and can be used.
This is under investigation by ITS.
Apple/Android based do not appear to be affected.
As a workaround EDUROAM does work for those devices and can be used.
This is under investigation by ITS.
Apple/Android based do not appear to be affected.
Resolution: 23 Jan 2023, 10:00 AM
UQ Cert had expired earlier than noted in our cert calendar.
Has been renewed and working again
Service affected: Wireless Network
Start: 23 Jan 2023, 8:00 AM
End: 23 Jan 2023, 10:00 AM
Login to request notification
Aggregation Upgrade project - Gatton router line card installation
Resolved
Additional capacity is being added to the Gatton routers to support upcoming aggregation upgrades. There will be no outage. If any questions, please contact the Project Manager, Keryn Masters 0458 585 405.
Resolution: 22 Jan 2023, 3:30 PM
Automatic resolution.
Service affected: Building Networks
Start: 22 Jan 2023, 10:30 AM
End: 22 Jan 2023, 3:30 PM
Login to request notification
Replace F5 load balancers: CCR
Resolved
ITS engineers will be replacing the F5 Load balancers in CCR DC
Resolution: 22 Jan 2023, 5:00 PM
Automatic resolution.
Service affected: NGDC Serverfarm Load Balancer
Start: 22 Jan 2023, 7:00 AM
End: 22 Jan 2023, 5:00 PM
Login to request notification
Replace F5 load balancers: PRN
Resolved
ITS engineers will be replacing the F5 Load balancers in PRN DC
Resolution: 21 Jan 2023, 6:00 PM
Automatic resolution.
Service affected: NGDC Serverfarm Load Balancer
Start: 21 Jan 2023, 8:00 AM
End: 21 Jan 2023, 6:00 PM
Login to request notification
Aggregation Upgrade Project - Network outage Hibiscus and Kurrajong buildings
Resolved
ITS are building a new communications cabinet in the Hibiscus building. The communications rack is being replaced. The network outage will impact the connected Kurrajong building also. Please avoid the area as alarms will not be active. For further information, please call the Project Manager, Keryn Masters on 0458 585 405.
Resolution: 21 Jan 2023, 4:00 PM
Automatic resolution.
Service affected: Building Networks
Start: 21 Jan 2023, 6:00 AM
End: 21 Jan 2023, 4:00 PM
Login to request notification
No access to Workday while maintenance is performed
Resolved
No access to Workday while maintenance is performed
Resolution: 21 Jan 2023, 2:00 AM
Automatic resolution.
Service affected: Workday - Human Capital Management System
Start: 20 Jan 2023, 5:00 PM
End: 21 Jan 2023, 2:00 AM
Login to request notification
Switchover Aurion Database to PRN
Resolved
Switchover Aurion Database to PRN
Resolution: 19 Jan 2023, 7:00 PM
Due to scheduled patching changes, this change will be cancelled
Update: 13 Jan 2023, 3:27 PM
Due to scheduled patching changes, this change will be cancelled
Service affected: Oracle Production DB Hosting
Start: 19 Jan 2023, 6:30 PM
End: 19 Jan 2023, 7:00 PM
Login to request notification
Gatton Campus network link in non-redundant state
Resolved
Hazardous awareness only - Due to an Aarnet fibre cut, the Gatton campus is now actively on the secondary link.
Redundancy is expected to to back in place around midnight.
Redundancy is expected to to back in place around midnight.
Resolution: 20 Jan 2023, 6:00 AM
Aarnet has repaired the fibre
Service affected: Networks
Start: 19 Jan 2023, 6:06 PM
End: 20 Jan 2023, 6:00 AM
Login to request notification
On Premise Exchange Mail Services - Exchange Patching
Resolved
Apply Security Updates for Microsoft Exchange Server 2013 (January 2023).
Resolution: 19 Jan 2023, 5:00 PM
Automatic resolution.
Service affected: Exchange Mail and Calendaring
Start: 19 Jan 2023, 4:00 PM
End: 19 Jan 2023, 5:00 PM
Login to request notification
Network Services to Polaris are in Non-Redundant state
Resolved
Hazard Awareness Only - Due to damage to Aarnet fibre, Network services to Polaris are now running in a non-redundant state.
No users based services will be affected.
No users based services will be affected.
Resolution: 20 Jan 2023, 6:00 AM
Aarnet has repaired the fibre
Service affected: Polaris Ethernet Fabric
Start: 19 Jan 2023, 1:47 PM
End: 20 Jan 2023, 6:00 AM
Login to request notification