There are no current notices.

SQL 2014 Clustered Database Services - OS Patching

Resolved
This outage is to facilitate regular monthly operating system updates, security enhancements, and overall service improvements.

Clients and applications with databases hosted on the Microsoft SQL Server 2014 cluster, will experience a very brief service interruption whilst database instances are failed over between ITS data centres.

The service interruption duration is generally 30-60 seconds within the designated outage window. Affected applications and services typically reconnect automatically.
Resolution: 19 Jul 2019, 6:30 AM
Automatic resolution.
Service affected: SQL 2014 Clustered Database Hosting

Start: 19 Jul 2019, 6:00 AM
End: 19 Jul 2019, 6:30 AM
Login to request notification

UQ RDM HPC Collections Issue

Resolved
File access to UQ RDM collections connected to HPC storage may experience intermittent slow file access and the sync process between HPC file systems and UQ RDM is currently unavailable.
Resolution: 10 Jul 2019, 10:10 AM
HPC enabled UQ RDM collections are now able to sync to HPC file systems
Service affected: UQ Research Data Manager

Start: 9 Jul 2019, 3:48 PM
End: 10 Jul 2019, 10:10 AM
Login to request notification

Blackboard upgrade

Resolved
Upgrade Blackboard to q42018cu6
Resolution: 7 Jul 2019, 12:15 PM
Blackboard upgraded successfully to Q4 2018 CU6
Service affected: Blackboard

Start: 6 Jul 2019, 9:00 AM
End: 7 Jul 2019, 12:15 PM
Login to request notification

Emergency reboot of Jira server to fix temp space issues

Resolved
A reboot of the Jira Servers are required to fix a temp space issue.
Resolution: 5 Jul 2019, 5:40 PM
Reboot of Jira servers has been completed. Access to Jira has been tested and is operational.
Resolution: 5 Jul 2019, 5:40 PM
Resolution: 5 Jul 2019, 5:40 PM
Automatic resolution.
Resolution: 5 Jul 2019, 5:40 PM
Automatic resolution.
Service affected: ITS Jira

Start: 5 Jul 2019, 5:30 PM
End: 5 Jul 2019, 5:40 PM
Login to request notification

Adobe Internal Update Server Db Corruption

Resolved
A issue was found with the Adobe catalog that provides products to the Adobe Creative Cloud Client for download.

Users may see a download error when trying to install Adobe products.

A full re-sync of the catalog is currently being done on the server.

The sync may take several hours to complete.

If users require software today Fleet Management can provide work around on removing the config from using the internal server to connect externally over the internet.
Resolution: 3 Jul 2019, 6:00 AM
Sync with adobe services has completed. Adobe Creative Cloud clients can now download and use Adobe products.
Update: 2 Jul 2019, 3:22 PM
The server has processed 1/3 of the applications. We expect this to be completed by 7am Wednesday 03/07/2019
Service affected: Adobe Internal Update Server

Start: 2 Jul 2019, 10:07 AM
End: 3 Jul 2019, 6:00 AM
Login to request notification

Switchover Patched Databases Back to plover

Resolved
Switchover Patched Databases Back to plover-cluster.

approx. 5 minute minor disruption per database during switchover from Prentice to CCR.
Resolution: 30 Jun 2019, 10:00 AM
Automatic resolution.
Service affected: Oracle Production DB Hosting

Start: 30 Jun 2019, 8:00 AM
End: 30 Jun 2019, 10:00 AM
Login to request notification

Exadata Platinum patching - plover-cluster

Resolved
Patch the CCR Exadata with the Apr 2019 Bundle Patch.



Resolution: 29 Jun 2019, 8:00 PM
Automatic resolution.
Service affected: Oracle Production DB Hosting

Start: 29 Jun 2019, 8:00 AM
End: 29 Jun 2019, 8:00 PM
Login to request notification

Switchover Databases to magpie-cluster

Resolved
Switchover Databases to magpie-cluster.

approx. 5 minute minor disruption per database during switchover from CCR to Prentice.

Resolution: 29 Jun 2019, 10:00 AM
Automatic resolution.
Service affected: Oracle Production DB Hosting

Start: 29 Jun 2019, 6:00 AM
End: 29 Jun 2019, 10:00 AM
Login to request notification

UQ Book It Certificate Expired

Resolved
The security certificate for UQ Book It has expired and were working with the product vendor to apply a newly signed UQ certificate for use.
Resolution: 1 Jul 2019, 12:30 PM
A new certificate has been applied
Service affected: UQ Bookit

Start: 28 Jun 2019, 10:32 AM
End: 1 Jul 2019, 12:30 PM
Login to request notification

Switchover AURPROD to magpie-cluster

Resolved
Switchover Databases to magpie-cluster.

approx. 5 minute minor disruption during switchover from CCR to Prentice.

Resolution: 27 Jun 2019, 7:05 PM
Automatic resolution.
Service affected: Aurion

Start: 27 Jun 2019, 7:00 PM
End: 27 Jun 2019, 7:05 PM
Login to request notification

Maintenance of Unitask

Resolved
New set of permissions will be added in preparation for the deployment of new APIs used in the OH&S and Student Administration process. Staff and students may experience issues raising and processing requests from myUQ and UniTask.

Support after the release

Some users might experience issues submitting tasks in myUQ and authenticating into Unitask. To solve this, users should clear the browser cache before login into myUQ or Unitask using their appropriate staff or student account.

Technical information

The permissions for certified documents BDM save and get task or overview link will be applied to the Bonita installation.
Resolution: 26 Jun 2019, 6:30 PM
Automatic resolution.
Service affected: Unitask

Start: 26 Jun 2019, 6:00 PM
End: 26 Jun 2019, 6:30 PM
Login to request notification

Exadata Platinum patching - magpie-cluster

Resolved
Patch the PRN Exadata with the Apr 2019 Bundle Patch.

Resolution: 23 Jun 2019, 4:00 PM
The Prentice Exadata (magpie-cluster) is now patched to Apr 2019 Quarterly Full Stack Download Patch (QFSDP).
Service affected: Oracle Production DB Hosting

Start: 23 Jun 2019, 4:00 AM
End: 23 Jun 2019, 4:00 PM
Login to request notification

Switchover Databases to CCR Exadata

Resolved
Switchover Databases to plover-cluster to allow for the patching of magpie-cluster

approx. 5 minute minor disruption per database during switchover from Prentice to CCR.

Resolution: 22 Jun 2019, 7:00 AM
Automatic resolution.
Service affected: Oracle Production DB Hosting

Start: 22 Jun 2019, 6:00 AM
End: 22 Jun 2019, 7:00 AM
Login to request notification

Maintenance of Unitask

Resolved
Unitask application servers will be patched to resolve some security vulnerabilities.

Staff and students may experience issues raising and processing requests from myUQ and UniTask.

Support after the release

Some users might experience issues submitting tasks in myUQ and authenticating into Unitask. To solve this, users should clear the browser cache before login into myUQ or Unitask using their appropriate staff or student account.

Technical Information

One important and two moderate kernel vulnerabilities were identified affecting Linux systems and a patch needs to be applied to the application instances. https://access.redha...t.com/security/vulnerabilities/tcpsack

Read more
Resolution: 21 Jun 2019, 6:30 PM
Automatic resolution.
Service affected: Unitask

Start: 21 Jun 2019, 6:00 PM
End: 21 Jun 2019, 6:30 PM
Login to request notification

Heron Island Research Station power works

Resolved
The HIRS is undergoing scheduled power works on the island this week and subsequently the network on the Island is currently unreachable.
Resolution: 22 Jun 2019, 11:59 PM
Automatic resolution.
Service affected: Building Networks

Start: 21 Jun 2019, 1:30 PM
End: 22 Jun 2019, 11:59 PM
Login to request notification

Hazardous Works - Third Party AWS Direct Connect planned maintenance

Resolved
Planned maintenance has been scheduled on an AWS Direct Connect router in Global Switch SY6, Sydney, AUS. During this maintenance window, your AWS Direct Connect services associated with this event may become unavailable.

This maintenance is scheduled to avoid disrupting redundant connections at the same time.

If you encounter any problems with your connection after the end of this maintenance window, please contact us at https://aws.amazon.com/support

Resolution: 21 Jun 2019, 6:01 AM
Automatic resolution.
Service affected: Amazon Web Services - Direct Connect

Start: 21 Jun 2019, 12:01 AM
End: 21 Jun 2019, 6:01 AM
Login to request notification

ServiceView planned emergency maintenance

Resolved
ServiceView will be unavailable for up to 10 minutes in order for patches to be applied.

Users working in ServiceView during this time are advised to guard against possible loss of their work by saving it to an external text editor.

Information relating to service outages on the Service Status page will also be unavailable while ServiceView maintenance is under way.
Resolution: 20 Jun 2019, 5:40 PM
Automatic resolution.
Service affected: ServiceView

Start: 20 Jun 2019, 5:30 PM
End: 20 Jun 2019, 5:40 PM
Login to request notification

Service CRM (Oracle Service Cloud) unavailable.

Resolved
Users of Service CRM (Oracle Service Cloud) are unable to open incidents/enquiry records.

The system interface may hang or freeze when attempting to open a record.

CRM team has contacted vendor and is investigating
Resolution: 20 Jun 2019, 11:26 AM
Being investigated by vendor
Service affected: Oracle Cloud Services incl. CRM and Marketing Automation

Start: 20 Jun 2019, 11:05 AM
End: 20 Jun 2019, 11:26 AM
Login to request notification

Network outage affecting some of the remote sites connected via TPG

Resolved
Following remotes sites are experiencing a network outage. ITS staff are currently working with the carrier to restore the service.

Customs House
Foundation Year
Toowoomba Rural Clinical Service
Bundaberg - Rural Clinical School
Queenslnad Theological College
Toowoomba Grammer School
Resolution: 18 Jun 2019, 12:45 PM
Fault within the carrier network was rectified. ITS is waiting for a root cause analysis from the carrier.
Service affected: Building Networks

Start: 18 Jun 2019, 12:30 PM
End: 18 Jun 2019, 12:45 PM
Login to request notification

Maintenance of Unitask

Resolved
Removal of the special character filter and a patch to fix a bug in the recent 7.7.5 upgrade

During the change people accessing the system will see a maintenance message and they would not be able to view/action any task.

Support after the release

Some users might experience issues submitting tasks in myUQ and authenticating into Unitask. To solve this, users should clear the browser cache before login into myUQ or Unitask using their appropriate staff or student account.
Resolution: 17 Jun 2019, 6:30 PM
Automatic resolution.
Resolution: 17 Jun 2019, 4:30 PM
Automatic resolution.
Service affected: Unitask

Start: 17 Jun 2019, 6:00 PM
End: 17 Jun 2019, 6:30 PM
Login to request notification

Aurion migration to Oracle 12.2

Resolved
Aurion and MyAurion will be unavailable from 8 PM on Friday 14th June to 6 PM on Sunday 16th June during the migration to Oracle 12.2
Resolution: 16 Jun 2019, 12:00 PM
Aurion and MyAurion have now been upgraded to Oracle 12.2.
Service affected: Aurion

Start: 14 Jun 2019, 8:00 PM
End: 16 Jun 2019, 12:00 PM
Login to request notification

Hazardous Works - Third Party AWS Direct Connect planned maintenance

Resolved
Planned maintenance has been scheduled on an AWS Direct Connect router in Equinix SY1 - SY4, Sydney, AUS. During this maintenance window, your AWS Direct Connect services associated with this event may become unavailable.

This maintenance is scheduled to avoid disrupting redundant connections at the same time.

If you encounter any problems with your connection after the end of this maintenance window, please contact us at https://aws.amazon.com/support
Resolution: 14 Jun 2019, 6:01 AM
Automatic resolution.
Service affected: Amazon Web Services - Direct Connect

Start: 14 Jun 2019, 12:00 AM
End: 14 Jun 2019, 6:01 AM
Login to request notification

Deploy of new process for OH&S

Resolved
A new process for OH&S will be deployed to Unitask.

Staff and students may experience issues raising and processing requests from myUQ and UniTask.

Support after the release

Some users might experience issues submitting tasks in myUQ and authenticating into Unitask. To solve this, users should clear the browser cache before login into myUQ or Unitask using their appropriate staff or student account.

Technical Information

The following will be deployed to UniTask:

* Health, safety and wellness induction checklist
Resolution: 12 Jun 2019, 5:30 PM
The OH&S process was released
Service affected: Unitask

Start: 12 Jun 2019, 5:00 PM
End: 12 Jun 2019, 5:30 PM
Login to request notification

Symantec Endpoint Protection Deployment to Servers

Resolved
Symantec Endpoint Protection is being deployed to Windows servers with regular updates from the 12th until to the 18th of June. The existing Sophos anti-virus will be removed as part of this process.

Whilst no additional service impact is anticipated due to the Symantec deployment, some applications may be negatively impacted. In these cases the deployment team will resolve problems as quickly as possible. As usual, please report any service issues to the IT service desk who will escalate to the deployment team if required.
Resolution: 18 Jun 2019, 2:28 PM
Automatic resolution.
Service affected: Endpoint Protection Anti Virus

Start: 12 Jun 2019, 1:28 PM
End: 18 Jun 2019, 2:28 PM
Login to request notification

Unitask is temporarily unavailable

Resolved
The Unitask Worklist is currently down. The Worklist will not be available for staff users, but students and staff will be able to raise requests and action their tasks from myUQ.
Resolution: 10 Jun 2019, 11:00 AM
The Unitask Worklist is now available to staff users.
Service affected: Unitask

Start: 10 Jun 2019, 8:51 AM
End: 10 Jun 2019, 10:00 AM
Login to request notification

Unitask upgrade

Resolved
Unitask will be upgraded to a more recent version.

During the upgrade people accessing the system will see a maintenance message and they would not be able to view/action any task.

Support after the upgrade

Some users might experience issues submitting tasks in myUQ and authenticating into Unitask. To solve this, users should clear the browser cache before login into myUQ or Unitask using their appropriate staff or student account.

Technical Information

The version of Bonita running on the production environment will be upgraded from v7.3.0 to v.7.7.5.
Resolution: 8 Jun 2019, 11:00 PM
Unitask was successfully upgraded.
Service affected: Unitask

Start: 8 Jun 2019, 8:00 PM
End: 8 Jun 2019, 11:00 PM
Login to request notification

IMS capability upgrade

Resolved
ITS will be performing maintenance and hardware upgrades on the Internet Management Service Thursday night between 8pm and 10pm.

The Internet traffic flow will be changed from the primary (A) path to the secondary (B) path at the beginning of the change. Once the hardware upgrade has been completed the traffic flow will be reverted back to the primary (A) path.

When the traffic path changes, there may be a brief interruption to Internet traffic while the routes converge.
Resolution: 6 Jun 2019, 10:00 PM
Automatic resolution.
Service affected: Internet Management Service

Start: 6 Jun 2019, 8:00 PM
End: 6 Jun 2019, 10:00 PM
Login to request notification

Loss of redundancy to legacy server farm

Resolved
The link from hercules to zeus is in a faulty state and has stopped passing traffic. The alternative link via talon is passing all traffic at the moment so services are not impacted, however if this fails the legacy server farm will become unreachable.

We are investigating the cause and will endeavour to restore redundancy at a time of least impact users.
Resolution: 6 Jun 2019, 9:30 PM
Service affected: Data Centre Network

Start: 5 Jun 2019, 10:15 AM
End: 6 Jun 2019, 9:30 PM
Login to request notification

In Place Unscheduled Outage

Resolved
In Place Unscheduled Outage (Vendor currently investigating)
Resolution: 3 Jun 2019, 4:59 PM
Service restored by Vendor
Service affected: InPlace

Start: 3 Jun 2019, 3:20 PM
End: 3 Jun 2019, 4:59 PM
Login to request notification

Building 98d - Network outage

Resolved
Unscheduled network outage at Building 98d (Conifer Knoll Car park).
ITS Staff Investigating.
Resolution: 2 Jun 2019, 2:24 PM
Replaced Faulty Hardware.
Service affected: Building Networks

Start: 2 Jun 2019, 1:07 PM
End: 2 Jun 2019, 2:24 PM
Login to request notification

Pinjara Hills Link upgrade planned outage

Resolved
Pinjara Hills - Replacing Bronze Router with C9500-24Y4C routers and upgrading the current link to 40 Gig Link.
Resolution: 31 May 2019, 11:00 PM
Pinjarra Hills link upgrade completed successfully.
Service affected: Building Networks

Start: 31 May 2019, 6:00 PM
End: 31 May 2019, 11:00 PM
Login to request notification

Disruption to Internet access for Schoolsnet and Commercial customers

Resolved
There is a disruption to peering network traffic which will impact internet access for Schoolsnet and Commercial customers. Certain websites may be slow to load or not load. This is being investigated now by ITS staff.
Resolution: 30 May 2019, 11:45 AM
Remote switch in Equinox Datacentre has been reloaded. Peering network is still running very congested, however all websites will now be accessible by Schoolsnet and Commercial customers
Service affected: UQSchoolsNet

Start: 30 May 2019, 11:30 AM
End: 30 May 2019, 11:45 AM
Login to request notification

Delays to email delivery

Resolved
Delivery of email was delayed due to an underlying issues. Once the issue was resolved, the queued email was delivered to mailboxes
Resolution: 31 May 2019, 11:30 AM
Automatic resolution.
Service affected: Mail Routing

Start: 30 May 2019, 7:30 AM
End: 31 May 2019, 11:30 AM
Login to request notification

Production Xymon server OS patching

Resolved
OS patching will be performed on the production Xymon server during the outage window.

The development xymon server https://xymon-dev.its.uq.edu.au/ is available for use while the production system is unavailable.
Resolution: 28 May 2019, 12:00 PM
Automatic resolution.
Service affected: Xymon Monitoring System

Start: 28 May 2019, 11:00 AM
End: 28 May 2019, 12:00 PM
Login to request notification

Internet connectivity outage for dark-fibre connected schools

Resolved
UPDATE -

ITS Staff are replacing a faulty UQ switch at the Pipe Data Centre in Brisbane City.

Service expected to be restored by 3:30pm

Schools connecting via the Pipe DC are currently unreachable and can not reach UQ or the Internet.

Schools affected are:
Greggory Terrace
Brisbane Girls Grammar School
Brisbane Boys College
Clayfield College
Update: 28 May 2019, 2:46 PM
Power Issues at a Comms Rack in the Pipe Data Centre caused the circuit powering the UQ switch to trip. Power has been restored and the suspect equipment removed (not UQ's)
Update: 28 May 2019, 2:00 PM
The UQ switch failed again after being restarted and up for about 30 mins. A replacement switch has been configured and ITS Network engineers on route to Pipe DC to install
Resolution: 28 May 2019, 12:23 PM
Automatic resolution.
Resolution: 28 May 2019, 12:23 PM
UQ switch in the Pipe DC was reloaded which recovered connected services. A software fault brought on by a long uptime of the device is the most likely cause.
Service affected: UQSchoolsNet

Start: 28 May 2019, 10:20 AM
End: 28 May 2019, 2:46 PM
Login to request notification

Academic Portal Form A Export Failing

Resolved
Academic Portal Form A Export is Failing. ITS Staff are investigating.
Resolution: 30 May 2019, 11:30 AM
Rundeck was failing to send Online Form A .docx files via email. Rundeck has been temporarily disabled, Online Form A .docx files being sent via EC3 box instead.
Service affected: Academic Portal

Start: 28 May 2019, 10:00 AM
End: 30 May 2019, 11:30 AM
Login to request notification

UQ RDM LinkedIn Authentication Failing

Resolved
External collaborators using LinkedIn are unable to log in to access UQ RDM project data.
Resolution: 28 May 2019, 9:45 PM
External collaborators for UQ RDM are able to use LinkedIn authentication.
Service affected: UQ Research Data Manager

Start: 28 May 2019, 9:00 AM
End: 28 May 2019, 9:45 PM
Login to request notification

Archibus NFS migration

Resolved
Archibus NFS migration from teepee/wagon and samba from soefile to NGDC Isilon
Resolution: 27 May 2019, 7:00 AM
Automatic resolution.
Service affected: ARCHIBUS

Start: 27 May 2019, 6:00 AM
End: 27 May 2019, 7:00 AM
Login to request notification

Network Outage - some UQ remote sites

Resolved
A few UQ Remotes sites lost network connectivity
standrew.router.uq.edu.au
uqhc-varsity.router.uq.edu.au
qaafi-maroochy.router.uq.edu.au
dayboro.router.uq.edu.au
fyear.router.uq.net.au

ITS On Call engineer is investigating.
Resolution: 26 May 2019, 10:53 AM
Service Provider resolved power issue.
Update: 26 May 2019, 10:46 AM
Update from the Internet Service Provider. The AAPT (TGP) had unscheduled power outage at the main distribution rack in this morning. Five UQ remote sites have also been impacted by this event. Currently, the estimated time for resolution is unknown.
Service affected: Networks

Start: 26 May 2019, 8:17 AM
End: 26 May 2019, 10:53 AM
Login to request notification

Aurion and MyAurion not available during maintenance

Resolved
Aurion and MyAurion will not be available during patching and fail-over testing of application servers
Resolution: 25 May 2019, 8:00 AM
Patching and failover testing successfully completed
Update: 25 May 2019, 7:12 AM
Patching and fail-over testing successfully completed
Service affected: Aurion

Start: 25 May 2019, 6:00 AM
End: 25 May 2019, 8:00 AM
Login to request notification

Building Networks Outage - Multiple Buildings at St Lucia Campus

Resolved
Majority of Buildings and Services has now been restored. Final checks now being made.
Resolution: 23 May 2019, 3:58 PM
Services have all been restored
Service affected: Networks

Start: 23 May 2019, 3:52 PM
End: 23 May 2019, 3:58 PM
Login to request notification

Changed to assigned Office 365 license (ITS accounts)

Resolved
As part of the new Microsoft CAUDIT agreement, the assigned Office 365 license type has to be changes. ITS account are the first batch to be assigned the updated license. We are expecting no impact to ITS user accounts.
Resolution: 22 May 2019, 9:00 PM
Automatic resolution.
Service affected: Office 365 Provisioning and Maintenance

Start: 22 May 2019, 6:30 PM
End: 22 May 2019, 9:00 PM
Login to request notification

Office 365 Outage

Resolved
Microsoft has reported a worldwide issue with Exchange that may be affecting some users.

We are trying to contact microsoft for an update.

Current status from Microsoft:
User Impact: Users may be unable to connect to Exchange Online via any protocol.

Current status: Weve identified that a recent code regression within the MailTips feature has inadvertently affected how our systems process some traffic, causing intermittent access issues to Exchange Online through any protocol. Were working to temporarily disable the MailTips feature to mitigate impact. Additionally, we will continue to identify what has caused this regression to occur and develop... a more permanent fix which will allow us to re-enable MailTips functionality.

Scope of impact: This issue may potentially affect any of your users attempting to access Exchange Online via any protocol.

Read more
Resolution: 22 May 2019, 4:00 PM
User who experienced issues accessing Exchange Online should have been resolved by 22/5/19 4.00pm. Microsoft has disabled mailtips from their systems which resolved issues. Users may notice that mailtips are unavailable in their outlook clients. There is no ETA from Microsoft for when this feature will be re-enabled. However, the main issue relating to user access has been resolved.
Resolution: 22 May 2019, 4:00 PM
User who experienced issues accessing Exchange Online should have been resolved by 4.00pm 22/5/19. Microsoft has disabled mailtips from their systems which resolved issues. Users may notice that mailtips are unavailable in their outlook clients. There is no ETA from Microsoft for when this feature will be re-enabled. However, the main issue relating to user access has been resolved.
Resolution: 22 May 2019, 4:00 PM
Automatic resolution.
Update: 22 May 2019, 7:28 PM
Microsoft believe the issues has been mitigated (by disabling the Mailtips in Office 365) and users should no longer be affected by the issues. However, we will keep the service status entry opened for another day to confirm UQ users are good again.
Update: 22 May 2019, 2:31 PM
Microsoft have advised that disabling the MailTips from their end has mitigated the issues. The Service Desk has been contact clients to confirm their issues have been resolved.
Update: 22 May 2019, 1:38 PM
Microsoft has advise us the issues should be resolving for users. Microsoft has no ETA on when the issue will be full resolved.
Update: 22 May 2019, 11:24 AM
We have talked to Microsoft and the issue has not been resolved. We cannot tell the number of people who have been affected. Currently there has been 35 Service Desk tickets logged. The next update from Microsoft is scheuled at 3pm AEST. Some users have reported the issue has been resolved for their mail account.
Update: 22 May 2019, 10:17 AM
We ask users to try and use Outlook on portal.office.com. However, from Microsoft's outage notifcation, all protocols are affected which would included access from a browser.
Service affected: Exchange Mail and Calendaring

Start: 22 May 2019, 6:00 AM
End: 22 May 2019, 4:00 PM
Login to request notification

ESXi host offline - Prentice NGDC

Resolved
An ESXi host in the Prentice NGDC has Crashed.
The following Virtual servers are affected:

cai-printsrv
cmm-arivis
cottonmouth
itsprnpvrops
mx1new
plasma
saap03
UQ-XA-DC01

Resolution: 21 May 2019, 1:12 PM
ESXi Host as recovered and VMs have been moved to other Cluster nodes.
Service affected: VMware NextGEN PRN DC

Start: 21 May 2019, 12:45 PM
End: 21 May 2019, 1:12 PM
Login to request notification

CCHR/LCCH Buildings - Network outage affecting wired only network

Resolved
ITS are working closely with QLD Health IT to resolve issues affecting several groups of UQ at the Mater Hospital. Affected areas include CCHR, LCCH and CHQCU.

Staff are onsite to assist staff with laptops in using wireless/vpn as a workaround.
Resolution: 22 May 2019, 3:49 PM
Mater Router R1 has been restarted which has resolved the issue. Staff on site have confirmed network access has been restored
Update: 22 May 2019, 1:13 PM
Mater Router 2 (a Juniper) was bypassed with teh backup FW connections from QH moved to Router 1. This has made no difference. QH NOC have been engaged to investigate. They are looking into potential ISE and routing issues within their network. Mater Router 2 is to be replaced to bring back redundancy to this site. ITS staff will assist staff with laptops on site to connect to Eudroam wireless and use VPN to access UQ Network.
Update: 21 May 2019, 1:15 PM
Router hardware has failed; working to replace it now. ITS NOC
Service affected: Networks

Start: 21 May 2019, 10:00 AM
End: 22 May 2019, 3:49 PM
Login to request notification

TRI Voicemail outage

Resolved
An issue impacting the TRI voice mails file system is preventing it from restarting.
ITS and TRI staff are investigating.
Resolution: 20 May 2019, 5:00 PM
Service affected: Voice Services

Start: 20 May 2019, 1:30 PM
End: 20 May 2019, 5:00 PM
Login to request notification

Network Outage in Lvl 3 GP South

Resolved
There is an outage occuring on lvl 3 GP South (B78). This is currently being investigated by the Network Team.
Resolution: 20 May 2019, 8:54 AM
Two switches have been replaced that support Lvl 3. Service has now been restored.
Service affected: Building Networks

Start: 20 May 2019, 7:30 AM
End: 20 May 2019, 8:54 AM
Login to request notification

Major UQ INTERNET service outage

Resolved
UQ INTERNET service is experiencing an unexpected outage and ITS is investigating now.
Resolution: 19 May 2019, 7:40 PM
UQ Internet has been restored. It is understood that the outage is likely to have resulted from a network equipment hardware fault, however, the root cause is still under investigation.
Update: 19 May 2019, 7:43 PM
We've been affected by a vendor bug during a different change on the device. The UQ Internet service has been recovered by moving the service to different device ports.
Service affected: Internet Services UQ

Start: 19 May 2019, 6:30 PM
End: 19 May 2019, 7:40 PM
Login to request notification

Eduroam unavailable to UQ users connecting via non-UQ networks

Resolved
Eduroam is currently unavailable for the UQ users connecting via non-UQ networks.

The issue is currently under investigation however at this stage it appears the AARnet RADIUS servers may be implicated.
Resolution: 17 May 2019, 6:00 PM
AARnet corrected a routing issue which restored Eduroam for all users.
Update: 17 May 2019, 2:10 PM
The issue is known by AARnet and they are working to resolve it.
Service affected: Wireless Network

Start: 17 May 2019, 12:30 PM
End: 17 May 2019, 6:00 PM
Login to request notification

Maintenance of Unitask

Resolved
The encoding of the production database will be changed to support a bigger set of characters.

During the change people accessing the system will see a maintenance message and they would not be able to view/action any task.

Support after the release

Some users might experience issues submitting tasks in myUQ and authenticating into Unitask. To solve this, users should clear the browser cache before login into myUQ or Unitask using their appropriate staff or student account.

Technical Information

The encoding of the production database was partially changed this week from latin-1 to utf-8 mb3. The job was not completed as the da...tabase ran out of disk space and the two biggest tables (arch_contract_data and queriable_log) are remaining from being converted.

Read more
Resolution: 17 May 2019, 12:00 AM
Automatic resolution.
Service affected: Unitask

Start: 16 May 2019, 5:00 PM
End: 17 May 2019, 12:00 AM
Login to request notification

Prentice building (B42) Level 3 has lost data/wifi network

Resolved
ITS investigating.
Resolution: 16 May 2019, 4:31 PM
Service has now been restored. The cause was due to a failed AC unit in the comms room, which stopped working and caused the network equipment to overheat.
Resolution: 16 May 2019, 4:11 PM
Network connectivity was restored by rebooting the network switch.
Service affected: Building Networks

Start: 16 May 2019, 3:11 PM
End: 16 May 2019, 4:31 PM
Login to request notification

Degraded Event Processing

Resolved
A change in upstream data has resulted in the processing pipeline becoming completely stuck. This is causing a large delay in logs being processed and available in the cluster.

A lack of visibility and logging verbosity in Logstash is causing troubles in identifying the offending events.

The current mitigation is to restart the processing nodes, this resolve the issue temporarily, but even so, they are currently unable to keep up with the current event ingres.

Kibana and Elasitcsearch are unaffected and will be available during the service outage. New event logs may not appear or will appear patchy while the processing nodes try to catch up with the bac...k log.

Read more
Resolution: 15 May 2019, 8:30 AM
Primary logging flow was restored to normal operation at 8:30 AM this morning. Several message queue cache files have been since played into the cluster during the course of business to catch up historical data. One event queue cache file has been kept to perform a root cause analysis that is still ongoing. This cache file will be replayed after COB today.
Resolution: 15 May 2019, 8:30 AM
Primary logging flow was restored to normal operation at 8:30 AM this morning. Several message queue cache files have been since played into the cluster during the course of business to catch up historical data. One event queue cache file has been kept to perform a root cause analysis that is still ongoing. This cache file will be replayed after COB today.
Service affected: Centralised Logging Cluster ELK

Start: 14 May 2019, 5:56 PM
End: 15 May 2019, 8:30 AM
Login to request notification

PrISM minor release with initial AD integration implementation (v2.4)(CR#5485)

Resolved

A minor PrISM update (v2.4) will be released to implement initial integration in line with Phase 1 of the PrISM-Active Directory (AD) Integration project.

This upgrade will redirect AD account creation and password reset transactions to a new RPC API. This will alleviate the dependency on Webkerb and a downstream web service hosted on its-gas1 which currently perform new account creations and password resets in AD.
Resolution: 14 May 2019, 8:00 PM
Automatic resolution.
Update: 14 May 2019, 7:15 PM
Backfilling of objectGuid from AD still in progress
Resolution: 14 May 2019, 6:00 PM
Automatic resolution.
Service affected: PrISM-Active Directory Integration Project

Start: 14 May 2019, 5:00 PM
End: 14 May 2019, 8:00 PM
Login to request notification

Oral Health Building Network Issues

Resolved
Currently ITS are investigating the issue
Resolution: 14 May 2019, 8:30 AM
Network connectivity has been restored
Resolution: 14 May 2019, 8:30 AM
Service affected: Building Networks

Start: 14 May 2019, 8:30 AM
End: 14 May 2019, 8:30 AM
Login to request notification

Maintenance of Unitask

Resolved
The encoding of the production database will be changed to support a bigger set of characters. Also, the application instances will be patched to resolve some security vulnerabilities.

Support after the release

Some users might experience issues actioning tasks in myUQ and authenticating into Unitask. To solve this, users should clear the browser cache before login into myUQ or Unitask using their appropriate staff or student account.

Technical Information

The encoding of the database will be changed from latin-1 to utf-8 mb3. Also the application instances will be patched as recommended by AWS.
Resolution: 13 May 2019, 6:30 PM
Automatic resolution.
Service affected: Unitask

Start: 13 May 2019, 5:00 PM
End: 13 May 2019, 6:30 PM
Login to request notification

TRI Eduroam/TRI Guest Wireless Outage

Resolved
Eduroam and TRI Guest wireless networks at TRI do not have Internet access.

This is currently under investigation by ITS.
Resolution: 13 May 2019, 10:33 AM
Automatic resolution.
Service affected: Wireless Network

Start: 13 May 2019, 7:05 AM
End: 13 May 2019, 10:33 AM
Login to request notification

Wireless Network Outage - Parnell building level 2 lecture room

Resolved
After scheduled network upgrade work in the Parnell building, unknown issue caused wireless service in two lecture rooms in level 2. The wireless coverage in other area of the building is not impacted.
Resolution: 13 May 2019, 12:30 PM
ITS has restored wireless to these rooms
Service affected: Wireless Network

Start: 12 May 2019, 4:35 PM
End: 13 May 2019, 12:30 PM
Login to request notification

Aurion and MyAurion not available during upgrade

Resolved
Aurion and My Aurion will be unavailable from 7.00 AM to 9.00 AM on Saturday, 11 May 2019 for system upgrade. We apologise for any inconvenience this may cause.
Update: 11 May 2019, 9:11 AM
Upgrade completed as scheduled
Resolution: 11 May 2019, 9:00 AM
Automatic resolution.
Service affected: Aurion

Start: 11 May 2019, 7:00 AM
End: 11 May 2019, 9:00 AM
Login to request notification

Unitask has intermittent access issues

Resolved
An issue in the network is causing the system to run out of memory and to stop responding to connections.

Technicians are investigating the problem.
Resolution: 10 May 2019, 12:04 PM
The issue has been resolved and the direct connect to the VPC has been resolved.
Service affected: Unitask

Start: 10 May 2019, 12:04 PM
End: 10 May 2019, 12:04 PM
Login to request notification

PrISM minor release with initial AD integration implementation (V2.4) (CR#5485)

Resolved
**Please note that the start time for this change has been updated from 12:00pm to 6:00pm.**

A minor PrISM update (v2.4) will be released to implement initial integration in line with Phase 1 of the PrISM-Active Directory (AD) Integration project.

This upgrade will redirect AD account creation and password reset transactions to a new RPC API. This will alleviate the dependency on Webkerb and a downstream web service hosted on its-gas1 which currently perform new account creations and password resets in AD.
Resolution: 8 May 2019, 6:00 PM
Automatic resolution.
Resolution: 8 May 2019, 8:00 PM
Automatic resolution.
Update: 8 May 2019, 9:17 AM
Updated start time from 12:00pm to 6:00pm.
Service affected: PrISM-Active Directory Integration Project

Start: 8 May 2019, 12:00 PM
End: 8 May 2019, 6:00 PM
Login to request notification

Global DHCP scope options update to WINS server addresses (CR#5322)

Resolved
This change will update the WINS server addresses provided to DHCP clients on all DHCP scopes. Currently DHCP scope settings provide the following WINS server addresses to clients:

Primary WINS Server: 130.102.2.112
Secondary WINS Server: 130.102.5.110

These will be updated to:

Primary WINS Server: 10.208.9.101
Secondary WINS Server: 10.208.137.102
Resolution: 8 May 2019, 2:00 PM
Automatic resolution.
Service affected: WINS

Start: 8 May 2019, 10:00 AM
End: 8 May 2019, 2:00 PM
Login to request notification

Student Tutor Survey with Test URL

Resolved
Some student may have received an email to the Tutor Surveys with a test URL link to testsurvey.eval.uq.edu.au.

The issue has been fixed and the test URL should now be redirected to the correct URL - https://eval.uq.edu.au/eus.onlinesurveyportal/.
Resolution: 6 May 2019, 5:30 PM
Automatic resolution.
Service affected: EvaSys

Start: 6 May 2019, 4:00 PM
End: 6 May 2019, 5:30 PM
Login to request notification

Cluster not operational

Resolved
Elasticsearch cluster is in a red state, investigating cause
Resolution: 6 May 2019, 6:40 PM
A performance state caused the JVM heap to be over-allocated and the several cluster nodes became unresponsive. Upgrading to version 6.7.2 to address some cluster management related bug-fixes and restarting supporting services resolved the issue.
Update: 6 May 2019, 6:40 PM
JVM heap was exceeded and nodes became unresponsive. Upgrading to version 6.7.2 and restarting supporting services restored cluster into an operational state.
Service affected: Centralised Logging Cluster ELK

Start: 6 May 2019, 3:45 PM
End: 6 May 2019, 6:40 PM
Login to request notification

Intermittent access issues to external servers

Resolved
Connection to chalkandwire.com & api.cloud.bb were intermittent. Connections to these third party sites can be established again.
Resolution: 4 May 2019, 8:39 AM
Automatic resolution.
Service affected: Blackboard

Start: 4 May 2019, 7:01 AM
End: 4 May 2019, 8:39 AM
Login to request notification

New version of a process to be released

Resolved
A new version Of the Staff Vaccination process will be deployed to UniTask to support recall events. Staff and students may experience issues raising and processing requests from myUQ and UniTask.

Support After the release
Some users might experience issues actioning tasks in myUQ and authenticating into Unitask. To solve this, users should clear the browser cache before login into myUQ or Unitask using their appropriate staff or student account.

Technical Information
The following will be deployed to UniTask:
* SV recall events from Prism
* SV autocase creation
* Change to immunisation status
Resolution: 29 Apr 2019, 6:00 PM
Automatic resolution.
Service affected: Unitask

Start: 29 Apr 2019, 5:00 PM
End: 29 Apr 2019, 6:00 PM
Login to request notification

CCR Data Centre - Building Power Issues

Resolved
UQ CCR Building at Herston lost power.

UPS and Generator kept power to DC.

Building power was restored.

No services were affected.
Resolution: 29 Apr 2019, 9:25 AM
Automatic resolution.
Service affected: Data Centre

Start: 29 Apr 2019, 8:13 AM
End: 29 Apr 2019, 9:25 AM
Login to request notification

Migrate ITSWEBPD to NGDC Exadata and Upgrade to Oracle 18c

Resolved
Migrate ITSWEBPD to NGDC Exadata and Upgrade to Oracle 18c
Resolution: 27 Apr 2019, 1:00 PM
Automatic resolution.
Service affected: ITS SD ADS Shared Web Database

Start: 27 Apr 2019, 8:00 AM
End: 27 Apr 2019, 1:00 PM
Login to request notification

mySI-net outage scheduled for 24 to 26 April

Resolved
Due to a system update, mySI-net will be unavailable from 4.30pm on Wednesday, 24 April until 9am on Friday, 26 April 2019.
Resolution: 26 Apr 2019, 8:00 AM
Resolution: 26 Apr 2019, 8:00 AM
Automatic resolution.
Update: 26 Apr 2019, 8:04 AM
SI-net is now available
Service affected: SI-net

Start: 24 Apr 2019, 4:30 PM
End: 26 Apr 2019, 8:00 AM
Login to request notification

Outage impacting uq.edu.au, my.uq.edu.au, courses.uq.edu.au

Resolved
An outage is currently impacting uq.edu.au, my.uq.edu.au, courses.uq.edu.au and a number of other sites that are serving requests off the reverse proxy nginx layer service. Technicians are currently investigating.
Resolution: 23 Apr 2019, 3:18 AM
An issue was encountered whilst initiating preparations for upcoming maintenance. This triggered a cascade of events which resulted in network misconfiguration routing traffic to the incorrect destination. The initial changes have been reverted an service is now back online.
Update: 23 Apr 2019, 3:21 PM
An issue was encountered whilst initiating preparations for upcoming maintenance. This triggered a cascade of events which resulted in network misconfiguration routing traffic to the incorrect destination. The initial changes have been reverted an service is now back online.
Service affected: Web Serving Architecture

Start: 23 Apr 2019, 2:57 PM
End: 23 Apr 2019, 3:18 AM
Login to request notification

Xymon DNS Test Issues

Resolved
Xymon DNS tests are generating false positive alerts.

ITS Staff are investigating the issue.

Resolution: 23 Apr 2019, 10:13 AM
ITS Staff investigated and found that multiple services had not terminated correctly. These services were killed and the issue was resolved. The cause of this issue is being investigated.
Service affected: Xymon Monitoring System

Start: 23 Apr 2019, 7:00 AM
End: 23 Apr 2019, 10:13 AM
Login to request notification

Migrate Credit Precedents production database to NGDC Exadata and Upgrade to Oracle 18c

Resolved
Migrate SASCRP database to NGDC Exadata and Upgrade to Oracle 18c
Resolution: 16 Apr 2019, 8:00 PM
Automatic resolution.
Service affected: Credit Precedents

Start: 16 Apr 2019, 5:00 PM
End: 16 Apr 2019, 8:00 PM
Login to request notification

Therapies Buildings access network down

Resolved
The network in the Therapies and Therapies Annex is currently down. The root cause is under investigation
Resolution: 16 Apr 2019, 5:51 PM
Network services have been restored to Therapies Building. An investigation is ongoing with Cisco Engineers into the exact cause of the outage.
Service affected: Building Networks

Start: 16 Apr 2019, 2:27 PM
End: 16 Apr 2019, 5:51 PM
Login to request notification

BWC (B61a) builiding network outage

Resolved

Brian Wilson Chancellery (61A) building is currently experiencing a wired and wireless network outage. ITS staff is investigating.
Resolution: 14 Apr 2019, 10:15 AM
Reloaded building agg routers
Service affected: Networks

Start: 14 Apr 2019, 5:00 PM
End: 14 Apr 2019, 10:15 AM
Login to request notification

Migrate Headshot production database to NGDC Exadata and Upgrade to Oracle 18c

Resolved
Migrate ITSHSP database to NGDC Exadata and Upgrade to Oracle 18c
Resolution: 14 Apr 2019, 10:00 AM
Automatic resolution.
Service affected: Headshot Database

Start: 14 Apr 2019, 7:00 AM
End: 14 Apr 2019, 10:00 AM
Login to request notification

Aurion and MyAurion not available during maintenance

Resolved
Aurion and MyAurion will be unavailable due to NGDC migration of database and batch components.
Resolution: 13 Apr 2019, 1:30 PM
The outage took longer than expected due to unexpected issues. Aurion database and batch components have successfully been migrated to NGDC and full service is restored.
Service affected: Aurion

Start: 13 Apr 2019, 7:00 AM
End: 13 Apr 2019, 1:30 PM
Login to request notification

SQL 2014 Clustered Database Services - OS Patching

Resolved
This outage is to facilitate regular monthly operating system updates, security enhancements, and overall service improvements.

Clients and applications with databases hosted on the Microsoft SQL Server 2014 development cluster, will experience a very brief service interruption whilst database instances are failed over between ITS data centres.

The service interruption duration is generally 60-120 seconds within the designated outage window. Affected applications and services typically reconnect automatically.
Resolution: 12 Apr 2019, 8:00 PM
Automatic resolution.
Service affected: SQL 2014 Clustered Database Hosting

Start: 12 Apr 2019, 7:00 PM
End: 12 Apr 2019, 8:00 PM
Login to request notification

SQL 2017 Clustered Database Services - OS Patching

Resolved
This outage is to facilitate regular monthly operating system updates, security enhancements, and overall service improvements.

Clients and applications with databases hosted on the Microsoft SQL Server 2017 cluster, will experience a very brief service interruption whilst database instances are failed over between ITS data centres.

The service interruption duration is generally 30 - 60 seconds within the designated outage window. Affected applications and services typically reconnect automatically.
Resolution: 12 Apr 2019, 7:00 PM
Automatic resolution.
Service affected: SQL 2017 Clustered Database Hosting

Start: 12 Apr 2019, 6:00 PM
End: 12 Apr 2019, 7:00 PM
Login to request notification

UQ SSO unavailable

Resolved
The UQ SSO service, available at

https://auth.uq.edu.au/


was unavailable during the specified period.
ITS staff identified and corrected the issue.
Resolution: 12 Apr 2019, 4:29 PM
Automatic resolution.
Service affected: Single SignOn

Start: 12 Apr 2019, 4:22 PM
End: 12 Apr 2019, 4:29 PM
Login to request notification

Network Outage in Therapies Buildings (84, 84A)

Resolved
Building Agg switch has been restarted, network now restored.

Known bug issue with cisco IOS causing memory leaks. No fix available yet.
Resolution: 11 Apr 2019, 3:34 PM
Therapies Building Agg switch has been restarted. Caused by a memory leak... known bug with Cisco IOS and no fix yet available. This fault will cause both building Agg Switches to fail, therefore taking out the redundant link. Actively investigating pre-emptive actions to help mitigate this issue until a Cisco fix is available.
Resolution: 11 Apr 2019, 3:34 PM
Therapies Building Agg switch has been restarted. Caused by a memory leak... known bug with Cisco IOS and no fix yet available. This fault will cause both building Aggs to fail, therefore taking out teh redundant link.
Update: 11 Apr 2019, 3:34 PM
Bug with Cisco IOS caused memory leak. Agg pair have rebooted and building is back on line
Service affected: Networks

Start: 11 Apr 2019, 3:13 PM
End: 11 Apr 2019, 3:34 PM
Login to request notification

Network Outage at 74 High St Toowong

Resolved
Network services to 74 High St have been affected by a fibre cut on our providers network.

Temporary 4G Wireless service has been installed until the fibre cut has been repaired.
Resolution: 12 Apr 2019, 8:00 AM
Damaged fibre has been repaired by the carrier.
Update: 11 Apr 2019, 1:38 PM
4G Wireless connection has been put in to provide temporary network access for staff on site.
Update: 11 Apr 2019, 1:27 PM
Fault identified as a fibre cut near 50 Ann St. Crews on the way to fix now, no ETR has been provided.
Update: 11 Apr 2019, 1:22 PM
Fault with carrier service which appears to be an issue they have in the fibre network in the CBD. Currently waiting for an ETR. In the interim ITS Networks have setup a temporary wireless service to provide network connectivity to staff at High St.
Service affected: Networks

Start: 11 Apr 2019, 12:03 PM
End: 12 Apr 2019, 8:00 AM
Login to request notification

Student Surveys are current unavailable

Resolved
ITS are currently investigating the issues.
Resolution: 9 Apr 2019, 1:51 PM
There was a issues with the underlying database cluster. This has now been resolved and the Student Survey system is back online.
Service affected: EvaSys

Start: 9 Apr 2019, 1:51 PM
End: 9 Apr 2019, 2:00 PM
Login to request notification

Office 365 logins failing

Resolved
Single Sign-On to the Office 365 is failing for some accounts.

ITS staff are working on the problem. Most accounts are functioning properly again, and the remainder are expected to be fixed by 7:00 AM this morning.
Resolution: 3 Apr 2019, 7:00 AM
Automatic resolution.
Service affected: Office 365 Sign On

Start: 3 Apr 2019, 3:30 AM
End: 3 Apr 2019, 7:00 AM
Login to request notification

UQ Organisational Database Migration

Resolved
The UQ Organisational Database will be migrated to a newer version of the underlying database in the NextGen data centre.

The administration website will be offline, and any datafeeds will not recieve data during that period.
Resolution: 31 Mar 2019, 11:15 AM
The migration has worked successfully in a shorter time than we expected. The website and database are back up, including for data feeds.
Service affected: UQ Organisational Database

Start: 31 Mar 2019, 8:00 AM
End: 31 Mar 2019, 11:15 AM
Login to request notification

Aurion and MyAurion not available during maintenance

Resolved
Aurion and MyAurion will be unavailable from 6 AM to 9 AM on Saturday 30th March 2019 for patching and fail-over testing
Resolution: 30 Mar 2019, 9:00 AM
Patching and fail-over testing completed successfully
Service affected: Aurion Application Servers

Start: 30 Mar 2019, 6:00 AM
End: 30 Mar 2019, 9:00 AM
Login to request notification

Migrate Collation production database to NGDC Exadata and Upgrade to 12.2

Resolved
Migrate COLLP dataabse to NGDC Exadata and database Upgrade
Resolution: 30 Mar 2019, 10:00 AM
Automatic resolution.
Service affected: Collation Database

Start: 30 Mar 2019, 6:00 AM
End: 30 Mar 2019, 10:00 AM
Login to request notification

Therapies Building network outage

Resolved
The building access network in the Therapies building is down at the moment. Although it may be due to a power outage in the buildings we are still investigating the fault.
Resolution: 29 Mar 2019, 12:30 PM
The building aggregation switch reloaded because of a software bug. Preventative measures are being investigated. The building network has now recovered.
Resolution: 29 Mar 2019, 12:30 PM
The building aggregation switch reloaded because of a software bug. Preventative measures are being investigated. The building network has now recovered.
Service affected: Building Networks

Start: 29 Mar 2019, 12:08 PM
End: 29 Mar 2019, 12:30 PM
Login to request notification

Learn.uq.edu.au reduced redundancy

Resolved
Learn.uq.edu.au is running under reduced redundancy due to an underlying data centre issue.
Service is currently being restored.
Users may be experiencing issues during this period.
ITS apologises for any inconvenience caused.
Resolution: 27 Mar 2019, 11:57 AM
Learn.uq.edu.au has been returned to full service.
Service affected: Blackboard

Start: 27 Mar 2019, 11:00 AM
End: 27 Mar 2019, 11:57 AM
Login to request notification

Total VMware outage

Resolved
Complete power loss to rack in Data centre has caused critical outage to Infrastructure.

All Virtual Guest Machines are impacted.
Resolution: 27 Mar 2019, 11:20 AM
Power has been restored to Rack and infrastructure is fully functional. ITS Engineers has restarted all affected Virtual servers.
Resolution: 27 Mar 2019, 11:20 AM
Automatic resolution.
Service affected: VMware NextGEN PRN DC

Start: 27 Mar 2019, 10:56 AM
End: 27 Mar 2019, 11:20 AM
Login to request notification

Loss of power Prentice DC2 B4

Resolved
Due to a loss of power, servers located in Prentice DC2 B4 were unavailable.

Engineers are in the process of making sure all impacted servers have recovered.
Resolution: 27 Mar 2019, 11:05 AM
Automatic resolution.
Service affected: Data Centre

Start: 27 Mar 2019, 10:50 AM
End: 27 Mar 2019, 11:05 AM
Login to request notification

Apply ESXi patches to CCR Nextgen Nutanix cluster

Resolved
Apply latest ESXi patches to CCR Nextgen vSphere Datacenter - CCR.
Resolution: 27 Mar 2019, 11:00 AM
Automatic resolution.
Service affected: VMware NextGEN CCR DC

Start: 27 Mar 2019, 9:00 AM
End: 27 Mar 2019, 11:00 AM
Login to request notification

Maintenance of Unitask

Resolved
The production instances will be patched to resolve some security vulnerabilities.

Support after the release

Some users might experience issues actioning tasks in myUQ and authenticating into Unitask. To solve this, users should clear the browser cache before login into myUQ or Unitask using their appropriate staff or student account.

Technical Information

The instances will be patched as recommended by AWS, also the PDF generator connector will be upgraded to resolve an issue causing PDFs not being generated.
Resolution: 26 Mar 2019, 5:45 PM
Automatic resolution.
Service affected: Unitask

Start: 26 Mar 2019, 5:00 PM
End: 26 Mar 2019, 5:45 PM
Login to request notification

Apply ESXi patches to PRN cluster

Resolved
Apply latest ESXi patches to vSphere Datacenter - Prentice Legacy.
Resolution: 26 Mar 2019, 3:45 PM
Automatic resolution.
Service affected: VMware PRN DC

Start: 26 Mar 2019, 1:00 PM
End: 26 Mar 2019, 3:45 PM
Login to request notification

Apply ESXi patches to GPN cluster

Resolved
Apply latest ESXi patches to Datacenter - GPNorth Legacy.
Resolution: 26 Mar 2019, 12:00 PM
Automatic resolution.
Service affected: VMware GPN DC

Start: 26 Mar 2019, 10:00 AM
End: 26 Mar 2019, 12:00 PM
Login to request notification

Apply ESXi patches to ESXi hosts CCR

Resolved
Apply latest ESXi patches to All ESXi hosts in Datacenter - UQCCR Legacy.
Resolution: 26 Mar 2019, 10:00 AM
Automatic resolution.
Service affected: VMware NextGEN CCR DC

Start: 26 Mar 2019, 8:00 AM
End: 26 Mar 2019, 10:00 AM
Login to request notification

AOS (NOS) 5.10.2 upgrade to Prentice NextGen Nutanix cluster

Resolved
NOS (Acropolis) 5.10.2 upgrade to Prentice02 NextGen Nutanix cluster.
Resolution: 25 Mar 2019, 3:45 PM
Automatic resolution.
Service affected: VMware NextGEN PRN DC

Start: 25 Mar 2019, 2:30 PM
End: 25 Mar 2019, 3:45 PM
Login to request notification

AOS (NOS) 5.10.2 upgrade to CCR NextGen Nutanix cluster

Resolved
NOS (Acropolis) 5.10.2 upgrade to CCR02 NextGen Nutanix cluster.
Resolution: 25 Mar 2019, 2:30 PM
Automatic resolution.
Service affected: VMware NextGEN CCR DC

Start: 25 Mar 2019, 1:30 PM
End: 25 Mar 2019, 2:30 PM
Login to request notification

Intermittent Network Issues for some Non-St Lucia Sites

Resolved
We are experiencing intermittent network issues that are affecting some of UQs sites external to St Lucia Campus.

This is currently being investigated by ITS NOC
Resolution: 25 Mar 2019, 11:35 AM
Configuration issue found on MAN1. Now fixed
Service affected: Networks

Start: 25 Mar 2019, 11:15 AM
End: 25 Mar 2019, 11:35 AM
Login to request notification

AWS Direct Connect experiencing intermittent issues

Resolved
Due to an ongoing network outage on one of the AWS Direct Connect terminating routers at St Lucia (Event 17436), some traffic may be getting dropped.

ITS are currently investigating the problem.
Resolution: 25 Mar 2019, 11:35 AM
The underlying issue (https://www.its.uq.edu.au/?incident_id=17436) has been resolved.
Service affected: Amazon Web Services - Direct Connect

Start: 25 Mar 2019, 10:50 AM
End: 25 Mar 2019, 11:35 AM
Login to request notification

AOS (NOS) 5.10.2 upgrade to Prentice Nutanix cluster

Resolved
NOS (Acropolis) 5.10.2 upgrade to Prentice Nutanix cluster.
Resolution: 25 Mar 2019, 1:30 PM
Automatic resolution.
Service affected: VMware PRN DC

Start: 25 Mar 2019, 10:30 AM
End: 25 Mar 2019, 1:30 PM
Login to request notification

VDI Services - Reduced Redundancy

Resolved
The VDI 02 server has had critical hardware failures.

Service is still running from VDI 01 server.

Infrastructure Staff are investigating the issue.

Resolution: 28 Mar 2019, 2:40 PM
Hardware was replaced. Service owner confirmed system is ok.
Service affected: VDI Servers

Start: 25 Mar 2019, 8:42 AM
End: 28 Mar 2019, 2:40 PM
Login to request notification