Learning from Audits – IT Systems and Processes

We recently conducted a suite of thematic IT audits of several awarding bodies (ABs) focussing on the IT arrangements (including governance, resources, capacity and capability) that these ABs had in place to undertake the development, delivery and award of qualifications in Wales. The key areas of focus in our audit were; IT strategy and planning, risk management, data protection and records handling and business continuity and resilience planning. Audits enable us to identify potential risks and key areas for improvement. Key findings are summarised below.

Findings from the four audits were positive with no instances of non-compliance with the Standard Conditions of Recognition (SCoR) identified. We recognise the efforts made by ABs to ensure that they have effective and robust IT systems in place. However, despite several strengths being identified we also found key areas for improvement that ABs should address to further improve the quality of their provision.

In order to raise standards across all our regulated ABs, not just those included within the audit cycle, we have summarised the key thematic findings from the IT audits.

Key thematic findings by lines of inquiry

IT Strategy and Planning

Risks: The IT function and its capabilities within the awarding body do not adequately enable and support the development, delivery and award of qualifications.

Relevant SCoR: A5.1 & A5.3

Good Practice Identified: ABs that performed well on this line of inquiry had a business plan in place to drive the development, delivery and award of qualifications. The IT initiatives were in alignment with business objectives and IT investments were planned and formally approved. There was evidence of an established process for proposing and approving IT projects and IT strategies were approved by the management board. ABs agreed and set key performance indicators (KPIs) to measure delivery against strategic IT objectives and IT systems are developed in an agile way enabling the AB to frequently reprioritise work to support the business.

Key Areas for Improvement: For some ABs the vision, priorities and principles for areas of IT supported business were not always articulated within their business plan, and detailed modelling of IT project benefits such as efficiency savings were not always conducted in advance. In some cases, there was a lack of an operating level agreement or KPIs to measure the quality of the IT service delivery. Where third parties were used to provide IT infrastructure support, service level agreements were not always in place. This has the potential to affect the timeliness and the quality of the service provided by the third party and may result in business requirements not being met. Some ABs did not have real time access to some of the IT management systems that were used by the IT infrastructure provider which could result in delays in resolving incidents.

Risk Management

Risk: A lack of information security policies, practices and controls could threaten the confidentiality, integrity and availability of data.

Relevant SCoR :A6.1, A6.2, A7.1 & H6.1

Good Practice Identified: ABs that performed well on this line of inquiry had a governance structure that provided oversight of risk management by Directors and relevant committees. There was evidence of a standardised risk assessment framework which provided clear definitions for rating the likelihood and impact of risks. Risks were categorised according to their area of impact on the delivery and award of qualifications including the risk of regulatory non-compliance. ABs used risk registers to track risks at both a strategic and operational level, owners are assigned to individual risks and the register is reviewed regularly. ABs had a suite of IT policies and controls to cover common risks, for example; the acceptable use of IT, information security, passwords, network monitoring, malicious software prevention and social media. Some of these policies and controls are supported by external assurance certificates such as Cyber Essentials. Some ABs were working towards achieving Cyber Essentials Plus certification.

Key Areas for Improvement: Some ABs had not defined the risk appetite for any area of risk management therefore the level of mitigation for each risk may not be aligned with business objectives and regulatory requirements. Common risks to IT operations such as, loss of IT property, inappropriately assigned user privileges, and network connectivity failures were not always captured on risk registers. In some cases, there was evidence that the ABs risk assessment framework did not provide a means to calculate the overall rating for each risk based on its likelihood and impact. This could potentially lead to risks not being reviewed and prioritised efficiently, which may cause delays in planning activities to mitigate the risk. There was evidence that in some instances the effectiveness of individual controls were not being tracked on the IT operational risk register which may expose the AB to unacceptable levels of risk. 

Data Protection and Records Management

Risks: Poor practices for learner unique identifiers could result in qualifications being awarded to the wrong learner and increases the awarding body’s risk of being defrauded into issuing qualifications. Inadequate records management processes to evidence learner identity checks increases the risk of identity theft or other fraudulent activity. Personal data which is not being managed appropriately could result in a data breach and fines from the Information Commissioner’s Office.

Relevant SCoR:A5.2, A6.1, A6.2, A7.1

Good Practice Identified: Stronger performing ABs could evidence a strong privacy aware culture which is embedded throughout their practices and attitudes. They had a clear plan for the IT focussed approach to GDPR and a good governance structure in place to implement GDPR requirements. Stronger ABs also had effective controls over records with a clear review procedure. There was evidence of controlled management of data including retention schedules and quality monitoring of third parties to ensure that service providers comply with privacy obligations.

Key Areas for Improvement: In some cases, there appeared to be a lack of understanding regarding the ABs obligations under the GDPR and as a result fundamental GDPR requirements have been omitted from their planning. Some ABs lacked documentation outlining the justifications for their GDPR programme and implementation decisions. Without documenting the rationale behind key decisions that anchor their GDPR implementation strategy, it may be challenging to readily demonstrate a risk-based approach to GDPR if requested by the Information Commissioner’s Office.

Business Continuity and Resilience Planning

Risks: A lack of business continuity planning and IT disaster recovery planning could result in the awarding body being unable to develop, deliver and award qualifications in the event of an incident that disrupts normal operations.

Relevant SCoR:A6.1, A6.2, A7.1 & H6.1

Good Practice Identified: ABs that performed well on this line of inquiry could evidence that a Business Impact Analysis (BIA) had been performed to identify critical activities and the priorities within the organisation. This included the identification of dependencies with other departments and resources which are critical to activities, such as desired and actual recovery times. ABs had put in place resilience measures for online exams to ensure that the learners are not disrupted in the event of an IT issue. ABs could evidence that both business continuity plans and data recovery testing had been conducted and that any issues which were identified during testing were logged and tracked. 

Key Areas for Improvement: Some ABs had not performed a BIA to identify the critical activities and required recovery times which means planning may be incomplete. This could result in the AB focussing on the incident response and not providing sufficient details for the continuation of activities. It was also apparent that disaster recovery exercising had not been performed across some ABs, which increases the risk that some staff are not aware of the expectations and their responsibilities during an incident.

For those ABs who employed the services of a third party it was not always clear how disaster recovery would be invoked by that third party.