Review of the Performance Assurance Framework
ELEXON is currently undertaking a review of the Performance Assurance Framework. This page tells you about the review and provides the latest update on progress.
Issue 69 Engagement
An Issue 69 Workgroup has been arranged to review the BSC Audit Performance Assurance Technique.
Discussion topics will include the audit cycle, on-site vs desktop, entity selection, and objectives. Suggested attendees for this session are those that have been involved in the BSC Audit and/or have experience of other internal and external audits and managing the logistics and outputs of audits.
The date for the session is as follows:
- BSC Audit Review: 10.00 – 16.00 on Wednesday 15 May 2019 at ELEXON’s offices.
Lunch and refreshments will be provided on the day. There will also be teleconference facilities available for anyone who cannot attend in person.
If you would like to be part of either of these Issue Groups, please email email@example.com.
2019/20 Risk Management Now Live
The new Risk Evaluation Register is now published and has been effective since 1st April 2019. Webinars introducing the Risk Evaluation Methodology and the Risk Evaluation Register are up on YouTube for anyone who wants to learn more about the BSC risk management approach. The Risk Operating Plan was approved in the March PAB meeting. It describes:
- The PAB’s risk appetite for each Settlement Risk.
- The planned technique deployment to achieve the target impact.
- The planned technique deployment to strengthen controls or mitigate events.
- The estimated costs of such deployment.
Review of the Assurance Techniques Workstream
The first technique, Technical Assurance of Metering (TAM), was reviewed and the findings presented to PAB in February, who approved the recommendations. These proposals will now be taken forward – contact the TA Metering team for any questions.
Supplier Charges are currently being reviewed to ascertain whether or not they are effective in their current form, and how they could be amended going forward. The BSC Audit is also being assessed for improvements within the existing approach, and for potential changes to the current audit cycle.
Further work includes a review of the Qualification/Re-Qualification Performance Assurance Technique (PAT). At present, the content of the Self-Assessment Document is being compared with the risk register to highlight any gaps in assurance.
Data Provision Workstream
A paper was delivered to the February PAB meeting, presenting the work done to date and including an assessment of current data available for the PAF. Proofs of concept are being developed around additional data sources that could be obtained, and current sources that are underutilised. Currently, Theft Risk Assessment Service (TRAS) data is being assessed, as it could offer insight around the revenue protection risk.
Risk Visualisation Tool
An online risk visualisation tool has been developed to help BSC parties identify where settlement risks occur in BSC processes. In addition, it contains basic information about each risk, the parties affected by the risk, and relevant controls. At present, the tool only applies to the SVA market, however it is due to be expanded to include CVA risks. We welcome your feedback on the tool to firstname.lastname@example.org.
Review of the Assurance Techniques Workstream
We expect to report our recommendations to the PAB in May for the BSC Audit and Supplier Charges. Before this, we will be engaging on both techniques with the Issue 69 Working Group.
Data Provision Workstream
We are continuing to assess alternative data sources through proofs of concept. This will allow us to test the specific data items and ascertain how useful and usable they might be, as well as test our systems and processes. We expect to need your contribution on the options through the Issue 69 workgroup in the coming months.
Key aims of the review
- Better engagement with parties about issues that do and don’t matter to them (their risk appetite)
- Increase the quantity and quality of participation in consultations
- Meet the current and future assurance needs of the Panel, the PAB and the wider electricity industry e.g. smart metering, alternative business models, CVA risk
- Improve the measurability of Settlement error and the mitigating effect of assurance activities
- Deliver a valued and trusted assurance service to BSC Parties under the strategic and tactical guidance of the BSC Panel and PAB
The review is projected to run to Q2 2020.
A core ELEXON team, supported by risk management and data experts, will deliver the review in four workstreams. Industry experts signed up to the Issue 69 working group will be called upon to review and comment on key approaches and proposals.
The PAB will be the primary decision making body, and will receive regular updates on the project’s progress.
The four workstreams will culminate in a revised framework, a new set of risks with estimated error, a refreshed set of mitigating techniques and improved reporting. This proposed framework will be implemented via the BSC Change process where required, after PAB and Panel approval. Any improvements that can be made sooner will be progressed as necessary.
Workstream 1 – Smart Metering Rollout
To provide immediate assurance against the challenges of the smart metering rollout by developing Proof of Concepts to enable the effective monitoring of significant Settlement Risks associated with the rollout.
This workstream has now completed. We identified a set of related Settlement Risks; we developed a report on the production and submission of Meter Technical Details (MTDs); we held an industry day; and we handed over management of the risks to the operational teams.
Workstream 2 – PAF Procedures
To deliver recommendations to improve the operability and effectiveness of the REM and other PAF procedures that ensure that risks are more meaningful and reflective to participants.
This workstream has reached its conclusion, having discussed with the Issue 69 workgroup:
- How risk areas should be identified
- How these risks should be evaluated, including the workgroup’s endorsement of a new measurement approach
- How ELEXON could achieve greater engagement with PAF stakeholders
- How the identified risks should be evaluated in the future
- Some examples of the proposed PAF design in use
Workstream 3 – Review of the PATs
This workstream will assess, in light of feedback received on the PAF, if there are any gaps in the current set of mitigating assurance techniques (using the new Risk Evaluation Methodology and risk register to identify gaps in risk mitigation), which techniques are cost effective for the level of risk they mitigate, and which techniques are not.
It will then produce a set of recommendations for retaining, altering, adding or removing techniques.
This workstream will also consider the general working practices that ELEXON and the PAB adopt in relation to coordinating and applying the techniques.
Workstream 4 – Data Provision
To create Proofs of Concept to test the viability of alternative methods of data provision that place less burden on participants to provide data and that supports more accurate risk appraisal.
As well as this webpage, we will use Newscast and the OSMs to provide key information and how you can get involved. The Issue 69 webpage holds documents related to that group’s work.
We will also be attending BSC and external forums to present on the objectives and activities of the review. If you would like us to present to your forum, please email us at email@example.com.
What is the Framework?
Around £1.7bn in transactions went through the BSC Settlement systems in 2016/17. BSC Parties require an assurance framework that allows them to have confidence in the accuracy of the allocations of energy and charges.
The current approach to the Performance Assurance Framework (PAF) was introduced in 2007 under Modification P207.
With 16 assurance techniques, over 200 identified Settlement Risks, an annual central cost of around £3m, and impacts on all BSC Parties (plus their agents and BSC Agents/systems), the PAF must be delivered in an effective and efficient way.
Why review the PAF?
After 10 years of live operations, and some significant industry changes such as smart metering and innovative business models, the BSC Panel considered the PAF should be reviewed and included it in its strategic forward work plan 2016/17.
ELEXON initiated a project and the Panel approved the preliminary scope and approach at Panel 255 in July 2016.
ELEXON then engaged with a broad spectrum of stakeholders to validate the scope and approach for the design phase of the review. The Performance Assurance Board (PAB) endorsed these proposals and Panel approval was gained in March 2017 (PAB 264/07 – PAF Review Scope and Approach Approval).