The Management and Resolution of Erroneous Large EACs and AAs |
Guidance Note |
This document provides guidance to Suppliers and Agents on managing and resolving Erroneous Large Estimated Annual Consumption (EAC) and Annualised Advances (AA). It describes -
the issue and its impact on the market;
what Elexon is doing to monitor the problem;
what the industry can do to address the issue;
the Trading Disputes process used to correct errors.
Erroneous Large EACs and AAs
An average domestic customer (on Profile Class 1) uses about 3,500 kWh per year or approximately 10 units a day. Most domestic customers will use less than 25,000 units per year. A 100kW business customer (on Profile Class 8) who maintained full load for the entire year (a theoretical situation, never achieved in real life) would use:
100kW × 8760 hours = 876,000 kWh
However, we have discovered many EACs and AAs that are tens of millions in size - consumption more common for steel works. These EACs and AAs are unlikely to reflect actual consumption. Not only is the consumption unrealistic, the distribution network infrastructure would not support such demands. These excessive values do not occur very often but their size makes their impact on Supplier Volume Allocation (SVA) significant.
To identify excessive consumption, Elexon produced a set of thresholds above which Erroneous Large EACs and AAs can be identified. The thresholds are set per Profile Class and represent a kWh per register per year value. For each Profile Class there is also a negative threshold below which negative Erroneous Large EACs and AAs can be identified. Consumption that exceeds Elexon’s consumption thresholds is not necessarily wrong. There may be genuine consumption above the thresholds and conversely there may be erroneous large consumption below these thresholds.
Figure 1: Excessive Consumption Thresholds by Profile Class
Profile Class | Description | Threshold (kWh/Year/Register) |
Upper | Lower |
1 | Domestic Unrestricted | 128,000 | -50,000 |
2 | Domestic with Switched load | 88,000 | -50,000 |
3 | Non-Domestic Unrestricted | 200,000 | -35,000 |
4 | Non-Domestic with Switched load | 140,000 | -35,000 |
5 | Load Factor < 20% | 220,000 | -35,000 |
6 | Load Factor 20% to 30% | 320,000 | -35,000 |
7 | Load Factor 30% to 40% | 430,000 | -35,000 |
8 | Load Factor > 40% | 552,000 | -35,000 |
We identified the issue of erroneous large EACs and AAs in Settlement in 2000. The issue was of sufficient materiality to contribute to a qualified BSC Audit Opinion. In April 2000, the Pool Executive Committee (PEC) and Trading Stage 2 Committee (TS2) authorised the processing of Post-Final (DF) Settlement Runs to correct Settlement data. DF runs started in July 2000. It was agreed to use the disputes process to re-run Settlement until the issue was “substantially addressed within the Final Reconciliation Runs”. 14 years later, DF Settlement Runs are not processed in any GSP Groups. Until quite recently (mid-2013), most of the DF Runs were slight open, however. Erroneous Large EACs and AAs are a major Settlement issue in their own right, and are symptomatic of a variety of other Settlement issues.
The Settlement Impact of Erroneous Large EACs and AAs
Erroneous Large EACs and AAs can significantly impact Suppliers. At a market level, error is absorbed through the application of Group Correction Factor. The issue affects Suppliers through the misallocation of the error during Supplier Volume Allocation. Erroneous Large EACs and AAs also distort a Supplier’s share of market-level consumption. Suppliers pay for any resulting erroneous electricity consumption and also face additional cost through higher Distribution Use of System (DUoS) and Transmission Network Use of System (TNUoS) charges.
Elexon’s Large EAC/AA System
Elexon has built a system to monitor the issue and provide reporting to Panel sub-committees and the industry. The Large EAC/AA system uses data collected from Non Half Hourly Data Aggregators (NHHDAs), Suppliers and Non Half Hourly Data Collectors (NHHDCs). Data is collected monthly before running the system. The monthly process is illustrated in figure 2 below.
The primary information is obtained from NHHDA databases. Each NHHDA, using a standard script, provides us with details of all consumption values above the agreed Large EAC/AA thresholds (see figure 1 above). This information is provided each month and processed by the Large EAC/AA system. The system compiles a complete list of the instances of excessive consumption that currently exist in NHHDA.
From the reports the system produces, Suppliers can notify us of consumption values they believe are genuine. The Supplier/NHHDC can notify us that a specific value (EAC or AA) is correct and provide a reason why that Metering System genuinely consumes over the threshold. Instances of genuine high consumption confirmed by the relevant Supplier are removed from materiality calculations.
Figure 2: The Large EAC/AA System and the Monthly Process
The Large EAC/AA system is the data source for all the reports that Elexon produces on the issue. We produce a number of standard monthly reports on the large EAC/AA issue:
| | A report for Suppliers and NHHDCs identifying all instances of large EACs and AAs that exist in Settlement. The Instance Report is explained in more detail later in this section. |
| | Graphs showing the Supplier Energy Allocation Error (SEAE) by GSP Group for each Settlement month. These are used to make decisions on Entry and Exit from large EAC/AA Disputes, and where any GSP Group is in a large EAC/AA Dispute, authorise DF Settlement Runs. The Trading Disputes Process is described in greater detail later in Section 12 of the guidance. |
| | A report for Suppliers and the Performance Assurance Board (PAB). These show gross error figures by Supplier and are used to monitor performance. The PAB’s monitoring process is described in greater detail in Section 10 of the guidance. |
Measures of Materiality – SEAE and GUEE
Elexon uses two measures to report the Large EAC/AA issue: Supplier Energy Allocation Error (SEAE) and Gross Uncorrected Energy Error (GUEE). Both measures are produced by the Large EAC/AA system.
SEAE is a measurement to show the Settlement impact of a particular Supplier’s error on other Suppliers. It incorporates the effects of GSP Group Correction Factors and the netting of positive and negative errors within the Supplier calculations.
GUEE indicates the extent of the issue in terms of the amount of energy that is incorrect. It does not net positive and negative values and does not take into account the effects of GSP Group Correction Factor.
SEAE is used by the Auditor as a primary measure of materiality. The materiality reports use GUEE because Suppliers can measure reductions in gross error without reference to the errors produced by other Suppliers.
The Instance Report details, for a given Supplier or NHHDC, all the EACs and AAs that exist in NHHDA that are above Elexon’s excessive consumption thresholds (see figure 1). Its purpose is to inform Suppliers and NHHDCs of the errors that need to be addressed.
For each large EAC or AA identified on the Instance Report full Settlement details are provided, including GSP Group, NHHDC ID, Supplier ID, MSID, Profile Class, Standard Settlement Configuration (SSC) ID, Time Pattern Regime (TPR) ID, Effective From Settlement Date and Effective to Settlement Date. In addition, the Report contains several system-assigned fields:
| | Indicates whether the consumption is an EAC (E) or an AA (A). |
| | These indicate the dates where the instance is being used in Settlement. This will not necessarily be the same as the Effective to and From Dates (any difference between the two is usually due to a change of NHHDA appointment) |
| | The annual consumption value of the EAC or AA. |
| | A realistic consumption value for the MSID. Where provided by the Supplier this will be the Annual Billed Units, otherwise a Class Average EAC will be used. |
| | This is assigned by the system and indicates the Elexon view of the consumption, whether genuine or erroneous. Process indicators range from 0 – 10. Process indicators 0, 6, 8 and 9 are treated as erroneous, all others are considered to be genuine. Appendix 1 includes a complete breakdown. |
Identify Genuine Consumption
There are a number of secondary issues that result in genuine EACs and AAs above the Elexon thresholds. These include incorrect Measurement or Profile Class, erratic consumption, or a long period of estimated consumption.
The Instance Report contains a complete list of all EACs and AAs in NHHDA that exceed the Large EAC/AA thresholds. This will include instances where the consumption is genuine. It is important that Suppliers and NHHDCs inform Elexon of genuine consumption to ensure the resulting monitoring information is accurate. We ask Suppliers and NHHDCs to submit an Excel file to Elexon each month confirming any consumption on the Instance report they believe to be genuine.
Once a
Metering System has been identified as being on the wrong
Profile Class or
Measurement Class and the instances flagged as genuine, they are not included in the materiality calculations. The secondary issue should still be addressed. Advice and guidance can be found in
Change of Measurement and Profile Class.
For repeatedly flagged genuine instances, Suppliers can also notify Elexon of annual billed units. These can be used to validate future excessive instances that occur for the same metering system. If any excessive instance is less than twice the annual billed units for that Metering System, the new instance will be confirmed as genuine and removed from any materiality calculations. This prevents Suppliers from repeatedly confirming instances for Metering Systems with consistently excessive consumption.
Instances will continue to be reported so long as they remain in the NHHDA database. Suppliers and agents need to adopt procedures to recognise reporting occurrences of high but genuine consumption (where the Elexon system is not able to do so automatically).
The BSC Auditor will check that any excessive EAC/AAs excluded from the materiality calculations have been excluded on reasonable grounds. Keep appropriate audit trails and make them available when you are audited.
Correcting erroneous large EACs and AAs is often a lengthy process and involves cooperation between Suppliers and Agents. Effectively managing corrective activity is crucial. Monitoring systems should include an action tracking and escalation process to ensure resolutions are successful. The Elexon Instance Report is a useful check of whether instances have been successfully cleared from NHHDA. The Instance Report can be the basis for managing corrective action. This can be enhanced by adding additional columns or by developing applications / databases into which the Instance Report can be loaded.
Managing Erroneous Large EACs and AAs is a largely manual process where staff have specialised skills. It is important to capture and document processes in Local Working Practices (LWIs) to ensure that the knowledge base is not hindered by staff turnover.
The Instance Report will in most cases contain a large number of instances. It includes all excessive consumption that exists in NHHDA databases. This includes instances that have crystallised and those that have previously been confirmed as genuine consumption. There are about 6,000 erroneous Large EACs and AAs in the market at any time, compared to some 200,000 that will be reported in total. It is crucial that Suppliers and NHHDCs can identify and prioritise corrective action.
All instances with a process indicator of 0 are considered to be erroneous. The remaining process indicators (1, 2, 3, 4, and 5) indicate that we consider the consumption to be genuine. Suppliers and NHHDCs should first focus on the erroneous process indicators, but should also, where possible, ensure that any instance with a process indicator of 1,3,4 or 5 is really genuine, as these process indicators are based on system rules, rather than Supplier/NHHDC investigation.
Suppliers and NHHDCs should be left with a manageable number of erroneous large EACs and AAs to resolve. Of these it is likely that a small proportion of very large instances will cause the majority of the error. Typically about 20% of the instances will cause 80% of the total error. Correcting erroneous large EACs and AAs is largely a manual process. It is therefore important that Suppliers and NHHDCs prioritise which they focus on.
Suppliers and NHHDCs should aim to resolve all error before it crystallises. It is therefore essential to prioritise instances approaching the relevant last reconciliation Run whether RF or DF.
We have created a tool to use in conjunction with the Instance Report to prioritise work. This is called the Large EAC/AA Materiality Calculator, and is provided along with the monthly large EAC/AA report, which is emailed to a contact the
Supplier nominates. If you are not receiving the report and believe you should be, speak to your Operational Support Manager, or email
large.aa@elexon.co.uk.
The calculator recreates Elexon’s error materiality calculation for any large EACs and AAs in the Instance Report. It does so for any given range of Settlement Dates. The result is a total positive or negative for the given period. This can be sorted to identify the MSIDs contributing the greatest error.
An audit trail must be maintained irrespective of the approach taken.
Incorrectly Processed Readings
If the reading is valid but it has been incorrectly processed to give erroneous EAC/AAs (for example in an incorrectly identified meter rollover scenario) and these erroneous EAC/AAs have not been used in the last reconciliation of a Settlement Date, correctly re-process the reading.
If the EAC/AAs derived from the invalid reading were not used in the last reconciliation of any Settlement Date, the invalid reading should be withdrawn. Once withdrawn, re-process subsequent readings. It is important to check the subsequent readings don’t generate further erroneous large EACs or AAs.
When an invalid CoS reading occurs, Suppliers can agree a revised reading through the Disputed Read process (MRA Agreed Procedure 08). If the agreed revised reading is reasonable, this should be processed.
Suppliers and their NHHDCs should ensure that an erroneous large EAC/AA caused by an invalid change of Supplier reading is resolved before the EAC/AA is used in the last reconciliation of a Settlement Date. When the erroneous CoS reading is subject to the large EAC/AA Trading dispute, the Supplier can use the DF period to withdraw and correct the erroneous CoS reading.
Part-Crystallised Consumption
If an erroneous large EAC or AA is part-crystallised in
Settlement, Gross Volume Correction can be applied to minimise its impact in the fluid period. Gross Volume Correction (GVC) is a process described in
BSCP504: Section 4.14.
GVC Guidance is also available on the
BSC Website. Ideally, errors should be corrected prior to the last reconciliation run, making GVC unnecessary. GVC cannot be used to compensate for consumption that has completely crystallised in
Settlement, and should only be used where on-going read validation issues would result from the part-crystallized erroneous instance remaining in
Settlement.
Meter read quality is fundamental to reducing the number of erroneous large EACs and AAs. Accurate and frequent meter reads minimise the amount of remedial work required at a later date. Initial investment in training meter readers and developing high quality systems to support them can therefore result in lower running costs and better performance figures.
Training of meter readers should include an overview of the full set of implications of poor installation, and encourage a good understanding of the implications of incorrect readings, i.e. errors in customer bills, Settlement and Distribution Use of System (DUoS) charging.
Training and incentive schemes should focus on quality as well as quantity. This should be reflected in performance measures.
Monitoring should trap repeated errors, e.g. registers entered in the wrong order, tenth digit included and wrong meters read in multi-meter installations. The meter reader’s identifier should be attached to the readings to support the monitoring process. Results can be fed back to the meter reader, and incorporated into future training.
All meter readers’ reports must be reviewed and passed to the appropriate area for action. If they contain information necessary to successful reading of the meter, this should be available to future readers of the meter.
Some human error is unavoidable. NHHDC systems can identify erroneous meter readings. On-site controls in data collection hand held units can confirm the readings entered are accurate. However, they cannot confirm that the advance generated is valid as this requires additional information only available in the NHHDC’s main systems. We recommend that on-site validation is used as a complement, rather than a substitute for validation performed in the main systems.
Entry of meter readings that give rise to advances outside the expected range (half to twice the expected advance) should require positive verification, for example, re-keying the reading and entering the meter serial number. There should be additional safeguards where the reading is completely inconsistent with the expected value (e.g. five times the expected value or a credit reading).
NHHDC systems should include tight controls against erroneous readings. These should incorporate;
Reasonableness checks are incorporated into the EAC/AA calculation software used by NHHDCs. The tolerances used by the calculator are included in Market Domain Data (MDD) as an attached Word document. They do not form part of the D0269/270 or MDD online. DCs can use thresholds lower than these when calculating EAC/AAs, but cannot use thresholds that are higher.
Elexon’s thresholds for the reporting of large EAC/AAs are shown in Figure 1. These thresholds are currently the same as the thresholds used by NHHDCs in calculating large EAC/AAs.
Associated changes to the BSCPs and Party Service Line 100 place obligations on NHHDCs to investigate all AAs exceeding the tolerances, and report actions to resolve the exception to the relevant Supplier. As the EAC/AA database does not include Supplier details, NHHDCs will need to align the exception log of excessively large AAs produced by the EAC/AA software with their appointment data. The exceptions can then be distributed to the relevant Suppliers and the status of each reported following initial investigation.
Expected Reading Calculation
Meter read validation should include a comparison of the calculated meter advance with the expected advance. The reading is rejected when the difference between the calculated advance and the expected advance exceeds the tolerance. This process relies on reasonable accuracy of the expected advance. Some NHHDCs cannot always establish a reasonably accurate expected advance. This is often due to incorrectly migrated data, incorrect initial EACs, lack of consumption history and/or incorrect acceptance of a previous reading.
We recommend that NHHDCs review their method of establishing the expected advance. Ideally, it should be calculated using the current EAC and profile data, although any equivalent process can be used. When calculating expected values before profile data is produced, use substitute profile data from the previous year. For new connections, it is important to establish an expected consumption to calculate an expected advance. NHHDCs and Suppliers must provide a realistic initial EAC for this.
We advise Suppliers to feedback to the NHHDC when readings the NHHDC marked as valid are not accurate enough for Supplier billing. The NHHDC can then consider withdrawing the reading from Settlement.
Conversely, provide feedback when readings marked as invalid by the NHHDC are accurate enough for Supplier billing. In these situations the NHHDC should not automatically accept the reading. Instead the reading should be reviewed, taking the Supplier feedback into consideration, and should only enter Settlement if the reviewer is satisfied that it is correct. It may be appropriate to accept a reading that generates a meter advance three times the expected advance on the basis of the Supplier’s feedback. It isn’t appropriate to accept a reading that generates a meter advance of one hundred times the expected advance, solely on the basis of the Supplier’s feedback.
Review of Suspect Readings
Where a reading fails automated validation, it can be overridden via a manual review process. It is important that this process is robust and readings aren‘t accepted too readily. For example, if a NHHDC is under pressure (e.g. through commercial incentives) to progress as many readings as possible to help the Supplier achieve performance levels for the Energy on Annual Advances (serial SP08), this can result in poor readings going through the Suspect reading review process. Suppliers should incentivise the quality as well as the quantity of readings.
It is important that reviewers have enough information to determine whether readings are genuine. This includes a complete history of Meter Readings and the associated EACs and AAs for each register on the Metering System. It is also useful to have a record of what type of reading each is, for example, Deemed, Customer Own Reads or corrective (GVC). Where possible, Supplier billed units, the business type and details of any previous Change of Agent or Supplier activity are a useful resource.
Filters have an important role in preventing erroneous large EACs/AAs from entering Settlement. However, they only remove the main symptom of the problem – the erroneous large EACs/AAs – they do not address the root cause. It is important that root causes are identified and addressed. NHHDC systems should contain feedback loops to correct errors that led to erroneously large EACs/AAs. If they don’t, the NHHDC system is left with an invalid reading, an incorrect expectation for the next advance and is out of step with the NHHDA system.
For all exceptions identified by filters it is important that NHHDCs resolve the instance moving forward. Identified root causes should be investigated and preventative controls put in place. Some known issues include:
It is common that
Meter readings on multi-register
Meters are switched over by the
Meter Reader. This often leads to the generation of two erroneous large AAs. In this case the readings can be switched back and re-validated - as described in
BSCP504.
Our analysis has shown that multi-register Meters, while accounting for only 20% of the market, contribute almost half of the erroneous large EACs and AAs in Settlement. Suppliers and NHHDCs should ensure that they have robust processes for identifying and correcting switch register readings.
Where meter exchanges are not processed in a timely fashion, meter readings for the new meter can enter data collection systems before the final and initial readings associated with the meter change. This can cause the NHHDC to identify a meter rollover incorrectly.
By implementing controls to trap such situations NHHDCs can prevent readings for a new meter being processed before its initial reading being processed. We also advise that metering management information be generated from this, so that the NHHDC can report where excessive delays occur and where excessive numbers of these delays occur.
Step changes in consumption following a meter exchange may be due to incorrect Meter Technical Details (e.g. Pulse Multiplier) for the new meter. Potential pulse multiplier errors should be checked with the Meter Operator.
Meter Rollover Algorithms
Failure to identify a ‘rolled over’ or ‘clocked around’ meter can lead to incorrect meter advances. Depending on what the previous reading was, this can lead to advances that have the same number of digits as the meter. In the worst cases, credit readings (readings that are less than the previous reading) are automatically assumed to be a positive advance caused by a meter rollover.
We recommend NHHDCs check their meter rollover algorithm. It is important that errors such as swapped registers and incorrect number of digits do not cause a meter rollover to be incorrectly identified. It is also worth checking that if an algorithm is specified more than once, for example in validation and later on during processing, that they are the same.
If an erroneous Change of Supplier occurs, it is usually rectified by a Change of Supplier back to the original Supplier; this is done for the next day and with a meter advance of one unit. This one unit is usually below the expected advance range for one day and consequently fails validation.
If not reviewed and accepted promptly, it can be automatically replaced by a cyclical or customer own reading that is available or a deemed reading may be triggered. This can cause the meter advance to be more than one unit.
Any consumption errors between the two Change of Supplier readings are concentrated into a meter advance of one day and can lead to very large AAs. These consumption errors may be caused by a deemed reading for the first Change of Supplier that underestimates the reading, followed by a cyclical or customer own reading for the second.
We advise NHHDCs to check that their procedures surrounding treatment of erroneous transfers are appropriate.
Gross Volume Correction is described in the section on correcting error. However it can lead to an excessive value, especially if the error in the “crystallised” period is compensated for in a short period.
Where readings are not obtained for a Metering System for a long time, the consumption in Settlement is unlikely to reflect the actual consumption. When a reading is obtained it may result in a large corrective reading that breaches the excessive consumption thresholds. The result is likely to be more extreme when a meter is de-energised or disconnected. This can result in a compensatory negative advance, which can be extremely large if applied over a short period.
Daily Profile Coefficients
Daily Profile Coefficients normally sum to one over a year. However, DPCs can sum to less than one in extreme cases. This usually occurs for seasonal registers, where a register is only active for a few hours of the day for a few months of the year. The correct volume of energy will be settled, but the AA will be too high. These values are genuine. However, if readings are transposed on a seasonal time of Day (SToD) Meter, this can lead to large values of erroneous consumption.
Suppliers should have controls to monitor the Large EAC/AA issue in addition to Elexon’s reporting. Monitoring D0019 flows sent by NHHDCs for values that appear to be unreasonable is one way of doing this. Initially, the Supplier should monitor any values that exceed Elexon’s monitoring thresholds. We will include these in error materiality calculations unless the values are confirmed as genuine. However, once these levels are under control the Supplier should monitor below the Elexon thresholds.
Suppliers systems should link D0019s to their billing system as an additional check. Suppliers’ billing systems typically include a comprehensive range of checks to minimise the number of inaccurate bills sent to customers. Whether a reading is suitable to bill a customer is an extremely good independent check on the validity of the reading. D0019s can also be compared against previous D0019s for the same register and any extreme changes to the level of consumption can be identified.
The Performance Assurance Board and Supplier Performance Monitoring
At its meeting on 27 April 2023, the Performance Assurance Board (PAB) approved the deployment or Error and Failure Resolution (EFR) for Risk 8, based on the Large EAC/AA Materiality Report.
The deployment of EFR will be based around a Supplier ID being above or below the materiality threshold located in its monthly Materiality Report. This threshold is determined by multiplying market-level error by the Supplier ID’s Non Half Hourly (NHH) energy share down to a minimum threshold of 3,300MWh.
Suppliers will be considered for EFR if they spend three consecutive months above threshold to trigger assessment from the EFR Analyst.
If the sending of the Instance and Materiality reports for a given month are delayed, and such delays affect the Party’s ability to resolve instances, the following month will be omitted from the three month count and the next month of timely correct reporting sent will be counted instead.
Parties in EFR will be considered for exit following three consecutive months below the threshold.
If the sending of the Instance and Materiality reports for a given month are delayed, and such delays affect the Party’s ability to resolve instances such that it breaches the threshold, this month will not be considered as having broken a run of consecutive months below the threshold
Elexon will contact those Suppliers that are currently above the threshold for the March 2023 reporting period shortly after the sending of the April 2023 Materiality Reports to confirm that two further months above the threshold will mean they will be assessed for EFR.
The PAB Error Calculation explained
To calculate Supplier error for PAB reporting purposes, we aggregate the error from individual instances reported by NHHDAs against each Supplier ID and add it together, ignoring the sign of the error. So, for example, a Supplier ID with 50MWh and -50MWh of error against two instances would have 100MWh of error reported for PAB purposes.
The PAB only monitor error that has been in our reporting for more than a month. We usually refer to this as ‘Updated’ error.
The PAB only consider error inside of a 15-month window in Settlement. Error within the 3 months immediately post-RF, and the 12 months immediately pre-RF, is added up to produce a single error figure for each Supplier ID. This post-RF window is due to the ability to correct error post-RF through the Trading Disputes process.
Supplier Error Thresholds for the PAB
The BSC Auditor has determined a level of acceptable error for the entire market, which is 0.5% of annual demand, or 1.65TWh. Risk 008 has been allocated 10% of the BSC Auditor’s total value of acceptable error in the market, or 165,000MWh.
Individual Supplier ID thresholds are determined by multiplying market-level error by the NHH energy share of the Supplier ID. For example, a Supplier ID with a NHH energy share of 20% would be allocated an acceptable error threshold of 33,000MWh (165,000MWh x 0.2).
The only exception to this is where a Supplier ID has less than 2% NHH energy share, in which case it will be given a minimum threshold of 3,300MWh (equivalent to the threshold allocated at 2% NHH energy share).
We provide Suppliers with information on their performance for PAB purposes through the Materiality report. This report provide a view of each Supplier IDs threshold for error, compared to their actual error levels, across the last six months of reporting. We also provide a peer comparison, based on the difference between error threshold and reported error.
Error and Failure Resolution
This section will help you to understand the process of EFR, what it means for your organisation to be put into EFR and how to improve your performance to come out of it.
What are the major steps to follow when put into EFR?
When put into EFR, action plans will need to be implemented. They consist of a range of actions that will address the error(s) or failure(s). These will be broken down into milestones and corresponding target dates. All plans should be submitted in accordance with
BSCP538 which includes the action plan form at the end of it.
The action plan should be prepared following the next processes:
Look at the Elexon’s monthly instance report to determine the root cause(s) of poor performance. Once you have located the weakness(es), investigate each one to find the reason(s) why your performance level is low. It is essential that you focus primarily on the common causes of poor performance, as described in preceding sections of this guidance.
Categorise each one of the root causes and explain to us how you will address them specifically, e.g. introduce new controls to the DC validation process. You will also need to establish a clear set of timescales for completion of each action, and when it will be reported to the PAB and Elexon.
Establish clear deadlines for predicted reduction in error to gradually reach the required level of performance across the agreed reporting period.
The following timeline describes the different steps of the EFR process:
In order to come out of EFR, you will have to comply with the following criteria:
Three consecutive months of ‘GREEN’ performance against the relevant Settlement Risk (SR0072)
Successful and timely completion of the action plan, communicated effectively on the milestone due dates
The Trading Disputes process
Post Final Reconciliation Settlement Runs
The Erroneous Large EAC and AA Trading Disputes enable Suppliers and Supplier Agents to correct instances of erroneous large consumption after the Reconciliation Final (RF) Settlement Run. As part of the Trading Disputes Process the Trading Disputes Committee (TDC) authorises Post Final (DF) Settlement Runs approximately 14 months after RF. Suppliers and Supplier Agents have until the DF Run to correct the erroneous large consumption in Settlement using the corrective procedures explained above.
The TDC authorises post RF changes to Settlement Data for Metering Systems that exceed Elexon’s monitoring thresholds. These instances are identified by Elexon in the xxxx_instance.csv report. Suppliers and Agents can use the DF period to address the identified Large EACs and AAs.
Trading Dispute can be managed on a GSP Group level. Given consistently good performance a GSP Group may be granted exit from the Trading Disputes process. Conversely, poor performance may require a GSP Group to re-enter the Trading Disputes process. The TDC has agreed a set of Entry and Exit guidelines to determine whether Post-Final Settlement Runs should take place in a given GSP Group. These are included in Appendix 2.
It is important that Suppliers and Agents are aware of which GSP Groups are in the Disputes Process to ensure they take advantage of the DF period to clear Large EAC/AA errors.
Current information on which GSP Groups have DF Runs available for large EAC/AA error can be found on the ‘Disputes’ worksheet of the NHHDC and Supplier Instance Reports that Elexon provides on a monthly basis.
An authorisation process has been established for Dispute Final Runs. This process is used to ensure that Disputes are only run once the NHHDA databases are fit for purpose. This authorisation process is as follows:
A separate Dispute is raised for each Settlement Month in each GSP Group. Each month, Elexon presents a new Settlement Month to the TDC for authorisation. The TDC makes it is decision based on Elexon Monitoring Information. Each month Elexon produces monitoring information based on the results from the latest NHHDA extract scripts and the Supplier and NHHDC confirmations of genuine consumption. This shows for each GSP Group the level of crystallised SEAE at RF and DF with a forward projection of the SEAE that will crystallise for future Settlement Months. All SEAE levels are compared to a threshold of ‘acceptable’ error.
Elexon’s monitoring information enables the TDC to see a projection of the error that will crystallise should the DF run be carried out for each disputed period. This is used to determine whether to process the run, i.e. is there sufficient benefit to justify the Run. If Settlement Data in a GSP Group deteriorates after the RF Run is processed, the TDC may decide not to authorise the DF Run for a specific range of Settlement Dates.
The projections give the TDC early visibility of any performance issues that may occur. These are communicated to the industry and action taken. Where performance is sufficient, the TDC may provisionally authorise a DF Run in a GSP Group well ahead of the NHHDA Run.
The industry is informed of TDC decisions including GSP Group Entries or Exits through the Newscast.
The Trading Disputes timetable is not included in the Data Aggregation and Settlement Timetable (D0286) flow that some NHHDAs use to electronically schedule aggregation runs. Consequently, all NHHDAs need to manually schedule the aggregation runs required for Trading Disputes. Guidance on this is included in the NHHDA Operations Guide.
The Settlement Code used for these Post Final Settlement Runs is “DF”. We ask NHHDAs to carry out DF Runs for all GSP Groups whether the GSP Group is in or out of the Disputes Process. The SVA Agent controls which DF files are submitted to the SVA Run. Where a GSP Group is not to carry out a DF Run the SVA Agent will process the RF file in its place.
The Trading Disputes timetable contains a contingency period between the NHHDA aggregation run and SVA run. This contingency period provides time to resolve any operational issues before the SVA Agent performs the Settlement run. NHHDCs and NHHDAs should notify Elexon as soon as a problem appears so appropriate guidance can be given.
Intellectual Property Rights, Copyright and Disclaimer The copyright and other intellectual property rights in this document are vested in Elexon or appear with the consent of the copyright owner. These materials are made available for you for the purposes of your participation in the electricity industry. If you have an interest in the electricity industry, you may view, download, copy, distribute, modify, transmit, publish, sell or create derivative works (in whatever format) from this document or in other cases use for personal academic or other non-commercial purposes. All copyright and other proprietary notices contained in the document must be retained on any copy you make. All other rights of the copyright owner not expressly dealt with above are reserved. No representation, warranty or guarantee is made that the information in this document is accurate or complete. While care is taken in the collection and provision of this information, Elexon Limited shall not be liable for any errors, omissions, misstatements or mistakes in any information or damages resulting from the use of this information or action taken in reliance on it. |
Appendix 1: Process Indicators
Process Indicator | Elexon View | Explanation |
0 | Erroneous | No information has been provided by NHHDC or Supplier to suggest that the EAC/AA is genuine. |
1 | Genuine | The EAC/AA is within twice the MPAN's Annual Billed units. |
2 | Genuine | The Supplier has advised that they and their NHHDC believe the EAC/AA is correct. |
3 | Genuine | The AA*Meter Advance Period ≤ 500,000kwh (PC1-4) or ≤ 1,000,000 (PC5-8); |
4 | Genuine | the DPCs for the settlement register sum to < 0.5 and the EAC/AA ≤ 500,000kwh (PC1-4) or ≤ 1,000,000 (PC5-8); |
5 | Genuine | The EAC/AA is within twice a previous EAC/AA that the Supplier has confirmed as genuine; |
Appendix 2: Guidelines for Entry to and Exit from Trading Disputes
At its meeting of 03 May 2012, the TDC agreed the following guidelines to be used with the historical SEAE graphs when considering whether a GSP Group should be brought into the Trading Dispute process.
“A GSP Group may be brought into the Trading Disputes process if;
The level of monthly crystallised SEAE at Final Reconciliation exceeds the threshold for the last three consecutive months;
The level of monthly projected SEAE at Final Reconciliation exceeds the threshold for the next three consecutive months; and
Any further evidence that the TDC may deem necessary reveals that there is a material risk of SEAE going over threshold, if the GSP Group remains outside of the Disputes Process.
On 03 May 2012, the TDC approved revisions to the Exit Guidelines for granting a GSP Group an exit from the Disputes process (TDC151/01). The current guidelines are detailed in the table below:
Guideline | Evidence |
Erroneous AA and EAC values impacting Settlement Dates for which Dispute Runs would not take place have been substantially cleansed ahead of Final Reconciliation.
| |
The level of any residual error is consistent with any guidance on acceptable materiality obtained from the Panel or its subcommittees.
| Graphs showing the historic monthly SEAE performance of the GSP Group against the threshold. The GSP Group will be expected to be within the threshold at Final Reconciliation for a period of at least 3 consecutive months. The GSP Group should also be projected to stay within target for a further 3 month period if no further errors were introduced and no further data cleansing took place. The GSP Group will also be expected to show a long-term downward trend in SEAE levels, such that its projected error levels will not lead to it re-entering the Disputes Process within the next 6 months.
|
Controls and procedures have been put in place by the Non Half Hourly Data Collector(s) active in the GSP Group to prevent excessive levels of further erroneous AA and EAC values being submitted to Settlement or that they can be withdrawn ahead of the relevant Final Reconciliation Runs.
| |
The Non Half Hourly Data Collector(s) active in the GSP Group is in the position to implement the principle of “Gross Volume Correction” (as defined in NHHTAG/116/04) in the eventuality that any erroneous EAC and AA values are used in Final Reconciliation of Settlement Dates that would not be subject to Dispute Final Runs.
| Written confirmation received from the major NHHDCs that GVC can be carried out at RF. A central check will be conducted on a sample from data provided by the DC/DA as requested, to ensure GVC is not being carried out post-RF.
|
Confirmation obtained that Suppliers active in the GSP Group during the Settlement Dates in question are satisfied that they and their agents have put the necessary processes in place to ensure that erroneous consumption has ceased to be a material issue.
| |
The NHHDC must have rectification plans in place to address the root causes of the issue.
| If there is an Error and Failure Resolution (EFR) Action Plan held by Elexon, then progress against the plan is to be satisfactory. Consideration will also be given to any other relevant performance issues captured within the Audit or the Technical Assurance processes or the PAB escalation cycle.
|
A communications process must be in place between the NHHDC and all relevant Suppliers to assist in the resolution of erroneously large AA/EAC values.
| |
Appendix 3: Use of the Instance Report and the Materiality Calculator
Every month, based on timescales published in the
Large EAC AA Reporting Calendar, Elexon sends active
Suppliers an Instance Report and the Materiality Calculator.
Your Supplier Instance Report contains a breakdown of the instances which have been reported by your NHHDA(s) in that month, against your Supplier ID, as being over the threshold for their Profile Class (Appendix A lists these thresholds). In most cases, Elexon then expects Suppliers to confirm genuine consumption to it using a Supplier Confirmed file, which is created in the Materiality Calculator.
Instances that you/your NHHDC determine to be error need to have their associated EAC/AA withdrawn and replaced by your NHHDC. At that point the instance should disappear from our reporting, because it will be replaced in NHHDA systems with a genuine EAC/AA.
The Materiality Calculator allows you to calculate the impact instances are having across a period that you specify, e.g. the 12 months post-RF and pre-RF. It also doubles as a tool for producing two files which Suppliers can submit to Elexon to confirm genuine consumption.
The Supplier Instance Report
The Supplier Instance Report is your primary tool for understanding where reported Large EAC AA error comes from. Firstly, it provides you with a summary of current Disputes status for all GSP Groups, a table summarising your instances for the month, a bar graph that provides a breakdown of how these instances are distributed and a table listing the GSP Groups with an open Large EAC AA Dispute (see Appendix B).
The ‘Instances’ worksheet is the most important, as it contains MSID-level information needed to investigate instances.
Column J will have either an A or an E in it, to indicate whether the instance relates to an Annualised Advance (AA) or an Estimate of Annual Consumption (EAC). Column M contains the size of the instance as reported by your NHHDA, in KWh. Column N, ‘Realistic’, is either the Profile Class average consumption for that instance, or that MSIDs ABU value, both in KWh.
Column O, ‘Status’, tells you if Elexon currently believes the instance to be error or genuine consumption. An instance is labelled as error where the system has no reason to believe it is genuine consumption.
There are three common reasons why the system might believe the instance is genuine consumption. If you have previously submitted an ABU file against the instance’s MSID, anything within twice the value of that ABU will be flagged as genuine. If you have previously confirmed an instance as genuine consumption against the same MSID, any new instance within twice the value of the confirmed instance will be flagged as genuine. Anything a Supplier tells us is genuine consumption will also be flagged as genuine. A full list of reasons for files being flagged as genuine can be found in Appendix 1.
Column Q, ‘Previous Flag’, tells you if this is the first month in which an instance has been reported. An instance with an ‘N’ in this column is ‘new’ this month. An instance with a ‘U’ in this column is ‘updated’, i.e. has appeared in our reporting before. ‘New’ instances do not count toward your Large EAC AA error performance for that month. However, if they aren’t confirmed as genuine or corrected as error by the following month, they will become ‘Updated’ and count toward your reported error levels.
Columns R and F tell you how many days you have left to confirm or correct an instance. If a
GSP Group has an open Large EAC AA
Trading Dispute, you have until DF to confirm or correct instances. Currently, there are no open Disputes, and instances cannot be corrected after the RF run, except in limited circumstances through the use of
Gross Volume Correction (GVC) or through an upheld
Trading Dispute.
Column T tells you the absolute size of the instance, in MWh. It is worth noting that columns M and N are in KWh. The realistic value in Column N is subtracted from the excessive value in Column M and then divided by one thousand to give the value in MWh value of error seen in Column T.
The Materiality Calculator
The Materiality Calculator has two functions. Firstly, it acts as a way for Suppliers to see the impact of their reported error across a specified period of time. Since the Performance Assurance Board measure Supplier performance across a 15-month window, you may need to be able to see the impact of instances across only this period when prioritising instances to correct. Secondly, the Materiality Calculator includes two macros you can use to generate files for submission to Elexon.
Calculating error for specific date ranges using the Materiality Calculator
To use the Materiality Calculator you will need your most recent Instance Report (and a copy of the Materiality Calculator!). Open the Instance Report, go to the ‘Instances’ worksheet, and copy Columns A-T for all of the instances you want to investigate (Columns U and V are purely for your use in communicating internally or with your Agents).
Columns U to AC in the Materiality Calculator are automatically populated using the information you copied into Columns A-T above. The most useful column is AC, which tells you the absolute impact of the instance, adjusted for the dates input into cells B1 and D1.
So, for example, if you wanted to work out the impact of the instances we reported in December 2013 reporting month, for only the 15-month period that the PAB monitors, you would check the Materiality Report (which we send you with the Instance Report each month) for the relevant date range, and then input the first and last dates into cells B1 and D1. Absolute Impact in AC will then change to a value for just the reported period (note that the absolute impact in T will remain the same, as this has just been copied across from your report).
To create a Supplier Confirmed Consumption file with the calculator, you first need to select the instances that you wish to confirm as genuine. To do this, change the ‘N’ in all relevant cells in Column AD to a ‘Y’.
Creating a Supplier Confirmed Consumption File
Next, you should press the button called ‘Create SCC’ in cells I2 and I3. You will then be prompted for a location to which to save the Supplier Confirmed Consumption file. This file can then be sent to us without amendment. If you open the file for any reason, ensure that the MSID column does not default to scientific format for display of numbers. If it does, we will receive a file with rounded MSIDs, i.e. some of the digits at the end of the MSID number will be rounded to zeroes, and the consumption values you confirm will not be discounted from our reporting.
Creating an Annual Billed Units File
To create an Annual Billed Unit file, you first need to input your annual billed units into the relevant cells in Column AE. Annual Billed Units should be submitted in KWh. You should then press the ‘Create ABU’ button in cells K2 and K3. You will then be prompted for a location to which to save the ABU file. As above, the file can be sent to us without amendment, but if you do open it, ensure that MSIDs do not save in scientific format.
