Today the ONS has published its latest annual estimates of employee earnings in the UK. Recognising recent challenges with the data, we have introduced methods and processing improvements this year, which have resulted in some bigger revisions than usual. Liz McKeown explains the impact of the changes and looks ahead to further improvements in the quality of the data.
The Annual Survey of Hours and Earnings (ASHE) is an important annual survey that provides a detailed picture of employees’ wages, obtained from employers. It provides vast amounts of data on who is earning what, their location, industry and type of job. It is one of our largest surveys – sampling around 1% of all National Insurance numbers being paid through HMRC’s Pay As You Earn (PAYE) system and it is widely used, including to analyse gender pay gaps and to inform the setting of the National Minimum Wage.
Different earnings estimates
Since the pandemic we have seen some divergence between the figures estimated from the ASHE survey and our other sources of earnings data. These other sources, notably the Average Weekly Earnings (AWE) – obtained from a separate survey of businesses – and HMRC PAYE RTI administrative data are more timely and frequent estimates and provide the best view of economy wide trends and changes. ASHE, however, uniquely provides very granular information, not available in these other sources.
The differences we have seen between these sources has prompted us to review the methods and processes underpinning the ASHE survey in order to ensure that we can explain and where possible address them, thus improving the overall quality of ASHE estimates.
Processing improvements
This year we have focused on the treatment of high earners across professions within the ASHE data set and reviewed how we process information about them. This has led us to introduce improvements to the methods for processing returns to this survey. The result has been more high earners in each profession being counted in the final data, thus increasing the achieved sample size and improving the quality of ASHE estimates.
We have applied these improvements to 2024 data, published today for the first time, and also to the previously published 2023 provisional results, which have been revised in line with normal practice. The impact of this is that we have larger than average revisions for 2023 data, with median hourly pay for full time employees rising from £17.40 to £17.52 and annual median earnings, for full time employees, now estimated to be £35,004, up from £34,963.
We will shortly be engaging with users about whether we can bring this new method in for 2022 and earlier data. In the meantime, we caution against comparing 2023 and 2024 with 2022 and earlier years.
Improvements still to come
We acknowledge, however, that while changes made this year have increased the quality of our ASHE estimates there are further improvements that we plan to introduce next year.
Our starting point for that is undertaking a complete review of how we validate and process the raw data. As well as this, we are developing new machine learning techniques that will better assess and triangulate the data to make it easier and quicker to assign people to the correct job types.
In addition, we are planning to assess our sampling and weighting methods, to ensure we are using as up-to-date estimates about the structure of the economy and employment as possible.
Finally, we are looking at transforming the technology that underpins the ASHE collection and statistical production to be able to process data more efficiently. This includes improving the way we engage with businesses that provide us with their employees’ wage information.
Recognising the importance of this data to users we are committed to fully engaging them with this work and its results throughout the year and will also look to draw on external expertise and research to further inform and underpin our planned improvements. This will include, importantly, continuing to make use of our Stakeholder Advisory Panel on Labour Market Statistics.
We also welcome any feedback on the plans set out above as we build on improvements introduced this year and continue to work to improve the quality of these important ASHE estimates of wages, and crucially pay distribution, in the UK.
Liz McKeown is Director of Economic Statistics at the Office for National Statistics.