Building a clearer picture of the labour market by integrating survey and administrative data

Understanding what is happening in the labour market has become more challenging as different data sources have provided different signals about how the labour market is evolving. Today we are publishing early findings from research, which explores whether Pay As You Earn (PAYE) Real Time Information (RTI) administrative data can help us better understand potential non-response bias in the Labour Force Survey (LFS). The findings provide important new evidence, but further work is needed before we can assess whether and how these methods should be used in official statistics.  

Liz McKeown and Daniel Ayoubkhani explain what we have learned so far and how this work contributes to our broader efforts to build a more coherent picture of the labour market using surveys, administrative data and linked datasets. 

Exploring how we can identify and adjust for potential non-response bias 

Since late 2023, we have taken steps to improve response rates and achieved sample sizes for the Labour Force Survey (LFS). However, an important question has remained: how might the changing patterns of response observed during both the decline and subsequent recovery in LFS response have affected estimates of employment, unemployment and inactivity over time? Understanding this is important because it may help explain some of the differences that have emerged between survey and administrative measures of the labour market in recent years.  

The research we are publishing today explores whether linking survey and administrative data can help us better understand that question. While this remains early-stage research, publishing the findings now allows us to share the emerging insights and engage users on how it should be developed further. The findings also help inform existing user guidance, including our recommendation that users draw on administrative data when assessing changes in employment. 

In this research, we linked HM Revenue and Customs PAYE RTI data to LFS records to identify patterns of employment-related non-response and assess how labour market estimates would change if PAYE RTI information were incorporated into the survey weighting process. By accounting for systematic differences between the LFS and PAYE RTI in terms of how employees are defined, this provides new evidence on how changing patterns of response may have affected published estimates of employment, unemployment and inactivity over time. 

What does the new research show?

The research finds evidence that employment-related non-response bias in the LFS has varied over time, helping explain periods of divergence between survey and administrative measures of the labour market. As patterns of response changed, so too did the estimated impact on labour market measures.  

Our early findings suggest that applying the methods proposed in the article would lower employment rates compared to the published LFS estimates during the peak of the pandemic and raise them between 2022 and 2024. At the beginning of 2024 when the achieved sample size of the LFS had only just started recovering from its low point of approximately 14,000 households in Great Britain, the research suggests an increase in the employment rate of 0.8 percentage points using our proposed PAYE RTI weighting methodology compared with the published series. Throughout 2025, the reweighting would not result in a substantial increase or decrease in the employment rate.   

 

The research therefore points to a stronger recovery in employment following the pandemic than suggested by the published LFS estimates alone. The corresponding inactivity rate appears lower than indicated by the published survey estimates, suggesting that the rise in inactivity following the pandemic may have been less pronounced than implied by the published LFS estimates alone. However, unemployment is affected to a much lesser extent by reweighting with PAYE RTI information.  

Importantly, this remains proof-of-concept research, and we are not today publishing revised official statistics. The findings do not provide a definitive answer about the “true” level of employment, unemployment or inactivity. However, they do provide valuable new evidence about how different labour market measures relate to one another and help us better understand some of the differences that have emerged between survey and administrative sources in recent years. 

The research also suggests that the differences between survey and administrative measures became much smaller during 2025. This is encouraging and coincides with a period of improvements in LFS collection and response rates. However, further work is needed to understand the factors behind this change, whether it will persist over time, and what it may mean for the future development of official labour market statistics 

These findings are also consistent with the guidance we have provided to users in recent labour market publications. Improvements to the LFS, which have tended to reduce the potential impact of non-response bias, have also complicated interpretation of measures of change within the survey across 2024 and 2025. For that reason, we have recommended that users draw on administrative PAYE RTI data when assessing changes in employment. 

By linking data at the microdata level, this research is able to account for definitional and conceptual differences between PAYE RTI and the LFS that reflect genuine differences in what each source measures. This helps explain why the estimated impact of non-response bias is smaller than that suggested in some previous analyses, including work published by the Resolution Foundation. 

What else are we doing? 

This research is part of a broader effort to improve how we bring together evidence from surveys, administrative data and linked datasets to understand the labour market. In this case, linking survey and administrative records allows us to investigate questions about non-response that would be difficult to address using either source on its own. More broadly, these developments are helping us better understand how different measures of the labour market relate to one another and what each can contribute to our understanding of the economy. 

This includes work we are undertaking with the Economic Statistics Centre of Excellence (ESCoE) to better understand how evidence from different labour market sources can be brought together to assess the direction of change. While the methods are different, the objective is very similar: making better use of information across multiple sources, particularly where one alone cannot provide a complete picture.  

Development of the Linked Employer-Employee Dataset (LEED) is highlighting the opportunities created by linking information about workers and firms, including what it can show about the relationship between labour market outcomes, business performance and productivity. Experimental outputs from LEED are due to be published later this year, offering new insights into job hires, separations, and flows among people in payrolled employment.  

We have recently published a new ‘component’ method of calculating labour productivity, which brings together the best elements of multiple labour market sources to deliver a significantly improved measure that aligns with international best practice. Although focused on a different question, it follows the same principle as the research published today: combining evidence from different sources to improve our understanding of the economy. 

We are also scoping the development of the Labour Accounts. Operating on a similar principle to the national accounts, these will bring together the best and most appropriate survey, administrative and linked data sources to provide a single, coherent view of the labour market. This administrative data research will be an important input into this framework. 

What comes next? 

This research demonstrates the value of bringing together survey and administrative data to better understand the strengths and limitations of different labour market measures. It offers important new evidence on the role that employment-related non-response may have played in recent years and helps us better understand periods when survey and administrative sources have provided different signals about the labour market. 

At the same time, this remains proof-of-concept research. Further statistical development and operational testing will be needed before we can assess whether these methods can play a role in the production of official labour market statistics.  

Users of our labour market statistics should continue to use the routinely published data, but results so far support our existing user guidance that PAYE RTI currently provides the most reliable measure of employees, particularly during the period of changing non-response bias implied by this research. 

Our plan is to develop this initial research further and to assess whether these methods could be applied to the Transformed Labour Force Survey (TLFS), the online replacement for the LFS, once the survey methods underpinning the TLFS and the resulting data are sufficiently mature post-transition.  

We will be engaging users of our statistics on this research and welcome feedback, please get in touch via email at Labour.Market.Transformation@ons.gov.uk to share your views.