Revolutionising administrative data: a look into the future of population and migration statistics

Crowds of people walking through a busy street

Census 2021 has delivered a wealth of insight over recent months and will soon embark on an innovative new ‘open source’ phase. In addition, the Office for National Statistics is evolving to deliver more frequent, timely and relevant population statistics to better meet user needs. Underpinned by a suite of research and evidence, this year will see a call for views on our ambitious work. Here, Jen Woolford explains what’s coming in 2023. 

Using the breadth of census data, rich analysis has been building a fascinating picture of how we are changing as a society across England and Wales. This spring sees an exciting new phase when we will lift the lid on the depth of census data as we launch a flexible table builder. For the first time, this will allow users to look for interactions between topics which are important to them and build their own data sets. I’ll update on that in coming weeks. 

As we set out last year, we’re exploring how we could produce similar information – and address unmet user needs – more frequently than from a once-a-decade census. We want to ensure we are delivering a statistical system that is flexible and resilient enough to respond at pace to future and emerging issues. 

At the heart of this will be greater use of administrative data – existing data sources from across the public sector – to deliver the high-quality data and analysis needed to inform decisions.

Recently we have published feasibility research into characteristics and multivariate measures using new data sources and today sees the publication of the latest admin-based population estimates (ABPEs) and an update to our research into the dynamic population model (DPM), which aims to estimate population and population change in a timely way. The provisional results – this time for mid-year 2011 to 2022 for all 331 local authorities in England and Wales – show the potential of the new approach to produce estimates earlier than the current system.  

Alongside this we have also published a comparison of the official mid-2021 population estimates for England and Wales – based on Census 2021 data – with the estimate rolled forward since 2011 Census. Differences between population estimates based on Census 2021 data and annual mid-year population estimates are expected due to uncertainty building up over the decade.  

Following this reconciliation of our traditional Census based methods, we will conduct a rebasing exercise as we always do, providing a time-series of historical mid-year population estimates that benefit from the additional quality enabled by Census 2021 data and that are comparable with mid-year estimates produced after Census 2021. The DPM will provide insight to our rebasing exercise and will help us to assign differences in population estimates back across the decade to 2011.  

We have also been transforming our migration estimates. In November, we described progress we had made on a number of fronts and how our best estimates for long-term migration from June 2020 onwards are now based on administrative data sources. When the rebased population and migration statistics are published, we will show how our estimates compare with a number of migrant indicators to give users more confidence in the quality of our current estimates. 

The DPM uses both point in time (e.g. stock estimates from the SPD) and flow data (e.g. migration estimates) on an ongoing basis. Our ambition is to use a combination of sources for both stocks and flows to produce uncertainty measures for each. The DPM also uses these as part of the process to produce coherent population size estimates. This approach should substantially reduce the risk of drift, such that we no longer see a need for a whole population data collection ‘benchmark’ like the decennial census.  

To help users delve into comparisons between census-based and admin-based population estimates and Census 2021, we’ve also published a paper evaluating our progress towards a new system. This article compares these different estimates, explaining differences in quality and providing links to the more detailed articles that underpin these findings.  

Over the coming months, we will be providing a broad suite of case studies to evidence our capability of delivering our existing outputs differently, answering unmet user needs and improving the scope of longitudinal analysis. Today’s publications form part of this. In this, we will include research to better understand different populations, for example, workday populations at a local level and alternative definitions of international migrants that better reflect the reality of global mobility.  

We’re publishing this evidence for users to absorb and review the ongoing research so they can consider how potential new methods would match up to their needs. A new public consultation is planned for summer 2023, when we will be asking how far the proposal meets user needs for population and migration statistics and what we should prioritise in our ongoing research.    

The results of the consultation will inform a recommendation by the National Statistician at the end of the year on the future of population and migration statistics. This will include the role large surveys, or any future census could play in the statistical system of England & Wales in future. 

As we approach key milestones on this cutting-edge development of population statistics, more information will be shared. In the meantime, the census and the census based mid-year estimates provide the best picture of population.  

As we progress, we want to meaningfully engage with all users to ensure the future utility of our work. Please do share your feedback here 

Jen Woolford

Jen Woolford is Director Population Statistics