National Statistical

At the ONS, we have been working hard to transform our consumer price statistics by reducing our reliance on traditional sources and utilising new digital data feeds as well as improving our methods and systems. This is both an exciting and challenging time for the ONS. Mike Hardie brings us up to date on the transformation journey.  

A bit of background  

Our headline inflation metrics measure the change in prices across a virtual ‘shopping basket’ of around 750 frequently purchased goods and services. This includes your staple shopping items such as bread and milk to haircuts, clothing and the cost of gas and electricity, to name but a few. With recent events impacting the economic landscape, such as the COVID-19 pandemic and Ukraine War, our inflation data have been closely monitored, not just by policymakers, but by consumers looking at what the inflation rate means for them.  

Therefore, it has never been more important for the ONS to transform and improve the way it tracks the movement of consumer prices. At the forefront of this work is our development of dynamic consumer price statistics using digital data sources.  Perhaps the biggest, most complex and challenging transformation element to our inflation statistics is the addition of supermarket scanner data. For 50% of the groceries market, instead of collecting 25,000 prices per month directly from shops by price collectors, we will use approximately 300 million price points derived from sales of over a billion units of products per month, collected directly from supermarket scanners at the checkouts or online. For the remaining 50% of the groceries market, we will continue to manually collect prices in-store and online.   

While not revealing any information about individuals’ purchases, harnessing these data will give us unique insights and new levels of detail about what food is being bought, how prices are changing and what this means for people’s shopping habits.  

This marks a step change in our approach to measuring price changes and highlights our commitment to ensuring we have the best methods and sources in place to measure our changing and increasing digital economy.     

An update on our plans 

We are acutely conscious of the importance of these consumer price statistics, which are used to set monetary and fiscal policy and uprate a wide range of financial products across the economy. The implementation of scanner data is a significant step change, so we are ensuring a well-paced, thorough, and transparent approach to embedding this data into our headline figures. The capability to use scanner data will be delivered as planned by March 2025; the new data sources, methods, and system have been thoroughly quality assured and are ready to go.   

However, following extensive collaboration with key users, it has been decided that scanner data will now become part of our headline consumer price statistics from March 2026. It was our original intention to fully incorporate grocery scanner data in our headline measures of consumer price inflation from March 2025. However, as this is the first time that the ONS is using administrative data of this size and complexity in the monthly publication of economic statistics, the revised schedule is prudent given their significance.  

We have been issuing regular updates on our transformation progress, through blogs and articles. It is vitally important that we continue to be upfront and transparent, including how and why we have had to make changes to our plans or timescales. By dual running the production processes throughout 2025, we can lay a firm foundation for going ‘live’ in March 2026.   

To demonstrate our new capability and show the value of scanner data, today we have published a selection of low-level indices, to showcase the impact of using groceries scanner data on the inflation rate for products such as cheddar cheese, whole milk, and fresh berries. The impact of using these new data is mixed, with some showing higher and others lower inflation and while we need further time to understand what’s driving these differences, we wanted to show what’s possible with the capability that’s been developed. You can find the indices here.   

What have we done already?  

Improving the quality and breadth of our consumer price inflation statistics is a long-term programme of work, and we have already delivered some key milestones in 2023 and 2024.  

Incorporating train fare data from the Rail Delivery Group has increased our number of price points from a single growth rate to around 40 million prices per month, covering all consumer rail fares transactions in Great Britain.   

We now use Auto Trader data to measure change in the price of second-hand cars, which increased the number of price points available to us from 105 to 300,000 per month.    

We have also increased the scope of our rental statistics, previously the ONS was only able to produce data broken down to the UK nations and English regions. However, the improved methodology allows the estimates to be broken down to local authority, number of bedrooms and type of property.   

What else will we be doing in 2025? 

In 2025, we will move the processing of traditional data sources to the cloud which will significantly reduce reliance on aged infrastructure and increase the resilience, sustainability, and efficiency of the consumer prices monthly production round.  Delivering new infrastructure allows us to support the delivery of large administrative data at pace in future and provides a solid foundation for continuous improvement of these vital statistics for years to come, ensuring we are producing our best ever estimates of changing prices across the economy. 

 The largest changes include improving our method for imputing the base price of new items. These improved methods changes will be introduced into our headline measures from  March 2025. You can find out more about our improved methods and their impacts here in our article, also published today:  Impact analysis on transformation of UK consumer price statistics: January 2025 

 Inflation statistics for the future 

The quality of our headline statistics is paramount, and we want to ensure that they remain trusted and fit for purpose accredited statistics. This revised schedule will allow us to futureproof the quality of our consumer price inflation statistics, transforming them for the digital age.