ONS open microdata to more researchers – so long as public benefits

Will Leighton from PwC shares a researcher’s experience of working with ONS microdata

As a manager in PwC’s Data and Analytics team, I am all too familiar with analysing complex data on behalf of government and private sector clients. So when ONS updated its Approved Researcher scheme to extend access to its most detailed data to private sector organisations for the purpose of statistical research (and providing their research  delivers a public benefit to the UK), this presented a fantastic opportunity for us. PwC had been commissioned by London First (a not-for-profit consortium of businesses, including a number of important financial institutions ) to carry out analysis on the impact and contribution of migrant workers on the London economy.  This was an extremely important piece of research as the outcomes would inform government policy and decision making, including the arrangements for the UK’s exit from the European Union.

To gain access to the detailed ONS research data we were required to meet the accreditation criteria to be an ONS Approved Researcher (AR). I was able to demonstrate this with 5 years of experience in data analysis. We were also required to successfully complete a one-day Safe User of Research data Environments (SURE) training.  This was a very informative course and the ONS staff explained why and how we should keep data secure and prevent the disclosure of data subjects.  The administrative process to be approved as an AR is quite rigorous, but was very smooth and ONS staff were very helpful and responsive to our questions.

For our project, we identified that the most appropriate data for our analysis was the ONS Annual Population Survey (APS) – all years 2006 – 2016, the ONS Annual Business Survey (ABS) – all years from 2008, and the London Business Survey (LBS) 2014 that ONS carries out on behalf of the Greater London Authority. Our plan was to link and match de-identified data from the LBS and ABS at aggregate level with comparable characteristics from the APS/Labour Force Survey, in the ONS Virtual Microdata Laboratory (VML).

The goal of our study was to analyse the impact and contribution of migrants on London’s labour market, over the last ten years. Our scope was threefold:

  • Collect, aggregate and analyse of migrant worker data in London, e.g. length of time of stay and employment in London;
  • Evaluate the importance of migrant labour to the London economy, e.g. impact of overseas workers on London labour market by sector, type of role and skill level by EU and non-EU;
  • Assess of the overall economic contribution and impact of migrant workers to the London economy, e.g. the impact of migrants on wages, in different sectors and in a range of roles requiring different skill levels, and on the London housing market.

As our research proposal was from a private sector organisation using the AR scheme, it was referred to the National Statistician’s Data Ethics Advisory Committee (NSDEC) for advice on the ethical use of the data and the proposed public benefit. We worked very closely with the ONS Data Governance team to prepare the NSDEC application and the advice from the Committee was invaluable in helping us to develop our research methodology. Following the approval of the project, we had a very helpful meeting with the NSDEC Chair and Deputy National Statistician, Ian Cope to discuss the scope and timing of the project in more detail and explore areas where it would be helpful to collaborate.

Booking onto and using the VML was a very efficient process – and one far easier than agreeing not to use our mobiles phones or the internet whilst we accessed the data. Apparently we were cut off from the digital world for a total of fifty one hours and 10 minutes I was informed of by a colleague – not that he was counting! No access to the internet is a requirement of using the VML as the de-identified data is considered personal information and potentially disclosive – particularly if it is combined with other publicly available data.  Access to the requested datasets and our analysis was carried out in a designated folder within the VML and we were not permitted to remove or make a record of any data. We recognised the importance of these arrangements as they ensure that the confidentiality of data subjects is protected. All our data tables and analysis was checked by the VML statistical support team to ensure they weren’t disclosive. This was a fantastic help and provided us with reassurance about our research outputs.

The research findings were published on the PwC and London First websites on 2 March 2017 and concluded that London’s 1.8m full-time migrant workers each contribute £46,000 to the capital’s economy each year.  The total economic value is £83bn per year, roughly 22% of London’s Gross Valued Added (GVA). The study concludes that the growth of the capital’s EU population – at 7.7% a year over the last decade – has far outstripped the growth of London’s UK born population at 0.4%, and the non-EU population at 2.4%. These facts and many other findings in the report will be of immense value to government policy makers and to employers to inform their plans for how they are going to manage their changing resourcing and skills requirements given the combined effects of BREXIT and changes to the UK workforce.

Following publication of the findings, Julia Onslow Cole, Head of Global Immigration at PwC and also a member of Mayor’s Brexit Advisory Council, briefed the Mayor of London and London First met with senior Home Office officials.  The research outcomes were reported on by various media outlets including the Financial Times, City AM and the Evening Standard – which was hugely satisfying given all the hard work and mobile phoneless hours we had committed. We are also working with the ONS to produce a case study for the research to help highlight its impact on policy and decision making.

I would like to thank the ONS for making this data available for researchers and for their help in all stages of this fascinating and important project – access to the VML and subsequent support from the ONS allowed us to provide vastly greater value to our client, the government and the public. And thank you for also reducing my mobile phone bill for last December and January…

Will Leighton is a manager in the Data and Analytics team at PwC

Will’s views are his own and do not necessarily represent the views of PwC.