Department of Business, Energy and Industrial Strategy
Driving delivery of enhanced portfolio and reporting tools.
National Records of Scotland
Making sense of the census.
The ask
National Records of Scotland (NRS) is responsible for Scotland's Census, which takes place every 10 years. The information gathered helps build a picture of the country and is used to help inform important decisions made by the government and other service providers.
As digital delivery partner to NRS, we developed a new website for census data, ahead of the 2022 count. The site aims to engage audiences so that more people make use of outputs from the census. We were also tasked with delivering a data pipeline to process the massive dataset produced by the 2022 census, where 9 out of 10 questionnaires were submitted online.
What we did
An extensive user-centred design, development and testing process ensured the new website was as usable as possible, while keeping data secure. It needed to be fully accessible to ensure everyone in Scotland could use it. This was confirmed by the site passing the rigorous Digital Scotland Service Standard assessment.
The site allows users to easily search and filter data from previous census datasets. Users can search by year, location or topic, as well as compare data samples for similar topics from different years or locations. The site also asks users to provide feedback to help inform continual improvement.
Once the 2022 census collection was completed, we were tasked with helping process the huge volume of data received. We created a complex data pipeline that follows a sequential processing logic. The data was coded, cleansed, de-duplicated, edited, estimated, imputed and anonymised. The final output was a clean and complete data set that can be used for statistical analysis.
Results
The website was first launched as a public beta to get feedback from target service users ahead of live launch. The live website allows any interested parties, including the government, to more clearly and easily digest and understand the complex data produced by the census, and therefore understand Scotland better.
For the data pipeline, given the sensitivity of census data, security was a priority. Many of the security controls and configurations we implemented in Azure went far beyond the scope of the project itself. Which means any future project successes can be achieved with much greater velocity.
"The help and support received from Storm ID has been invaluable over the last few years and has consistently helped us deliver the best outcomes for NRS."
Related work
Crown Office and Procurator Fiscal Service
Transforming the Crown Office’s public and internal digital estate.
