An overview of the new ESRC Transformative research project given by Andy Newing to MRS Census and Geodemographic Group (CGG) hosted by GFK NOP, 19th November 2013 at Ludgate House, London
1. Census 2022: Transforming
Small Area Socio-Economic
Indicators through ‘Big Data’
An ESRC Transformative Research Project
Andy Newing and Ben Anderson
a.newing@soton.ac.uk
Sustainable Energy Research Group (SERG)
University of Southampton
5. Transforming Small Area Socio-
Economic Indicators through ‘Big Data’
5
Potential demise of decennial census
– Challenge: robust alternative methods for
creating small area socio-demographic and
socio-economic indicators
– Opportunity: transform the nature of these
indicators using new analytic methods and
datasets
Annual or sub-annual production?
6. Commercial transactional data
Geo-coded household data from utility
service providers - collected as part of
routine service provision
Can we derive traditional and novel small
area population estimates and social
indicators
Census-like and Census-plus
6
7. Owen. 2006. The rise of the machines—a
review of energy using products in the
home from the 1970s to today., Energy
Saving Trust, London.
7
8. What can utilities data reveal about
household characteristics?
Established links between household and
householder characteristics and energy
consumption (e.g. see AECOM 2011,
Druckman and Jackson, 2008)
Based on understanding relationship
between end use and end users
Consumption profiles reflect ownership and
use of appliances, household size and
characteristics and routines 8
9. What can utilities data reveal about
household characteristics?
Elexon. 2013. Load profiles and their use in Electricity Settement, Elexon, London.
9
10. What can utilities data reveal about
household characteristics?
10
Druckman, A. and Jackson, T. 2008.
Household energy consumption in the
UK: A highly geographically and socio-
economically disaggregated model.
Energy Policy, 36(8), pp.3177-3192
12. One-Minute Resolution Domestic
Electricity Use Data (2008-2009)
Household study
Sample of 22 households
1 minute resolution mean power import
Linked to (limited) survey data on
household characteristics
This dataset alone produces over 17m
recorded observations
12
13. 13Original dataset: Richardson, I. and Thomson, M., One-Minute Resolution Domestic Electricity Use Data, 2008-
2009 [computer file]. Colchester, Essex: UK Data Archive [distributor], October 2010. SN: 6583,
http://dx.doi.org/10.5255/UKDA-SN-6583-1.
14. 14Original dataset: Richardson, I. and Thomson, M., One-Minute Resolution Domestic Electricity Use Data,
2008-2009 [computer file]. Colchester, Essex: UK Data Archive [distributor], October 2010. SN: 6583,
http://dx.doi.org/10.5255/UKDA-SN-6583-1.
15. 15Original dataset: Richardson, I. and Thomson, M., One-Minute Resolution Domestic Electricity Use Data,
2008-2009 [computer file]. Colchester, Essex: UK Data Archive [distributor], October 2010. SN: 6583,
http://dx.doi.org/10.5255/UKDA-SN-6583-1.
16. 16
Original dataset: Richardson, I. and Thomson,
M., One-Minute Resolution Domestic Electricity Use
Data, 2008-2009 [computer file]. Colchester, Essex:
UK Data Archive [distributor], October 2010. SN:
6583, http://dx.doi.org/10.5255/UKDA-SN-6583-1.
17. UoS ‘Smart Meter’ Data
Household study in two areas (case and
control)
Up to 180 households over extended
period
Detailed survey of household
characteristics, behaviors and attitudes
Energy use (and other attributes) at one
second resolution 17
20. ‘Big Data’
Huge datasets
– UoS one second data produces over 500m
records per month!
– Aggregate and summarise but need to
understand missing data
– Nationally at 30 minute resolution – 94bn
readings per month
Need to sample
– Spatially representative and/or temporal
– New ways of thinking about census data 20
21. Scope
Not seeking to produce robust nationally
representative small area indicators ……
Identify available datasets
Focused exploratory work to demonstrate
the potential of these datasets
Develop methodologies and algorithms
Build up an interested expert stakeholder
group 21
23. Approaches
Time series and
periodicity –
repeating patterns
23
All can be explored using standard
settlement periods (30min) and alternative
temporal resolution – and disaggregation
by day, time of year etc.
24. Indicators
Traditional ‘census-type’
– household occupancy,
– household age structure
– household economic activity
– OAC classification?
….. based on energy consumption
Novel census-plus indicators
– energy inequality - energy consumption
gini coefficient? 24
25. Commercial Value
25
Almost all households consume electricity
New source of geo-coded address point
data with household attributes
Commercial transaction-driven ‘big data’
from utilities providers
Cannot mask consumption – reveals actual
habits and routines!
26. Impacts
Commercial data owners
Commercial data aggregators
Local and national authorities and similar
organisations
Future collaborations
26
28. Questions and discussion
To what extent is this type of thinking
already part of your business or
organisation?
What datasets of this nature do you have
access to or might be able to share?
What is your specific interest in work of this
nature?
Would you be interested in contributing
further?
28
29. Census 2022: Transforming
Small Area Socio-Economic
Indicators through ‘Big Data’
www.energy.soton.ac.uk
/category/research/energy-behaviour/census-
/
Andy Newing
a.newing@soton.ac.uk
Sustainable Energy Research Group (SERG)