Talk presented at the Conference of the Association of American Geographers, Tampa, April 8-12. First attempt at presenting a paper presently being written for publication.

1. Urban indicators, city benchmarking, and real-
time dashboards: Knowing and governing cities
through open and big data
Rob Kitchin, Tracey P. Lauriault, and Gavin McArdle
NIRSA, NUIM

2. Introduction
• Over past 25 years there has been a proliferation of urban
indicator and city benchmarking projects
• More recently such projects are becoming open, real-time,
and visualised through (interactive) dashboards
• This paper considers what these projects mean for how
cities are known and governed; how they are enrolled in
the production of smart cities
• We argue that indicator and benchmarking projects
promote a narrowly conceived but powerful realist
epistemology – the city as visualised numbers - that is
reshaping city governance

3. Framing
• Our discussion is framed by:
• a critical understanding of data that recognizes that
they do not exist independently of the ideas,
instruments, practices, contexts, knowledges and
systems used to generate, process and analyze them -
there is a politics to data assemblages, such as
indicator/benchmarking initiatives.

4. Data Assemblage
Attributes Elements
Systems of thought Modes of thinking, philosophies, theories, models, ideologies, rationalities, etc.
Forms of knowledge
Research texts, manuals, magazines, websites, experience, word of mouth, chat
forums, etc.
Finance Business models, investment, venture capital, grants, philanthropy, profit, etc.
Political economy Policy, tax regimes, public and political opinion, ethical considerations, etc.
Govern-mentalities /
Legalities
Data standards, file formats, system requirements, protocols, regulations, laws,
licensing, intellectual property regimes, etc.
Materialities &
infrastructures
Paper/pens, computers, digital devices, sensors, scanners, databases, networks,
servers, etc.
Practices Techniques, ways of doing, learned behaviours, scientific conventions, etc.
Organisations &
institutions
Archives, corporations, consultants, manufacturers, retailers, government agencies,
universities, conferences, clubs and societies, committees and boards, communities
of practice, etc.
Subjectivities &
communities
Of data producers, curators, managers, analysts, scientists, politicians, users,
citizens, etc.
Places
Labs, offices, field sites, data centres, server farms, business parks, etc, and their
agglomerations
Marketplace
For data, its derivatives (e.g., text, tables, graphs, maps), analysts, analytic
software, interpretations, etc.

5. Framing
• Our discussion is framed by:
• a critical understanding of data that recognizes that
they do not exist independently of the ideas,
instruments, practices, contexts, knowledges and
systems used to generate, process and analyze them -
there is a politics to data assemblages, such as
indicator/benchmarking initiatives.
• our own practices of creating/managing indicator
projects since 2005 through AIRO (All-Island Research
Observatory) and a new dashboard for Dublin City
Council – Dublin Dashboard (not yet launched)

6. • How’s Dublin Doing
• Dublin Indicators and Benchmarking tools
• Dublin Real-Time
• Real-time data from sensors across Dublin
• Dublin Mapped
• Detailed Census maps for 2006 & 2011
Census
• Dublin Planning
• Zoning and Planning Permission
• Dublin Near To Me
• Community and service accessibility maps
• Dublin Housing
• Various housing modules, commuter maps
• Dublin Reporting
• FixMyStreet/CityWatch
• Dublin Social
• Live map of activity in Dublin based on
social network interactions
• Dublin Data Stores
• Access to data used in the dashboard
• Dublin Modelled
• Proposed modelling and Scenario tools
• Dublin Apps
• Directory of apps relevant to Dublin
• Have Your Say
• Feedback from Users

7. Indicators, benchmarking and real-time dashboards

8. Indicators
• Different types of indicators,
generated for varying
purposes
• Single direct and indirect
indicators
• Composite indicators
• Descriptive/contextual
indicators
• Diagnostic, performance and
target indicators
• Predictive and conditional
indicators

9. City benchmarking
• Standardized indicators for comparison within
and across cities.
• Enables performance to be benchmarked with
other places and against best practice; to
identify relative strengths and weaknesses
• Can produce league tables and ranks of relative
performance and to set targets.
• Used to formulate policy and undertake place
promotion
• JLL detail over 150 city benchmarking
initiatives.
• cityindicators.org is a joint project of The World
Bank, UN-Habitat, the World Economic Forum,
OECD, the Government of Canada that are also
working on an ISO standard for city
benchmarking indicators.

10. Real-time dashboards
• Mainly real-time
operational data
• Generally feeding
control rooms,
sometimes open
• Big urban data
• Centro De
Operacoes
Prefeitura Do Rio
• 30 real-time
systems + public
administration +
crowdsourced data
• Surveillance +
dataveillance

11. Epistemology, politics of data, governance

12. Realist epistemology
• Indicators, benchmarking and dashboards promote a realist
epistemology by privileging a particular ontological framing (city
as numbers) and modes of analysis (data viz) with respect to
cities and their citizens
• Supposedly provide well defined measures that are:
• objective, neutral, value-free, and independent of external
influence;
• Systematic and continuous in operation and coverage (rather than
one off and constrained by time, geography and limit sampling)
• verifiable and replicable;
• timely and traceable over time;
• easy, quick and cost-effective to collect, process and update
• easy to present, interpret, and to compare across locales through
interactive graphs/maps
• Makes claims with respect to the truth about urban systems and
life and has utility by facilitating action in relation to that
knowledge

13. Critique
• Not simply technical tools: they are framed socially,
political, ethically, philosophically in terms of their
form, selection, analysis and deployment
• Indicators express a normative notion about what
should be measured, for what reasons, and what
they should tell us - full of values and judgements
shaped by a range of views and contexts
• And they have normative effect - being used to
influence decision-making, modify institutional
behaviour, condition workers, etc. ... but also enact
Campbell’s Law

14. Critique
• Indicator projects promote an instrumental rationality based on a
narrowly framed episteme and techne that:
• is reductionist – atomizing complex, contingent relationships into
simplified, one-dimensional measures that do not give full picture;
decontextualizes the city from history, political economy and wider
set of social, economic and environmental relations
• undermines and replaces other scientific forms of urban knowing
that are less systematic and continuous – policy analysis, interviews,
focus groups, surveys, etc; as well phronesis (knowledge derived
from practice and deliberation) and metis (knowledge based on
experience).
• enables longitudinal analysis, but this often ignores the temporal
register of urban processes (that different processes and policies
work at different speeds) demanding quick change and response
• Full of absences and silences – phenomena that are difficult to
quantify or are politically contentious.

15. Critique - indicators
• Quality of indicators is dependent on veracity and
provenance of data
• rarely are indicators published with metadata concerning
measurement, sampling frame, handling, veracity (accuracy,
fidelity), uncertainty, error, bias, reliability, calibration,
lineage.
• Composite indicators can be opaque in method (aggregation,
normalization, weightings) and sources, quality and
commensurability of data; and can have issues of
multicollinearity and be highly sensitive to adjustment (e.g.
of weightings)
• There are spatial boundary issues (where is the city?) and
leakage (cities are open, porous systems)
• Somewhere in the translation from data indicators gain
confidence and stature and shed constraints and parameters.

16. Critique - benchmarking
• Difficulty of standardizing measures across jurisdictions
• Selection of indicators, parameters, weightings inherently tend to
favour some locales over others
• Is set up as a zero-sum game – ranked 1 to n
• Benchmarking assumes there is a normative standard by which places
should be judged, some ideal state they are all seeking to achieve
• Glosses over fact that phenomenon/places differ from one another
often for good reason, and that different places should have varying
goals/policy
• Places have different histories and trajectories, varying political
economies and varieties of capitalism, different forms of state
apparatus and governance structures
• Promotes imitation and copying rather contextualised policy
• Despite criticism indicator/benchmarking projects are being widely
rolled and translated into policy, planning and decision-making; this
reinforces the rationale for their use

17. Indicator initiatives and city governance

18. Governance
• How cities view indicators, the kinds of indicators that are chosen and
deployed, and how cities employ them varies markedly
• Some municipalities use indicators to underpin forms of new
managerialism, wherein they are used to guide operational practices
with respect to specified targets and to provide evidence of the success
or failure of schemes, policies, units and personnel
• Metrics are used to discipline under-performance, reward those
meeting and exceeding targets, and to guide new strategies, policy, and
budgeting.
• Technocratic, proscriptive and mechanistic
• Underpinned by/promotes neoliberalism
• Baltimore’s Citistat; Atlanta dashaboard
• “The Atlanta Dashboard ... uses weekly meetings of the mayor’s
cabinet to review performance reports. Each week the performance of
selected departments is reviewed against the departmental plan, with
programmatic changes formulated as necessary to address shortfalls.”

19. Governance
• In other cases, indicators are used in a more descriptive way to provide robust
and clear city intelligence, which complements a variety of other information,
to help inform policy making and implementation
• Here indicators are more contextual rather than performance/target
orientated.
• E.g., Dublin and Belgium
• Formulation of initiatives can be expert-led, consultancy-led, stakeholder-led,
community-participatory-led; business-led
• Many benchmarking initiatives are created by businesses and supra-national
bodies
• In all cases indicators form a key element in the move towards data-driven,
evidence-based policy formulation and operations management
• Open systems promotes transparency, accountability and participation; also
reinforces the value of the realist epistemology by promoting the value of
indicator data as the means through which the citizen can make sense of the
city and engage the state

20. Conclusion
• Indicator, benchmarking, dashboard projects have proliferated in recent
years; trend is for such projects to become more open and real-time
• Form a central pillar in the conception and roll-out of ‘smart cities’
• Promote a realist epistemology for knowing cities
• Translated into how cities are managed and governed
• However, how cities develop and utilise indicators projects varies markedly
• And within city administrations there are complex and paradoxical
processes at work: open vs closed, regulation/control vs
transparency/participation, etc.
• Smart cities coming into being in different ways – need a variegated and
more nuanced narrative
• Need some in-depth comparative work to explore the ways in which
indicator projects are deployed in different cities; to tease apart their data
assemblage

21. Rob.Kitchin@nuim.ie
@robkitchin
http://www.nuim.ie/progcity
@progcity
MIT Press, 2011 Sage, Aug 2014
Kitchin, R. (2014) The real-time city? Big data and smart urbanism. GeoJournal 79(1): 1-14