Does Measurement Make Change? Balancing Urban Indicators’ Potential and Risks

Allison Lalla

Indicators are increasingly used to track policy progress. Are they data-driven tools to drive change or regressive illusions? This article debates the need for indicators and proposes frameworks and solutions to maximize their utility.

The paradigm of sustainable development has held the world’s attention since the late 1980’s. Bridging the components of environment, economy, and social issues, it enables an integrated approach to deal with the multiple and intersecting issues of the 20th and 21st century. This is most recently reflected in the creation of the United Nations’ Sustainable Development Goals (SDGs), which identify 17 broad goals.

Coupled with the impetus to develop sustainably has been a growing desire to develop measurement metrics across all levels of government, from the United Nations to municipalities. Indicators have arisen as a preferred accompaniment to sustainability policies. Briefly defined, they are quantitative variables used to assess changes in a phenomenon or goal and track progress towards defined targets. While significant effort has been devoted to developing frameworks and guidelines to select appropriate indicators, there remains debate about their usefulness to prompt meaningful action.

Indicators have the potential be a useful legal tool to enforce meeting targets for both international and local objectives, including but not limited to the SDGs. Taking an approach centred around cities and the urban environment, I examine current barriers in employing indicators, the need for binding mechanisms to track and enforce agreements and goals, particularly for climate change, and possible methods to make indicators legally binding. I will conclude with a discussion and caution about the ability of indicators to either perpetuate or break down existing inequities and the ensuing importance of employing a justice-based framework for sustainability.

I. Current Barriers Urban Indicators Face

            A significant barrier to employing indicators as a coordinated and legally binding mechanism is the current heterogeneity of indicator sets across cities. In a survey of the sustainable development plans of 18 major Canadian and American cities and London, England, 913 distinct indicators were identified[1]. This suggests a strong emphasis on the individual character and autonomy of major cities, with little to no coordination across metropolitan areas. An immediate issue with this trend is the inability to make meaningful comparisons across cities, particularly if each indicator employs different units of measurement and data collection methods[2]. In addition, a lack of cohesion and coordination of indicators points to important barriers for information sharing, thus delaying the adoption of innovative solutions to urban issues.

            Indicator heterogeneity may also occur across time within urban areas. Annual variation in statistics included in a city’s sustainability reports can hinder accessibility to data and prevent citizens from tracking meaningful change. This presents serious obstacles for transparency and implies a tendency to greenwash unfavourable data by only selecting ‘impressive’ statistics[3].

            Lastly, indicator sets, notably the SDG indicators, struggle to collect necessary data. There is a key trade-off between choosing resource intensive but descriptive indicators, and simple indicators that lack explanatory power[4]. Coupled with systemic non-adherence to the indicator list set out by the SDGs, variable and insufficient data are a chronic barrier in promoting accountability and progress on critical environmental and social targets.

II. A Growing Need for Binding Mechanisms

Despite these drawbacks, there exists a growing need to enforce various targets, most notably for the climate crisis. The increasingly urgent need to achieve net zero emissions by 2050 is limited by the combination of a lack of political will and binding mechanisms to enforce measures taken by states[5]. A key example is Canada’s inability to meet every climate and emissions-related target it has set; this is explained in part by its chronic omission of binding language in its climate legislation. As one of the highest per-capita emitters of greenhouse gases, Canada must both set ambitious emission reduction targets and hold itself accountable in order to meaningfully contribute to the Paris Agreement. The new federal Canadian Net-Zero Emissions Accountability Act bill unfortunately replicates this trend. Although a step in the right direction, it completely fails to incorporate mechanisms that actually hold the government to a legally binding risk of penalty if they fail to meet their targets- nor does it set out any pathways to meet them. [6]

There is a growing trend of ‘smart cities’ and data mining which has created unprecedented levels of knowledge of human activity. However, while indicators are popular across the globe and have aided significant data collection, there is a strong body of critics that argue that merely producing and measuring indicators has acted as a replacement for meaningful change[7]. Data-driven discourse creates an illusion of progress, whereby indicators are used to supplant analyses of inequities rather than as an effective tool to identify and drive solutions for problems[8].

III. Potential Legal Frameworks to Enforce Targets

            Given these difficulties, is it possible to create meaningful and binding legal frameworks to meet a diverse array of targets? Certainly, even the most binding of laws will suffer without the political will and internal accountability of the state. Nevertheless, there are several avenues that merit exploration to create binding legal frameworks to meet the SDGs and other goals.

            One such method for Canada may be a coordinated effort at the provincial level to impose guidelines and indicators on metropolitan areas, particularly those containing large cities. Cities are a useful unit of focus since they capture a majority of the country’s population, and urban areas contribute on average to 40-70% of greenhouse gas emissions[9]. Given that municipalities are creatures of statute, provinces retain significant power over the workings of cities and the degree of cohesion across them[10]. Creating a legal framework of coordination, information sharing, and required indicators has the potential to mitigate several of the barriers to using indicators effectively, particularly since municipalities would be held accountable to a higher level of government. While some leeway ought to be allocated to municipalities to pursue their unique goals in alignment with their differing focus areas and particularities, creating a centralized indicator framework with which to motivate and compare cities could foster a degree of friendly competition to accelerate progress with the added benefit of more robust and uniform data collection.

            A useful example of coordination improving progress on climate-friendly initiatives is that of public transportation. Developing public transport is a key goal to reduce emissions and ensure equity in accessibility and mobility, and it is a popular focus in urban sustainability indicator sets. China has facilitated inter-city rail development by allowing provinces to intervene in the coordination of projects, which alleviates fiscal burdens on individual municipalities, and also facilitates the proliferation of long-distance public transportation which would otherwise be disjointed, inaccessible, and more expensive[11]. A deeper analysis of the complexities that the Canadian context faces in implementing regional planning can be found in Wang’s analysis of transit planning in the greater Toronto area, which, although fraught, enabled a unified transit fare and greater accessibility for inter-city mobility[12]. A greater share of trips, notably those crossing municipal lines, can be captured by public transportation than by cities acting alone. More abstractly, provincially coordinated and enforced indicator sets may produce more robust and productive results than self-motivating and uncommunicative municipalities.

IV. The Inequity of Indicators & Just Sustainability

            Indicators can act as a useful measurement framework within a coordinated municipal-provincial system of accountability to meet national and international targets. However, as discussed above, indicators must be used carefully to avoid illusions of progress. If a just sustainability framework is appropriately employed when creating indicator sets and their associated programs, many of these issues may be avoided.

            Just sustainability, defined as “the need to ensure a better quality of life for all, now, and into the future, in a just and equitable matter, while living within the limits of the supporting ecosystems”[13],  sets its focus squarely on the convergence of environmental justice, social justice, and sustainable development. It asserts that social and economic inequities (notably those caused by racism) intersect inextricably with the environmental crisis. This powerful paradigm encourages a critical and nuanced examination of the multiple facets of any ‘urban issue’. It necessitates deep engagement with structural problems, which in turn requires delicately created indicators to track whether “progress” is in fact for all citizens or merely the privileged elite.

            Creating a framework that is binding, progressive, and just is a daunting but necessary task. Fortunately, the tools for this already exist, and there is no shortage of information for effective implementation. All that is needed is the political will to act. In the face of the climate crisis and systemic inequity, there is no reason not to.

[1] Allison Lalla, Are Sustainability Indicators Just? A Framework and Critical Review of Current Urban Sustainability Metrics (BA&Sc Environment Honours Thesis, McGill University, 2020) [unpublished].

[2] JM Klopp & DL Petretta, “The urban sustainable development goal: Indicators, complexity and the politics of measuring cities” (2017) 63 Cities, 92.

[3] For an example, see the variation of statistics presented in Chicago’s Year One and Year Two Sustainable Chicago Progress Reports at “Sustainable Chicago 2015”, online: City of Chicago <www.chicago.gov/city/en/progs/env/sustainable_chicago2015.html>.

[4] Supra note 2.

[5] V Masson-Delmotte et al, Summary for policymakers: Global Warming of 1.5°C. An IPCC Special Report on the impacts of global warming of 1.5°C above pre-industrial levels and related global greenhouse gas emission pathways, in the context of strengthening the global response to the threat of climate change, sustainable development, and efforts to eradicate poverty, IPCC, 2018, online: <www.ipcc.ch/sr15/chapter/spm/>.

[6] “Government of Canada charts course for clean growth by introducing bill to legislate net-zero emissions by 2050” (19 November 2020), online: Environment and Climate Change Canada <www.canada.ca/en/environment-climate-change/news/2020/11/government-of-canada-charts-course-for-clean-growth-by-introducing-bill-to-legislate-net-zero-emissions-by-2050.html>; John Paul Tasker, “Trudeau unveils new net-zero emissions plan to meet climate change targets” (19 November 2020), Online: CBC News <www.cbc.ca/news/politics/net-zero-emissions-1.5807877>.

[7] R Gahin et al, “Do indicators help create sustainable communities?” (2003) 8:6 661.

[8] M Kaika, “‘Don’t call me resilient again!’: the New Urban Agenda as immunology … or … what happens when communities refuse to be vaccinated with ‘smart cities’ and indicators” (2017) 29:1 89.

[9] UN Habitat, Cities and Climate Change: Global Report on Human Settlements (London: Earthscan, 2011).

[10] CR Tindal et al, “The Promise of Local Government”, in Local Government in Canada, 8th ed, (Toronto: Nelson Education, 2013).

[11] Guicai Li et al, “Value capture beyond municipalities: transit-oriented development and inter-city passenger rail investment in China’s Pearl River Delta” (2013) 33 J of Transport Geography 268.

[12] Jason Edwin Wang, “The Missing Link of Metrolinx: Examining the Regional Governance of Transit Planning in the Greater Toronto and Hamilton Area” (2014), online: uO Research <ruor.uottawa.ca/handle/10393/31058>.

[13] RD Bullard et al, Just Sustainabilities: Development in an Unequal World (London: Routledge, 2002).

*Edited by Émilie Weidl.