This blogpost was jointly written by Danny Lämmerhirt and Mária Žuffová (University of Strathclyde).
We are pleased to announce our latest report Governing by rankings – How the Global Open Data Index helps advance the open data agenda. The Global Open Data Index (GODI) is one of the largest worldwide assessments of how well governments publish open data, coordinated by Open Knowledge International since 2013. Over the years we observed how GODI is used to monitor open data publication. But to date, less was known how GODI may translate into open data policies and publication. How does GODI mobilise support for open data? Which actors are mobilised? Which aspects of GODI are useful, and which are not? Our latest report provides insights to these questions.
Why does this research matter?
Global governance indices like GODI enjoy great popularity due to their capacity to count, calculate, and compare what is otherwise hardly comparable. A wealth of research – from science and technology studies to sociology of quantification and international policy – shows that the effects of governance indicators are complex (our report provides an extensive reading list). Different audiences can take up indices to different (unintended) ends. It is therefore paramount to trace the effects of governance indicators to inform their future design.
The report argues that there are multiple ways of looking for ‘impacts’ depending on different audiences, and how they put GODI into practice. Does a comparative open data ranking like GODI help mobilise high-level policy commitments? Does it incentivise individual government agencies to adjust and improve the publication of open data? Does it open up spaces for discussion and deliberation between government and civil society? This thinking builds on an earlier report by Open Knowledge International arguing that indicators have different audiences, with different lived experiences, needs, and agendas. While any form of measurement needs to align with these needs to become actionable (which affects how the impact of indicators will take shape), it also needs to retain comparability.
Our findings
We used Argentina, United Kingdom and Ukraine as case studies to represent different degrees of open data publication, economic development and political set-up. Our report, drawing from a series of twelve interviews and document analysis, suggests that GODI drives change primarily from within government. We assume this finding is partly due to our limited sample size. While key actors in the government are easy to identify, as open data publication is often one of their job responsibilities, further research is needed to identify more civil society actors and how they engage with GODI.
Below we describe nine ways how GODI influences open data policy and publication.
- Getting international visibility and achieving progress in country rankings or generally high ranking may incentivise and maintain high-level political support for open data, despite non-comparability of results across years.
- In the absence of open data legislation, GODI has been used by Argentinian government as a soft policy tool to pressure other government agencies to publish data.
- Government agencies tasked with implementing open data used GODI to reward and point out progress made by other agencies, but also flag blockages to high-level politicians.
- GODI sets standards what datasets to publish and sets a baseline for improvement. Outcomes are debatable around categories where the central government does not have easy political levers to publish data.
- GODI may be confounded with broader commitments to open government and used as an argument to reduce investment in other aspects of open government agenda. In the past, some high-level politicians presented high ranking in GODI as evidence of government transparency and obsoletion of other ways of providing government information.
- This effect may possibly be exacerbated by superficial media coverage that reports on the ranking without engaging with broader information and transparency policies. An analysis of Google News results suggests that journalists tend to reproduce (mostly politicians’) misconceptions and confound a good ranking in GODI with a high degree of government transparency and openness.
- Our findings suggest that individuals and organisations working around transparency and anti-corruption make little use of GODI due to a lack of detail and a misalignment with their specialised work tasks. For instance, Transparency International Ukraine uses the Transparent Public Procurement Rating to evaluate the legal framework, aside from the publication of open data.
- On the other hand, academics show interest to GODI to develop new governance indicators. They also often use country scores as a proxy for measuring open data availability.
- GODI has a potential for use in data journalism. Data journalism trainers may use it as a source of government data during their trainings.
What we learned and the road ahead
Our research suggests that governments in all analysed countries pay attention to GODI. With a few exceptions, they use it mostly to support open data publication and pave the way for new open data policies. While this is a promising finding, it has important implications for GODI and its design. If GODI sets standards in open data publication, as some interviewees from the government suggest, it needs to make sure to represent different data demands in the assessment and to encourage the implementation of sound policies. The challenge is to support policy development, which is often a lengthy process as opposed to short-lived rank-seeking.
Some interviewees suggested valuable avenues for GODI’s design. For instance, assessing progress in open data publication perpetually rather than once a year over a limited timespan would require a long-term commitment to open data publication and better opportunities for civic engagement, as it would prevent governments from updating datasets once a year before GODI’s deadline only. Another route forward is discussed in another recent research by OKI, highlighting the potential to adjust an open data index to align it more closely to specific needs of topical expert organisations. Beyond engaging via GODI, civil society and academia might also participate in the development of new data monitoring instruments such as the Open Data Survey, that are relevant for their mission.
Danny Lämmerhirt works on the politics of data, sociology of quantification, metrics and policy, data ethnography, collaborative data, data governance, as well as data activism. You can follow his work on Twitter at @danlammerhirt. He was research coordinator at Open Knowledge Foundation.
