Support Us

New Report: “Open Budget Data: Mapping the Landscape”

Jonathan Gray - September 2, 2015 in Featured, Policy, Releases, Research

We’re pleased to announce a new report, “Open Budget Data: Mapping the Landscape” undertaken as a collaboration between Open Knowledge, the Global Initiative for Financial Transparency and the Digital Methods Initiative at the University of Amsterdam.

The report offers an unprecedented empirical mapping and analysis of the emerging issue of open budget data, which has appeared as ideals from the open data movement have begun to gain traction amongst advocates and practitioners of financial transparency.

In the report we chart the definitions, best practices, actors, issues and initiatives associated with the emerging issue of open budget data in different forms of digital media.

In doing so, our objective is to enable practitioners – in particular civil society organisations, intergovernmental organisations, governments, multilaterals and funders – to navigate this developing field and to identify trends, gaps and opportunities for supporting it.

How public money is collected and distributed is one of the most pressing political questions of our time, influencing the health, well-being and prospects of billions of people. Decisions about fiscal policy affect everyone-determining everything from the resourcing of essential public services, to the capacity of public institutions to take action on global challenges such as poverty, inequality or climate change.

Digital technologies have the potential to transform the way that information about public money is organised, circulated and utilised in society, which in turn could shape the character of public debate, democratic engagement, governmental accountability and public participation in decision-making about public funds. Data could play a vital role in tackling the democratic deficit in fiscal policy and in supporting better outcomes for citizens.

The report includes the following recommendations:

  1. CSOs, IGOs, multilaterals and governments should undertake further work to identify, engage with and map the interests of a broader range of civil society actors whose work might benefit from open fiscal data, in order to inform data release priorities and data standards work. Stronger feedback loops should be established between the contexts of data production and its various contexts of usage in civil society – particularly in journalism and in advocacy.

  2. Governments, IGOs and funders should support pilot projects undertaken by CSOs and/or media organisations in order to further explore the role of data in the democratisation of fiscal policy – especially in relation to areas which appear to have been comparatively under-explored in this field, such as tax distribution and tax base erosion, or tracking money through from revenues to results.

  3. Governments should work to make data “citizen readable” as well as “machine readable”, and should take steps to ensure that information about flows of public money and the institutional processes around them are accessible to non-specialist audiences – including through documentation, media, events and guidance materials. This is a critical step towards the greater democratisation and accountability of fiscal policy.

  4. Further research should be undertaken to explore the potential implications and impacts of opening up information about public finance which is currently not routinely disclosed, such as more detailed data about tax revenues – as well as measures needed to protect the personal privacy of individuals.

  5. CSOs, IGOs, multilaterals and governments should work together to promote and adopt consistent definitions of open budget data, open spending data and open fiscal data in order to establish the legal and technical openness of public information about public money as a global norm in financial transparency.

Global Open Data Index 2015 is open for submissions

Mor Rubinstein - August 25, 2015 in Featured, Global Open Data Index, open knowledge

The Global Open Data Index measures and benchmarks the openness of government data around the world, and then presents this information in a way that is easy to understand and easy to use. Each year the open data community and Open Knowledge produces an annual ranking of countries, peer reviewed by our network of local open data experts. Launched in 2012 as tool to track the state of open data around the world. More and more governments were being to set up open data portals and make commitments to release open government data and we wanted to know whether those commitments were really translating into release of actual data.

The Index focuses on 15 key datasets that are essential for transparency and accountability (such as election results and government spending data), and those vital for providing critical services to citizens (such as maps and water quality). Today, we are pleased to announce that we are collecting submissions for the 2015 Index!

The Global Open Data Index tracks whether this data is actually released in a way that is accessible to citizens, media and civil society, and is unique in that it crowdsources its survey results from the global open data community. Crowdsourcing this data provides a tool for communities around the world to learn more about the open data available in their respective countries, and ensures that the results reflect the experience of civil society in finding open information, rather than accepting government claims of openness. Furthermore, the Global Open Data Index is not only a benchmarking tool, it also plays a foundational role in sustaining the open government data community around the world. If, for example, the government of a country does publish a dataset, but this is not clear to the public and it cannot be found through a simple search, then the data can easily be overlooked. Governments and open data practitioners can review the Index results to locate the data, see how accessible the data appears to citizens, and, in the case that improvements are necessary, advocate for making the data truly open.

Screen Shot 2015-08-25 at 13.35.24

 

Methodology and Dataset Updates

After four years of leading this global civil society assessment of the state of open data around the world, we have learned a few things and have updated both the datasets we are evaluating and the methodology of the Index itself to reflect these learnings! One of the major changes has been to run a massive consultation of the open data community to determine the datasets that we should be tracking. As a result of this consultation, we have added five datasets to the 2015 Index. This year, in addition to the ten datasets we evaluated last year, we will also be evaluating the release of water quality data, procurement data, health performance data, weather data and land ownership data. If you are interested in learning more about the consultation and its results, you can read more on our blog!

How can I contribute?

2015 Index contributions open today! We have done our best to make contributing to the Index as easy as possible. Check out the contribution tutorial in English and Spanish, ask questions in the discussion forum, reach out on twitter (#GODI15) or speak to one of our 10 regional community leads! There are countless ways to get help so please do not hesitate to ask! We would love for you to be involved. Follow #GODI15 on Twitter for more updates.

Important Dates

The Index team is hitting the road! We will be talking to people about the Index at the African Open Data Conference in Tanzania next week and will also be running Index sessions at both AbreLATAM and ConDatos in two weeks! Mor and Katelyn will be on the ground so please feel free to reach out!

Contributions will be open from August 25th, 2015 through September 20th, 2015. After the 20th of September we will begin the arduous peer review process! If you are interested in getting involved in the review, please do not hesitate to contact us. Finally, we will be launching the final version of the 2015 Global Open Data Index Ranking at the OGP Summit in Mexico in late October! This will be your opportunity to talk to us about the results and what that means in terms of the national action plans and commitments that governments are making! We are looking forward to a lively discussion!

The 2015 Global Open Data Index is around the corner – these are the new datasets we are adding to it!

Mor Rubinstein - August 20, 2015 in Global Open Data Index

After a two months, 82 ideas for datasets, 386 voters, thirteen civil society organisation consultations and very active discussions on the Index forum, we have finally arrived at a consensus on what datasets will be including in the 2015 Global Open Data Index (GODI).

This year, as part of our objective to ensure that the Global Open Data index is more than a simple measurement tool, we started a discussion with the open data community and our partners in civil society to help us determine which datasets are of high social and democratic value and should be assessed in the 2015 Index. We believe that by making the choice of datasets a collaborative decision, we will be able to raise awareness of and start a conversation around the datasets required for the Index to truly become a civil society audit of the open data revolution. The process included a global survey, a civil society consultation and a forum discussion (read more in a previous blog post about the process).

The community had some wonderful suggestions, making deciding on fifteen datasets no easy task. To narrow down the selection, we started by eliminating the datasets that were not suitable for global analysis. For example, some datasets are collected at the city level and can therefore not be easily compared at a national level. Secondly, we looked to see if there is was a global standard that would allow us to easily compare between countries (such as UN requirements for countries etc). Finally, we tried to find a balance between financial datasets, environmental datasets, geographical datasets and datasets pertaining to the quality of public services. We consulted with experts from different fields and refined our definitions before finally choosing the following datasets:

  1. Government procurement data (past and present tenders) – This dataset is crucial for monitoring government contracts be it to expose corruption or to ensure the efficient use of public funds. Furthermore, when combined with budget and spending data, contracting data helps to provide a full and coherent picture of public finance. We will be looking at both tenders and awards.
  2. Water quality -Water is life and it belongs to all of us. Since this is an important and basic building stone of society, having access to data on drinking water may assist us not only in monitoring safe drinking water but also to help providing it everywhere.
  3. Weather forecast – Weather forecast data is not only one of the most commonly used datasets in mobile and web applications, it is also of fundamental importance for agriculture and disaster relief. Having both weather predictions and historical weather data helps not only to improve quality of life, but to monitor climate change. As such, through the index, we will measure whether governments openly publish data both data on the 5 day forecast and historical figures.
  4. Land ownership – Land ownership data can help citizens understand their urban planning and development as well as assisting in legal disputes over land. In order to assess this category, we are using national cadastres, a map showing land registry.
  5. Health performance data – While this was one of the most popular datasets requested during the consultation, it was challenging to define what would be the best dataset(s) to assess health performance (see the forum discussion). We decided to use this category as an opportunity to test ideas about what to evaluate. After numerous discussions and debates, we decided that this year we would use the following as proxy indicators of health performance:
      Location of public hospitals and clinics.
      Data on infectious diseases rates in a country.
    That being said, we are actively seeking and would greatly appreciate your feedback! Please use the country level comment section to suggest any other datasets that you encounter that might also be a good measure of health performance (for example, from number of beds to budgets). This feedback will help us to learn and define this data category even better for next year’s Index.

2015 Global Open Data Index

 

 

In addition to the new datasets, we refined the definitions to some of the existing datasets, while using our new datasets definition guidelines. These were written in order to both produce a more accurate measurement and to create more clarity about what we are looking for with each dataset. The guidelines suggest at least 3 key data characteristics for each datasets, define how often each dataset needs to be updated in order to be considered timely, and suggests level aggregation acceptable for each datasets. The following datasets were changed in order to meet the guidelines:

Elections results – Data should be reported at the polling station level as to allow civil society to monitor elections results better and uncover false reporting. In addition, we added indicators such as number of registered voters, number of invalid votes and number of spoiled ballots.

National map – In addition to the scale of 1:250,000, we added features such as – markings of national roads, national borders, marking of streams, rivers, lakes, mountains.

Pollutant emissions – We defined the specific pollutants that should be included in the datasets.

National Statistics – GDP, unemployment and populations have been selected as the indicators that must be reported.

Public Transport – We refined the definition so it will examine only national level services (as opposed to inter cities ones). We also do not looking for real time data, but time tables.

Location datasets (previously Postcodes) – Postcode data is incredibly valuable for all kinds of business and civic activity; however, 60 countries in the world do not have a postcode system and as such, this dataset has been problematic in the past. For these countries, we have suggested examining a different dataset, administrative boundaries. While it is not as specific as postcodes, administrative boundaries can help to enrich different datasets and create better geographical analysis.

Adding datasets and changing definitions has been part of ongoing iterations and improvements that we have done to the Index this year. While it has been a challenge, we are hoping that these improvements help to create a more fair and accurate assessment of open data progress globally. Your feedback plays an essential role in shaping and improving the Index going forward, please do share it with us.

For the full descriptions of this year’s datasets can be found here.

Onwards to AbreLatAm 2015: what we learned last year

Mor Rubinstein - August 7, 2015 in Community, open knowledge

This post was co-written by Mor Rubinstein and Neal Bastek. It is cross-posted and available in Spanish at the AbreLatAm blog.

IMG_5986 (1)AbreLatAm, for us “gringos”, is magical. Even in the age where everyone is glued to a screen, face to face connection is still the strongest connection humans can have; it fosters the trust that can lead to new cooperations and innovations. However, in the case of Latin America, it also creates a family. This feeling creates both a sense of solidarity and security that lets people share and consult about their open data and transparency issues with greater passion and awareness of the challenges and conditions we face daily in our own communities. It is unique, and difficult to replicate. You may not realise it, but in our experience, this feeling is not so common in other parts of the world, where the culture of work is more strict and, with all due respect for our differences, less personal. AbreLatAm therefore is a gift to the movement itself and not just to those of us lucky enough to attend.

For open data practitioners from outside of America Latina like us, AbreLatAm is a place to learn how communities evolve and how they work together. It is a place for us to listen, deeply. So, our command of the Spanish language is not so great (pero es mejor que ayer!) but we don’t need Spanish to feel the atmosphere, see the sparks and contribute, in English, with hand gestures to amplify the event. We try hard to understand the context and the words (and are grateful for the support we have from patient translators!) and are understand the unique problems in the region. For example, the high levels of corruption, the low levels of trust in government and highest rates of inequality in the world. However, other problems are universal, and we should all examine how to solve them together. The question is how?

The Open Knowledge Network has gained tremendous inspiration from AbreLatAm. What appeared early on as a good opportunity to promote the Global Open Data Index and build connections with the Latin American community has become so much more — a fertile ground for sharing and feedback. Some of the processes that we are doing now in this year’s Index, such as our methodology consultation and datasets selections, were the direct result of our participation in AbreLatAm last year.

Mor and Neal at AbreLATAM

Neal and Mor promoting the Index in last’s year AbreLatam

We are very excited to see what we will learn this year. As AbreLatAm matures, it also receives more attention and attracts more participants. AbreLatAm was, and still is, a pioneering community participatory event. The challenges now are about scaling, and it is a mirror to similar challenges around the globe. How can we harness the energy of an un-conference with such a vast amount of participants? How can we go from talking and sharing to coordinated global action?

The movement’s ability to scale will only be a success if it’s rooted in community-based, citizen driven needs and not handed down from on high by way of intellectual and academic arguments rooted in a Eurocentric experience. AbreLatAm is an ideal setting for discovering this demand in the Latin American context and matching it and adapting it to global practices and experiences that have succeeded elsewhere– be it in the North or South! Likewise, the LATAM community has much to share in terms of their own experiences and success, and at Open Knowledge we’re keenly interested in bringing those back to our global network for reflection and consideration.

Beauty behind the scenes

Tryggvi Björgvinsson - August 5, 2015 in CKAN, OKF Sweden, Open Data, open knowledge

Good things can often go unnoticed, especially if they’re not immediately visible. Last month the government of Sweden, through Vinnova, released a revamped version of their open data portal, Öppnadata.se. The portal still runs on CKAN, the open data management system. It even has the same visual feeling but the principles behind the portal are completely different. The main idea behind the new version of Öppnadata.se is automation. Open Knowledge teamed up with the Swedish company Metasolutions to build and deliver an automated open data portal.

Responsive design

In modern web development, one aspect of website automation called responsive design has become very popular. With this technique the website automatically adjusts the presentation depending on the screen size. That is, it knows how best to present the content given different screen sizes. Öppnadata.se got a slight facelift in terms of tweaks to its appearance, but the big news on that front is that it now has a responsive design. The portal looks different if you access it on mobile phones or if you visit it on desktops, but the content is still the same.

These changes were contributed to CKAN. They are now a part of the CKAN core web application as of version 2.3. This means everyone can now have responsive data portals as long as they use a recent version of CKAN.

New Öppnadata.se

New Öppnadata.se

Old Öppnadata.se

Old Öppnadata.se

Data catalogs

Perhaps the biggest innovation of Öppnadata.se is how the automation process works for adding new datasets to the catalog. Normally with CKAN, data publishers log in and create or update their datasets on the CKAN site. CKAN has for a long time also supported something called harvesting, where an instance of CKAN goes out and fetches new datasets and makes them available. That’s a form of automation, but it’s dependent on specific software being used or special harvesters for each source. So harvesting from one CKAN instance to another is simple. Harvesting from a specific geospatial data source is simple. Automatically harvesting from something you don’t know and doesn’t exist yet is hard.

That’s the reality which Öppnadata.se faces. Only a minority of public organisations and municipalities in Sweden publish open data at the moment. So a decision hasn’t been made by a majority of the public entities for what software or solution will be used to publish open data.

To tackle this problem, Öppnadata.se relies on an open standard from the World Wide Web Consortium called DCAT (Data Catalog Vocabulary). The open standard describes how to publish a list of datasets and it allows Swedish public bodies to pick whatever solution they like to publish datasets, as long as one of its outputs conforms with DCAT.

Öppnadata.se actually uses a DCAT application profile which was specially created for Sweden by Metasolutions and defines in more detail what to expect, for example that Öppnadata.se expects to find dataset classifications according the Eurovoc classification system.

Thanks to this effort significant improvements have been made to CKAN’s support for RDF and DCAT. They include application profiles (like the Swedish one) for harvesting and exposing DCAT metadata in different formats. So a CKAN instance can now automatically harvest datasets from a range of DCAT sources, which is exactly what Öppnadata.se does. For Öppnadata.se, the CKAN support also makes it easy for Swedish public bodies who use CKAN to automatically expose their datasets correctly so that they can be automatically harvested by Öppnadata.se. For more information have a look at the CKAN DCAT extension documentation.

Dead or alive

The Web is decentralised and always changing. A link to a webpage that worked yesterday might not work today because the page was moved. When automatically adding external links, for example, links to resources for a dataset, you run into the risk of adding links to resources that no longer exist.

To counter that Öppnadata.se uses a CKAN extension called Dead or alive. It may not be the best name, but that’s what it does. It checks if a link is dead or alive. The checking itself is performed by an external service called deadoralive. The extension just serves a set of links that the external service decides to check to see if some links are alive. In this way dead links are automatically marked as broken and system administrators of Öppnadata.se can find problematic public bodies and notify them that they need to update their DCAT catalog (this is not automatic because nobody likes spam).

These are only the automation highlights of the new Öppnadata.se. Other changes were made that have little to do with automation but are still not immediately visible, so a lot of Öppnadata.se’s beauty happens behind the scenes. That’s also the case for other open data portals. You might just visit your open data portal to get some open data, but you might not realise the amount of effort and coordination it takes to get that data to you.

Image of Swedish flag by Allie_Caulfield on Flickr (cc-by)

This post has been republished from the CKAN blog.

Launch of timber tracking dashboard for Global Witness

Sam Leon - July 31, 2015 in Data Journalism

Open Knowledge has produced an interactive trade dashboard for anti-corruption NGO Global Witness to supplement their exposé on EU and US companies importing illegal timber from the Democratic Republic of Congo (DRC).

timber_tracker.width-800

The DRC Timber Timber Trade Tracker consumes open data from DataHub.io to visualise where in the world Congolese timber is going. The dashboard makes it easy to identify countries that are importing large volumes of potentially illegal timber, and to see where timber shipped by companies accused of systematic illegal logging and social and environmental abuses is going on.

Global Witness has long campaigned for greater oversight of the logging industry in DRC which is home to two thirds of the world’s second largest rainforest. The logging industry is mired with corruption with two of the DRC’s biggest loggers allegedly complicit in the beating and raping of local populations. Alexandra Pardal, campaign leader at Global Witness said:

We knew that DRC logging companies were breaking the law, but the extent of illegality is truly shocking. The EU and US are failing in their legal obligations to keep timber linked to illegal logging, violence and intimidation off our shop floors. Traders are cashing in on a multi-million dollar business that is pushing the world’s vanishing rainforests to extinction.

The dashboard is part of a long term collaboration between Open Knowledge and Global Witness through which they have jointly created a series of interactives and data-driven investigations around corruption and conflict in the extractives industries.

To read the full report and see the dashboard go here.

If you work for an organisation that wants to make its data come alive on the web, get in touch with our team through services.okfn.org.

Introducing ContentMine

Marieke Guy - July 21, 2015 in Featured Project, Open Access, Open Data

If you are interested in Open Access and Open Data and haven’t hear about ContentMine yet then you are missing out! Graham Steel, ContentMine Community Manager, has written a post for us introducing this exciting new tool.

contentmine2ContentMine aims to liberate 100,000,000 facts from the scientific literature.

We believe that “The Right to Read is the Right to Mine“: anyone who has lawful access to read the literature with their eyes should be able to do so with a machine.

We want to make this right a reality and enable everyone to perform research using humanity’s accumulated scientific knowledge. The extracted facts are CC0.

The Content Mine Team at the Panton Arms in Cambridge

The ContentMine Team & Helen Turvey, Executive Director, Shuttleworth Foundation at the Panton Arms in Cambridge

Research which relies on aggregating large amounts of dynamic information to benefit society is particularly key to our work – we want to see the right information getting to the right people at the right time and work with professionals such as clinical trials specialists and conservationists. ContentMine tools, resources, services and content are fully Open and can be re-used by anybody for any legal purpose.

ContentMine is inspired by the community successes of Wikimedia, Open StreetMap, Open Knowledge, and others and encourages the growth of subcommunities which design, implement and pursue their particular aims. We are funded by the Shuttleworth Foundation, a philanthropic organisation who are unafraid to re-imagine the world and fund people who’ll change it.

Content Mine welcome session

ContentMine Wellcome Trust Workshop

There are several ways to get involved with ContentMine. You can find us on GitHub, Google Groups, email, Twitter and most recently, we have a variety of open communities set up here on Discourse.

This posh has been reposted from the Open Access Working Group blog.

Civic Tech and NGOs … wait, and donors – Can we be better collaborators in global Transparency and Accountability work?

kerstiru - July 10, 2015 in Community, open knowledge

This is a guest post by Kersti Ruth Wissenbach, our Open Knowledge Ambassador in the Netherlands.

At our Re:publica session in May we set out to bring together transparency and accountability practitioners from traditional NGOs as well as from the civic tech scene. We came to recognise that, to a large extent, we keep working in silos rather than merging together in new ways so as to collaborate and benefit from each others expertise and experiences. Thus, we wanted to identify the reasons and, together, explore how to enable closer collaboration.

Over the last few years, new global networks of civic activists have emerged and are rapidly spreading across the globe, building and sharing technologies, serving their advocacy for a more open and just society. We are witnessing a vast variety of collaboration, from individual hacker groups towards more structured networks working around different topics and eventually being connected to a more organisational body, such as Open Knowledge. Also, traditional NGOs increasingly embraced the potential opportunities emerging from the fast pervasion of ICTs, mirrored in the ICT4D, mobile and recent big data ‘hype’. Both groups clearly work in overlapping spaces, however, they are to a big extent disconnected from each other. Partly this seems to be due to the perception of the spaces; Civic tech NGOs are often relatively new and young, and want to move and work fast on “innovations” but the perception is that more traditional NGOs are slower moving and hampered by bureaucracy. This siloing of work from NGOs and civic tech activists becomes most apparent in transparency and accountability work. Therefore, Tech4TA, on a global scale, should very much be about active collaboration between traditional NGOs and civic tech groups, in order to -

  • Use existing and build upon tools (stop building from scratch but use what civic techies are building and sharing)
  •  Prevent exclusion and structural power imbalances in policies or deriving project activities (learn from NGOs experiences to actively work towards inclusive, context relevant project design and implementation, especially in areas where accessibility and availability of ICTs remains challenging and is oftentimes not a feasible mechanism at all)
  • Safeguard TA (Transparency and Accountability) practitioners (learning from digital activists in how to ensure security, privacy and enable responsible data handling)
  • Make the most use of the expertise in delivering locally appropriate projects in challenging situations
  • Understand the strengths and weaknesses of both types of organisation, so that they can work to complement each other

We gathered an unexpectedly big group of techies, NGOers and donors, which brought many different perspectives to the table. In two cross-disciplinary groups we gathered a profound first set of barriers for stronger collaboration and had some vital discussions on what to do about them.

 

14733577751_c74cc5958b_z

Collaborating at OKFest 14

 

What we need and how to get there

Active outreach It does not seem to be common practise within all NGOs to identify and reach out to local digital activists or civic tech groups on the ground when beginning to prepare new projects in specific countries. Those actors are the experts in the local technology context! Therefore, they are the best people to collaborate with in order to explore chances and boundaries when it comes to using tech or data in the respective contexts. Also do they know what is in place already and potentially just need some support in order to take the current projects to the next phase or scale them up. – The question is how can this become a fundamental practice for NGOs? What mechanisms need to be integrated into normal practice, that don’t require too much for both civic tech NGOs or traditional NGOs?

Speaking the same language Data activists and traditional NGO practitioners oftentimes do not speak the same ‘language’, creating a huge barrier for communicating and smooth collaboration. It can be difficult for the non-technical among us to understand why something that sounds so simple, could be hard to accomplish or take a long time, when conversely sometimes things that sound extremely challenging can be solved in a few simple steps. Finding (still rare) hybrids which are home on both sides or alternatively work with mediators who can bridge between both sites will be of great value to overcome this boundary. People that can bridge the technical and non-technical divide to explain how a project can work and where the pain points might be would smooth relations and enable a greater ease of understanding. – The question is what this could realistically look like. Who are those hybrids or mediators and how can they be strategically integrated?

Consider if Civic Tech is user friendly enough In line with a common lingua, it has been emphasized that often the willingness to use and reuse civic tech tools is there but that tools, code usage etc. are far too abstract for less techie people or the technologists in question normally use other languages as a matter of course. – The question here is how can we make civic tech apps more applicable for less tech-affine TA practitioners and more user friendly on the ground or what else would be needed to overcome such obstacles?

Moving at the same pace Civic tech groups and NGOs have a different pace of connecting and moving things. Given the different pace of groups, where different teams of digital activists and NGOs collaborate, it will help to depart with a clear shared vision. Clarifying if all parties are sharing the same cause from the start will help to prevent frustration further down the road. Making sure there’s a product or project owner that understands both the technical and non-technical sides of the project, and that can coordinate the competing priorities of both sectors, is a must for a smooth project. Otherwise misunderstandings can occur (again based on misunderstanding of the difficulties of each aspect of the work.)

More agile project design opportunities Achieving true transparency and accountability is a very long-term process, not a field for fast wins and not suitable to be pressed into 1 year NGO or donor project frameworks. The result focus of NGOs, often rooted in donor frameworks, are seen as a big obstacles in those regards. Opportunities for more agile project design would be a crucial precondition. NGOs and donors need to move from result focus to a more agile process focus in project design. Moving to this sort of project design would also open up frameworks for a more user-driven project design process which would benefit the donors and the users by creating projects that answer a real need and have a true impact. All participants agreed that educating the donors is needed in order to prevent the continuation of rigid and usually very tech/tool driven proposal frameworks. – Educating the donors, how do we best do that?

Rethink innovation Focus on problems, not solutions (often grant etc. framings already solution driven) Plenty of grants and project designs are already entailing the solution. Driven by e.g. what online platform will do the change rather than focusing on the actual problem or challenge at stake in all its facets necessary to be considered. Such prevailing tech and solution centrism is strongly interlinked with our oftentimes diverging and inflatious definitions of innovation. A common understanding is needed that innovation is not the latest tech but a combination of solutions that truly features into each very context and is self-sustainable (can be the radio or village gathering if that is the natural communication sphere is). Such understanding furthermore needs to derive in clear action. Allowing NGOs some pre-project funding to do their own needs assessments in order to inform project design and create user focused projects involving technology as part of the solution seems like it would have a greater impact. The traditional NGOs understand a needs assessment process and the Civic Tech NGOs understand user design processes. These could be combined to create a strong pre-project design framework, provided that donors could accept and understand that there would need to be a small amount of up-front funding with no specific targets (beyond a full project proposal) granted to organisations they were working with.

In conclusion, an open dialogue and clear commitments are needed between digital activists, NGOs and donors, seriously identifying steps to tackle the above outlined issues.

Some thoughts on the global Open Knowledge Community

As ambassador of Open Knowledge, and as a doctoral researcher exploring Open Knowledge as a case study in my research on transnational activism and community building, a few thoughts came to mind reflecting on those above mentioned issues. Our network has grown fast and widely over the last couple of years and it was great to see the community getting bigger. At the same time, working as a consultant in Tech4TA in various countries and often with less connected groups, I see how we still have the great opportunity to work on increasing the diversity of our community and to stronger invest in fostering inclusion and recognizing that it is not in everyone’s culture to naturally connect to dispersed, translocal or transnational networks. For a while it gave the impression that we easily label contributions from the ‘developing’ world under the development working group tag. Given contemporary dynamics of network expansion and opportunities, we may want to reconsider if we wish to maintain such thematic area at all. If we strive for open government, open science, open culture, you name it, requires context-sensitivity when it comes to potential tactics and tools to reach our goals in different countries and regions, no matter where those are located. So two things to consider within the OK community when it comes to global collaboration are Taking a stand in how OK addresses inclusion in its work and in expanding its community Gathering lessons learned and exploring an OK approach to context-relevant work and according tinkering of tools and tactics

New Discussion Paper: “Democratising the Data Revolution”

Jonathan Gray - July 9, 2015 in Campaigning, Data Journalism, Open Data, Open Government Data, open knowledge, Policy, Research

Democratising the Data Revolution

“New technologies are leading to an exponential increase in the volume and types of data available, creating unprecedented possibilities for informing and transforming society and protecting the environment. Governments, companies, researchers and citizen groups are in a ferment of experimentation, innovation and adaptation to the new world of data, a world in which data are bigger, faster and more detailed than ever before. This is the data revolution.” – UN Data Revolution Group, 2014

What will the “data revolution” do? What will it be about? What will it count? What kinds of risks and harms might it bring? Whom and what will it serve? And who will get to decide?

Today we are launching a new discussion paper on “Democratising the Data Revolution”, which is intended to advance thinking and action around civil society engagement with the data revolution. It looks beyond the disclosure of existing information, towards more ambitious and substantive forms of democratic engagement with data infrastructures.1

It concludes with a series of questions about what practical steps institutions and civil society organisations might take to change what is measured and how, and how these measurements are put to work.

You can download the full PDF report here, or continue to read on in this blog post.

What Counts?

How might civil society actors shape the data revolution? In particular, how might they go beyond the question of what data is disclosed towards looking at what is measured in the first place? To kickstart discussion around this topic, we will look at three kinds of intervention: changing existing forms of measurement, advocating new forms of measurement and undertaking new forms of measurement.

Changing Existing Forms of Measurement

Rather than just focusing on the transparency, disclosure and openness of public information, civil society groups can argue for changing what is measured with existing data infrastructures. One example of this is recent campaigning around company ownership in the UK. Advocacy groups wanted to unpick networks of corporate ownership and control in order to support their campaigning and investigations around tax avoidance, tax evasion and illicit financial flows.

While the UK company register recorded information about “nominal ownership”, it did not include information about so-called “beneficial ownership”, or who ultimately benefits from the ownership and control of companies. Campaigners undertook an extensive programme of activities to advocate for changes and extensions to existing data infrastructures – including via legislation, software systems, and administrative protocols.2

Advocating New Forms of Measurement

As well as changing or recalibrating existing forms of measurement, campaigners and civil society organisations can make the case for the measurement of things which were not previously measured. For example, over the past several decades social and political campaigning has resulted in new indicators about many different issues – such as gender inequality, health, work, disability, pollution or education.3 In such cases activists aimed to establish a given indicator as important and relevant for public institutions, decision makers, and broader publics – in order to, for example, inform policy development or resource allocation.

Undertaking New Forms of Measurement

Historically, many civil society organisations and advocacy groups have collected their own data to make the case for action on issues that they work on – from human rights abuses to endangered species.

Recently there have been several data journalism projects which highlight gaps in what is officially counted. The Migrant Files is an open database containing information about over 29,000 people who died on their way to Europe since 2000, collated from publicly available sources. It was created by a network of journalists (coordinated by J++) who were concerned that this data was not being systematically collected by European institutions. In a similar vein The Counted project from The Guardian records information about deaths in police custody in the US, explicitly in response to the lack of official data collection on this topic.

The Migrant Files

The Role of the Open Data Movement

The nascent open data movement has often focused on the release of pre-existing information about things which are already routinely measured by public institutions. Advocates have pushed for the release of datasets under open licenses in machine-readable formats to facilitate widespread re-use – whether to develop new applications and services, or to facilitate new forms of journalism and advocacy. Datasets are often published via data portals, of which there are now hundreds around the world at local, regional, national and supranational levels.

As well as opening up new datasets, some public institutions have implemented mechanisms to gather input and feedback on open data release priorities, such as:

  • Advisory panels and user groups – e.g. as the UK’s Open Data User Group (ODUG);
  • Dedicated staff – e.g. community management or “Chief Data Officer” positions;
  • User engagement channels – e.g. social media accounts, forums and mailing lists;
  • Data request mechanisms – e.g. Data.gov.uk’s dataset request service or the EU Open Data Portal’s “Suggest a Dataset” form;
  • Consultation processes – e.g. Open Government Partnership National Action Plans;
  • Solicitation for input around data standards – e.g. the US’s Federal Spending Transparency issue tracker on GitHub.

In principle these kinds of mechanisms could be used not just to inform priorities for the release of existing datasets – but also in order to facilitate engagement between institutions and civil society actors around what should be measured by the public sector and how.

To use a metaphor, if data can be compared to photography, then might the open data movement play a role in intervening not just around access and circulation of snapshots taken by public institutions, but also around what is depicted and how it is shot?

Questions for Discussion

We would like to catalyse discussion and gather input about how to increase civil society engagement around the data revolution and questions about what should be measured and how. To this end, we invite advocacy groups, journalists, public institutions, data users, researchers and others to respond to the following questions.

What Can Civil Society Groups Do?
  • What can civil society organisations do to engage with the data revolution?
  • What role might the nascent open data movement play in mediating between civil society organisations and public institutions around what should be measured?
  • What opportunities does the data revolution present for civil society organisations?
  • What are the best examples of democratic interventions to change, advocate or create new forms of measurement (both present and past)?
  • What are the biggest obstacles to greater civil society engagement with the data revolution? How might these be addressed?
  • Which kinds of transnational challenges and issues (e.g. climate change, tax base erosion) are currently inadequately dealt with by national data infrastructures?
  • What areas might new kinds of measurement make the biggest difference, and how?
  • What factors are most important in ensuring that data leads to action?
  • What might civil society groups do to flag potential risks and unwanted consequences of data infrastructures as well as their benefits?
What Can Public Institutions Do?
  • What can public institutions do to better understand the interests and priorities of civil society organisations around what should be measured?
  • Are there examples of where open data initiatives have facilitated significant changes to existing datasets, or the creation of new kinds of datasets?
  • Which kinds of mechanisms might be most effective in understanding and responding to the interests of civil society organisations around what is measured and how?
  • What are the biggest obstacles to public institutions responding more effectively to the data needs and interests of civil society groups? How might these be addressed?

How to Respond

We welcome responses on these and other topics via the channels below:


  1. In this context we understand data infrastructures as composites of technical, legal and social systems (e.g. software, laws, policies, practices, standards) involved in the creation and management of data. 

  2. See: Gray, J. & Davies, T (2015) “Fighting Phantom Firms in the UK: From Opening Up Datasets to Reshaping Data Infrastructures?”. Working paper available at: http://ssrn.com/abstract=2610937 

  3. See: Bruno, I., Didier, E., and Vitale, T. (eds) (2014) Statistics and Activism. Special issue of Partecipazione e conflitto. The Open Journal of Sociopolitical Studies. Available at: http://siba-ese.unisalento.it/index.php/paco/issue/view/1248 

Just Released: “Where Does Europe’s Money Go? A Guide to EU Budget Data Sources”

Jonathan Gray - July 2, 2015 in Data Journalism, Featured, open knowledge, Open Spending, Policy, Research, Where Does My Money Go

The EU has committed to spending €959,988 billion between 2014 and 2020. This money is disbursed through over 80 funds and programmes that are managed by over 100 different authorities. Where does this money come from? How is it allocated? And how is it spent?

Today we are delighted to announce the release of “Where Does Europe’s Money Go? A Guide to EU Budget Data Sources”, which aims to help civil society groups, journalists and others to navigate the vast landscape of documents and datasets in order to “follow the money” in the EU. The guide also suggests steps that institutions should take in order to enable greater democratic oversight of EU public finances. It was undertaken by Open Knowledge with support from the Adessium Foundation.

Where Does Europe's Money Go?

As we have seen from projects like Farm Subsidy and journalistic collaborations around the EU Structural Funds it can be very difficult and time-consuming to put together all of the different pieces needed to understand flows of EU money.

Groups of journalists on these projects have spent many months requesting, scraping, cleaning and assembling data to get an overview of just a handful of the many different funds and programmes through which EU money is spent. The analysis of this data has led to many dozens of news stories, and in some cases even criminal investigations.

Better data, documentation, advocacy and journalism around EU public money is vital to addressing the “democratic deficit” in EU fiscal policy. To this end, we make the following recommendations to EU institutions and civil society organisations:

  1. Establish a single central point of reference for data and documents about EU revenue, budgeting and expenditure and ensure all the information is up to date at this domain (e.g. at a website such as ec.europa.eu/budget). At the same time, ensure all EU budget data are available from the EU open data portal as open data.
  2. Create an open dataset with key details about each EU fund, including name of the fund, heading, policy, type of management, implementing authorities, link to information on beneficiaries, link to legal basis in Eur-Lex and link to regulation in Eur-Lex.
  3. Extend the Financial Transparency System to all EU funds by integrating or federating detailed data expenditures from Members States, non-EU Members and international organisations. Data on beneficiaries should include, when relevant, a unique European identifier of company, and when the project is co-financed, the exact amount of EU funding received and the total amount of the project.
  4. Clarify and harmonise the legal framework regarding transparency rules for the beneficiaries of EU funds.
  5. Support and strengthen funding for civil society groups and journalists working on EU public finances.
  6. Conduct a more detailed assessment of beneficiary data availability for all EU funds and for all implementing authorities – e.g., through a dedicated “open data audit”.
  7. Build a stronger central base of evidence about the uses and users of EU fiscal data – including data projects, investigative journalism projects and data users in the media and civil society.

Our intention is that the material in this report will become a living resource that we can continue to expand and update. If you have any comments or suggestions, we’d love to hear from you.

If you are interested in learning more about Open Knowledge’s other initiatives around open data and financial transparency you can explore the Where Does My Money Go? project, the OpenSpending project, read our other previous guides and reports or join the Follow the Money network.

Where Does Europe’s Money Go - A Guide to EU Budget Data Sources

Get Updates