Support Us

You are browsing the archive for WG Development.

Organisational Identifiers Event at OGD Camp 2011

Theodora Middleton - October 4, 2011 in Events, External, OGDCamp, WG Development, Workshop

Open Government Data Camp 2011 is approaching fast! We’re really excited about all the brilliant talks, workshops, plots, plans and people that are going to be there. In the run-up to the camp we’re going to run a series of posts from a range of voices, talking about different aspects of open government data and the camp. This first post is from Tim Davies, who will be part of the satellite workshop on “organisational identifiers” and data on organisations, jointly organised by Open Corporates and AidInfo.

Many open data projects would benefit from having a shared and stable way to identify organisations within datasets. There are a number of ongoing efforts to collect open data on organisations, or to assign unique indentifiers to organisations. Establishing a universal scheme of organisational identifiers is particularly important for projects such as the International Aid Transparency Initiative (IATI), which need to identify organisations across many jurisdictions and across sectors (companies, charities, governments, other associations and organisations), covering countries with diverse approaches to, and quality of, official registration schemes. This workshop seeks to bring together different initiatives working on organisational identifiers to share knowledge and develop shared approaches to creating complementary and compatible access to organisational information.

The core aims of the workshop are to:

  • Bring together different actors exploring the organisational identifier question: including those focused on companies, charities, government agencies and other forms of organisation.

  • Identify a shared method (standard) for re-using existing identifiers (e.g. Company Registration; VAT Registration). This may focus on convergence and compatibility between schemes such as the IATI Identifier (a framework for Identifiers); Open Corporates URIs (seeking to cover all companies globally); ORGPedia entries (“facilitating the “mashing up” of disparate data sets about the ownership, structure, performance and regulatory compliance of organizations” [REF]); Guidestar Identifier (NGO focussed) and other open data sources and initiatives.

  • Identify existing efforts to collect open data on organisations and to map together different identifiers (e.g. Corporate groupings; mapping Charity and Company numbers etc.)

  • Identify gaps in current approaches (e.g. how to handle countries or organisation-types with limited registration options available to them; how to check the quality of data etc).

  • Develop a shared agenda for action in which partners can each take forward complementary efforts to define and use shared organisational identifiers, and to build the eco-system of open data available for as many of those identifiers as possible.

Registration for the workshop is open at and participation is free.

Participants may also wish to register for the main OGD Camp. Register here!

Geo-enabling Aid Data: What is, and what’s next

Theodora Middleton - August 31, 2011 in External, WG Development, WG Open Geospatial Data

The following guest post is by Josh Powell from Development Gateway, who works on their AidData programme.

Last week, the geocoded locations of all African Development Bank (AfDB) projects continent-wide approved from 2009-2010 were made available at The data include more than $10 billion in AfDB Group funding to 43 African countries, and were geocoded through a partnership between the AfDB and AidData. You can also explore the data for three of these countries— Cameroon, Morocco, and Tanzania—through an interactive map called Development Loop. Meanwhile, last autumn, the World Bank unveiled its website, which shows project locations around the world. Using these data sets, stakeholders can, for the first time, view the precise locations of schools, hospitals, roads, bridges, and other activities financed by these donors.

afdb projects screenshot

The underlying motivation behind these two data releases is the increasing citizen demand for donor transparency and open data. As IATI signatories, both the AfDB and World Bank (through its Open Data Initiative and Mapping for Results program) are finding new and innovative ways to inform the public about their activities. If other donors followed suit, the potential is huge — comprehensive, readily available geo- data for all donors could not only improve service delivery and decrease corruption (as researchers found when local public funds were announced in Uganda), but could improve donor division of labor and aid targeting to the neediest areas within a country.

An ongoing project, led by the Malawi Ministry of Finance (MoF), is attempting to demonstrate both the possibility and benefits of an open geo-data set for all of the 29 aid donors in its Aid Management Platform. In partnership with AidData and University of Texas researchers (from UT’s Climate Change and African Political Stability program), the Malawi MoF has been collecting data from its donors for geocoding. Coding activities are still ongoing, but the data set, which will include nearly 800 aid-funded projects worth nearly US$7 billion, will be made publicly available in advance of the High Level Forum in Busan, South Korea.

By providing open access to this type of comprehensive, granular data set, the Malawi MoF and its donor partners are providing researchers, beneficiaries, and civil society with the ability to examine donor activities and coordination, verify that promised aid is being delivered, and improve the impact of aid through increased efficiency. Moreover, by demonstrating the feasibility of multi-donor geocoding using a harmonized methodology, the Malawi project can increase the demand for open geo-data for all donors and governments committed to transparency and accountability.

Open Aid Data Conference and Hackday, Berlin, 28th-29th September

Theodora Middleton - August 23, 2011 in Events, OK Germany, Sprint / Hackday, WG Development

The following post is by Christian Kreutz, co-founder and board member of the Open Knowledge Foundation Germany.

Help us find innovative solutions for aid transparency and make development aid more effective.

Germany is one of the largest donors in development aid worldwide. Every year over 6 billion euros are spent by the Federal Ministry for Economic Cooperation and Development alone, to provide humanitarian relief and tackle poverty around the world. The Open Aid Data conference will bring together practitioners from various organizations for discussion and exchange about new solutions, and about how technology, the Internet, and particular open data can make aid more transparent – because not all of the money is spent effectively.

When? 29th September
Where? Heinrich-Böll-Stiftung, 10117 Berlin, Schumannstr.8 (See map)
Programme: available here
Hashtag: #OADC
Register: Here

Prior to the conference, we will organize a Hackday on the 28th of September at the Böll-Foundation in Berlin to bring developers together to experiment on technical and data solutions to improve development aid. We are looking for programmers, designers, coders and others who want to learn more about the field of development aid and would like to share their wisdom. In the morning we will introduce you to the theme, and then brainstorm on possible approaches to making aid more transparent. During the rest of the day we want to work through a code sprint on a real solution. Be part of the event!

There are a range of activities around open aid data worldwide, such as the recent conference in Amsterdam or a Barcamp in Kathmandu for aid transparency. By the way, an interesting fact: while the Kenyan government has already offered an open data portal, the German government is still debating such a platform.

Open data and new bottom-up solutions for development aid are rather new fields, but with some promising developments. Around data there is an initiative called IATI (International Aid Transparency Initiative), which propagates a common standard for data sets for financial and other project-related data. So far in development cooperation only a tiny fraction of financial data is openly available, which is, at the end of the day, the taxpayers’ money. Watch this excellent movie from Publish What You Fund on why financial aid transparency is needed.

One driver of IATI is, a co-organizer of the conference and member of the IATI secretariat, who have done some pioneering work in the area. Check out the AidInfo Labs to see what is possible through such data sets. We are curious to hear your ideas and projects.

Another driver of open aid data is the World Bank, who will also present their work on the conference. The World Bank has not only opened up its data, but also made an app competition, where many great solutions have been developed to use the data, for example games using development indicators, amazing visualizations, and crowdsourcing approaches. The aim is to make development aid more effective. Initiatives such as Ushahidi in Africa, demonstrate the potential of new forms of technology. Come join us at our Hackday to network with great people from the community.

You can apply here or contact Christian.Kreutz {at} for further questions. The event is organized by the Open Knowledge Foundation Germany.

Exploring open aid data with aidinfo labs

Theodora Middleton - August 16, 2011 in External, WG Development

The following is a guest post from Tim Davies, open data action researcher, currently curating the website for aidinfo, and a member of the OKFN Working Group on Open Development.

The International Aid Transparency Initiative (IATI) was set up in 2008 by 18 leading international development organisations. It aims to make information about aid spending easier to access, use and understand. Since the start of this year IATI has been making great progress securing access to data on aid activities and spending, with an agreed machine-readable data standard, a licensing framework, a registry of available data (based on the OKF’s CKAN software), and political commitments from over 20 donor countries and agencies to publish IATI data in the next year.

Alongside the IATI Standard and Registry which focus on making data accessible, provides a space to share efforts in making IATI data easier to understand and use: turning raw data into information that makes a difference. With a recent barcamp focussed on IATI data in Katmandu, a hack-day coming up in Berlin, and a growing community of users, we recently relaunched the aidinfo labs site to make sure we had all the ingredients that might be needed to make timely and accessible aid information a reality. We’ve created space for:

  • People: given the global development focus of IATI data, there can often be a distance between developers and data-wranglers working with data, and the people with some of the strongest demands for accessible aid information. We’ve put a ‘people’ section at the heart of the site where we’re going to be profiling real users of IATI data, and where we’re collecting a range of ‘open personae’ profiles, resources to give anyone working with the data a head-start in taking user-centred design approaches.

  • Plans: we’ve been finding that often similar ideas for working with a dataset can emerge at the same time in different places. By encouraging potential users of the data to share their plans and ideas at the start of a project, we can hopefully help collaborations emerge. The plans section also makes sure people with ideas for making aid information accessible and useful, but without the time or resources to put them into practice, can share the ideas.

  • Prototypes: there are already many prototypes built to show the potential of IATI data, from the IATI Explorer and IATI Data Ruby on Rails app, to visualisations of aid flows, graphs of funding allocated to projects, and ongoing work to convert IATI data for use in open spending, or to make it available as linked data. Some of these prototypes are ongoing works-in-progress, others were one-off experiments, created to explore and learn about the data. We’ve encouraged prototype creators to share source code or how-to’s so that others can build on their work in future.

  • Products: We’ve created a space for products that ‘graduate’ out of the labs and we’re working on a check-list that will help us identify the stable and supported IATI-using products that we can point users at when they want direct access to information and data, without sifting through different prototypes.

We’ve also brought together sections on the site for:

  • Datasets – showing that IATI is one amongst a number of open datasets. We’re using the CKAN API so that our list of datasets is kept up to date directly from The Data Hub.

  • Support – providing quick access to How To guidance, shared source code and tools that make working with the data easier, such as the IATI Explorer Toolkit, which takes all the different IATI XML files and gives easy access to query across them and fetch back the data in a number of formats. As we’ve been reminded recently from a number of angles, open data alone isnot enough. Aidinfo labs is part of our action learning to discover what else we need to add to data to make a difference, and discovering effective approaches to curating a growing global dataset.

Get involved:

With 10 more donors due to publish IATI data in the coming months, and the potential for IATI data to cover more than 50% of global aid by value by next year, there’s great potential for working with the data. If you’ve got ideas or experience to share, head over to the aidinfo labs site and get involved.

If you’re interested in wider issues of how open data and access to information can impact global development join discussions on the open-development working group mailing list.

Doing Good With Data: Data Without Borders

Theodora Middleton - July 19, 2011 in External, Featured Project, Open Data, WG Development

The following guest post is by Jake Porway from Data Without Borders.

We live in a time of unprecedented access to data and computational power. The open data movement is quickly digitizing and making available tomes of information about the way our governments work, the way our cities move, and the patterns of our daily lives. Moreover we have unbridled access to cloud computing resources and new open source data analysis to make sense of this data, placing us in an information age unrivaled in the course of human history. If that last sentence sounds a little grandiose, it’s because I want to emphasize that this burst in distributed computing and data collection feels like such a radical shift in the way we live that it should, accordingly, have a fundamental impact on the health of our cities, the state of our environment, and the quality of our lives as a species. Yet, as Jeff Hammerbacher correctly noted recently in BusinessWeek, “the best minds of my generation are thinking about how to make people click ads. That sucks.”

When I moved back to New York this year, I was blown away by the enthusiasm of the data and development community here, who would show up in full force to startup weekends and hack days to develop amazing tools in under 48 hours. Yet many of the end results of these events would feel like incremental improvements to already privileged lives, like taxi finders or restaurant review applications. At the same time, there were plenty of non-profits and NGOs who were striving to improve the world, yet had no resources to help them interpret the sudden deluge of newly available data. It seemed natural to get these two groups together.

In order to close this widening gap between the non-profits with the data and the people who had the skills and time to make sense of it, I’m founding a group called Data Without Borders. The goal is to provide a platform where socially conscious data scientists can work closely with NGOs on data-related projects to help them gain insight and improve their practices. Data do-gooders and needy NGOs would work together on both short- and long-term projects, ranging from a weekend to provide a quick analysis to month- or year-long placements to tackle larger problems. So far we’ve received an enormous response, with hundreds of people signing up to help out on both sides and I’ve been incredibly inspired by the enthusiasm on both sides of the table in making this project succeed.

It bears noting that there are many wonderful groups currently providing pro bono software development for non-profits, but Data Without Borders was designed to uniquely focus on data. With the role of “data scientist” still very new to the industry and many NGOs unaware of the power of using data, there is a serious need to connect those with the skills for data collection, analysis, and visualization, to the groups that don’t yet have those resources. We strive not to bring new software tools to NGOs, but instead to bring insight, analysis, and new data skills to these underserved groups. The hope is that this will empower NGOs who already recognize the power of data as well as inspire those who hadn’t thought along those lines before.

A critical thrust of Data Without Borders’s mission is to involve NGOs closely in this process. We want to avoid the scenario where an organization throws its data over the wall, we play with it by ourselves, then give them back what we found. If we’re truly going to affect the way that organizations think about data, they need to be an integral part of the process. Similarly, analysts and scientists need to understand the problems in the organization and will work closely with them to understand their needs. We’re planning to work with critical members of each NGO, from field workers to CTOs, to ensure that everyone in the process has their voice heard.

We recognize that data problems within large organizations can take the form of small projects to larger institutional changes. As such, we are planning to provide both short-term and long-term involvements with groups. We’ll be organizing a set of “Hack Days” or “Datathons” in which we pair NGOs that have targeted data needs with groups of data engineers to work on those problems over the course of a weekend or two. We are also looking to provide mechanisms for people to use their free time after work and on weekends to assist on longer running projects. Ideally we’ll be able to sponsor data scientists to work full time for extended periods of time with organizations, effectively giving them a pro bono data contractor for a couple of months to a year.

Data Without Borders is integrally tied to the open data community, both in using and sharing data wherever possible. While many organizations may already have their own data they’ve collected, there are many more that could benefit from the spate of open data resources that are popping up every day. We plan to integrate open datasets about education, government, and the environment with these projects, supplementing NGOs’ own data with openly available resources. In addition, any data that NGOs are willing to open up could be vastly useful for understanding their practices and an opportunity for the broader community to help them assess their impact.

We really feel that this organization has the potential to bring huge changes to the NGO and non-profit world in the way data is viewed and used. Moreover, it will provide exciting problems to the data community and give big hearted scientists a way to help. We’re busily getting organized for our kickoff event but, in the meantime, you can sign for our e-mail list at or follow us on Twitter or on Facebook. If you’re a non-profit or NGO who works with data, a socially conscious hacker, or someone who’s just generally interested in helping the world through technology, come join us in launching this exciting initiative!

The long road to open aid data!

Theodora Middleton - July 8, 2011 in External, WG Development

The following guest post is by Claudia Schwegmann from OpenAid, a member of the OKF’s Working Group on Open Knowledge in Development.

The road to open data in development cooperation has been a long one! 10 years ago, transparency, let alone open data, in development cooperation wasn’t an issue. In 2001 the Millennium Development Goals to reduce global poverty had just been formulated and there was considerable optimism that more of the same in development cooperation (more commitment, more money and more expertise) would help us to make tangible progress in health, education and other social sectors around the world.

Since then this optimism has faded. More of the same will not do and some serious changes in the aid system are needed. The need for accountability in development cooperation came into focus. At the High Level Forum on Aid Effectiveness in Paris in 2005 mutual accountability was highlighted as one of five prerequisites to effective aid. In 2006 key international non-government organisations launched the Accountability Charter for NGOs.

How can organisations and decision makers be accountable without being transparent about their activities and decisions? Accountability was very soon linked to the need for more transparency. Not surprisingly in 2008 donors, government representatives of aid recipient countries and civil society representatives declared at the High Level Forum on Aid Effectiveness in Accra “We will make aid more transparent” and the International Aid Transparency Initiative (IATI) was launched. Around the same time German NGOs got together and developed a transparency standard, which comprises a list of information items to be published either online or upon request by surface mail.

Only in 2010 did the notion of open data appear in development cooperation. In March 2010 the World Bank launched its open data initiative. Other big players like the Food and Agricultural Organisation, the Multi Donor Trust Fund and the British Department for International Development followed and at the beginning of 2011 the International Aid Transparency (IATI) agreed an international information standard largely in accordance with the open data definition of the OKF.

The good news is that a few powerful pioneers have adopted the concept open data in development cooperation. The bad news is that most donor staff, development workers, politicians and journalists working on development, even researchers have never heard of open data. And should you start to talk about databases, standardised formats, machine-readable data, APIs and data mash-ups you are very likely to instil fear and terror.

That is a shame. And it is time to reach out to people who are strongly committed to improving development cooperation and to reduce global poverty, but who do not see the potential of open data for their work. The Open Knowledge Foundation Germany, OpenAid e.V., the Heinrich Böll Foundation and Transparency International Germany are planning a large open aid data event in Berlin on the 28th and 29th of September to do just that. Other open aid data events are planned in Prague on the 4th of October, in Paris and in Stockholm.

At the Open Aid Data Event in Berlin the main day will be the conference on the 29th of September. Jörg Faust from the German Development Institute, Ronald Siebes from the Dutch Ministry of Foreign Affairs and a representative of the Worldbank will make the case for open data for better aid. AKVO, Development Gateway, Transparency International and other organisations will present examples of successful open data projects to explain the concept and the added value of open data. The specific examples will also allow to discuss concerns and challenges to open data.

The conference will be preceded by a more practical open aid data event on the 28th of September. The British NGO aidinfo, which is part of the IATI secretariat and one of the main driver of aid transparency internationally, will hold a data analysis workshop for NGO policy staff. Parallel to this will be a hackday with IATI data organised by the Open Knowledge Foundation Germany. Registration for the event is now open here.

Virtual Workshop on Linking Development Data, 12-13th May 2011

Guest - April 28, 2011 in Events, External, Open Data, WG Development, Working Groups, Workshop

The following guest post is from David Pidsley who is a member of the Open Knowledge Foundation’s Working Group on Open Knowledge in Development.

Open Data for Development Camp (ODDC) on 12th and 13th May 2011 in Amsterdam focuses on how developers, practitioners and policy makers can harness open international development data more effectively.

During the event we will be having an online Linking Development Data Workshop. The workshop format will be two days of intense online collaboration focused on ways of Linking Development Data.

The outcomes we are working towards are to empower recipients of aid, lead on transparency agenda, help reduce ineffective spending, increase visibility of social impact and to link aid spending to results.

If you’re interested in participating you can:

  • check out the Linking Development Data Google Doc,
  • watch the Twitter hash tag #openforchange,
  • participate via Skype by pinging davidpids
  • participate via IRC by joining #openforchange on Freenode

You can keep in touch by joining the following OKF mailing lists:

The Aid Revolution begins with XML

Guest - March 25, 2011 in Campaigning, External, WG Development, WG Open Government Data

The following guest post is by Claudia Elliot from Publish What You Fund.

IATI xml


After two years of negotiating, the 18 donors of the International Aid Transparency Initiative (IATI) agreed on February 9th the final details of a new global standard for publishing aid information. This format makes aid information internationally comparable, and so more information will now be better information.

DFID has already published its aid information, down to transaction-level data, in the IATI XML format. You can see their beautifully structured data on the IATI Registry. During the course of the year, more of the world’s biggest aid donors will begin to publish information on current aid projects in this standard format. With this format it will now be possible to build a bigger picture of aid activities which means that donors and recipients can coordinate their plans and complement the activities of others, reducing duplication and waste.

Publish What You Fund is currently working with the Open Knowledge Foundation to produce a visualisation of Uganda’s domestic budget with aid mapped onto it. This will show the real power of providing more information about aid flows in a standard format.

Use this data!

Of course, this information will only be useful if it is used. Please play with this data, and let us know what you make with it. When several donors are publishing to IATI, and then aid flows can be compared and combined, so the best part is yet to come. But even just looking at DFID’s data, it is possible for the first time to find the answers to all sorts of interesting questions:

  • Which companies or organisations are the biggest implementers of aid projects?
  • Are some better than others in starting on time (the difference between planned and actual start dates)?
  • Which sectors (health, education, etc.) does UK aid support, and how does that vary between countries?
  • What was the difference between commitments and disbursements in 2010?
  • How much money goes to climate change mitigation and adaption?

More data is now better

The new aid transparency standard provides a common language and format and a single way that donor countries can share information on the aid they are spending. However, to see the bigger picture, we need more data.

By the end of this month, we should know when each of the IATI signatories are planning to release their information in the IATI XML format. Major progress is also being made in the US. Although the US is not an IATI signatory, efforts to coordinate with the newly agreed standard suggest that as the US aid dashboard moves forward, foreign assistance could be mapped to both other donors’ and recipient countries’ own spending.

We’re also working with international partners to develop an Aid Transparency Tracker: a common framework and platform to track whether key types of aid information are available. As part of this, we want to create a public, online tool which tracks levels of available aid information, show changes over time, and facilitates clear, practical improvements in the levels of information.

If you think you could help us with this, please get in touch by emailing Mark at mark.brough[at]

Let’s build a Debian for Development Data

Guest - November 27, 2010 in CKAN, Events, External, Ideas and musings, OGDCamp, OKI Projects, Open Data, Open Knowledge Foundation, WG Development, Working Groups

The following guest post is from Rolf Kleef who is a member of the OKF’s Working Group on Open Knowledge in Development. It was originally posted here.

I just returned from an intense week in the UK: an IKM Emergent workshop in Oxford, and the  Open Government Data Camp in London had me almost drowning in “open data” examples and conversations, with a particular angle on aid data and the perspectives of international development.

As the result of that, I think we’re ready for a “Debian for Development Data”: a collection of data sets, applications and documentation to service community development, curated by a network of people and organisations who share crucial values on democratisation of information and empowerment of people.

“Open data” is mainstream newspaper content now

Mid 2009, after the 1%EVENT, a couple of innovative Dutch platforms came together to explore the opportunities of opening up our platforms: wouldn’t it be great if someone in an underdeveloped community had access to our combined set of services and information?

We had a hard time escaping new jargon (federated social networks, data portability, privacy commons, linked open data, the semantic web) and sketching what it would look like in five years. But then again, suppose it was five years earlier: in mid 2004, no-one could predict what Youtube, Facebook and Twitter look like today, even though many of us already felt the ground shaking.

  • The technical web was embracing the social web, of human connections.

  • The social web pushed "literacy": people wanted to participate and they learned how to do that.

A year and a half later, “open data” is catching up with us, and going through a similar evolution. Governments and institutions have started to release data sets (the Dutch government will too, the UK released data on all spending over £25,000 on Friday). So when will the social dimension be embraced in open data?

A week of open data for development

At an IKM Emergent workshop in Oxford, on Monday and Tuesday, around 25 people came together to talk about the impact of open data on international development cooperation. We discussed when we would consider "linked open data" a success for development. One key aspect was: getting more stakeholders involved.

Then at Open Government Data Camp (#OGDCamp) in London, on Thursday and Friday, around 250 people worked in sessions on all kinds of aspects of open data. Several speakers called for a stronger social component: both in the community of open data evangelists and in reaching out to those for whom we think open data will provide new opportunities for development.

At IKM, Pete Cranston described how his perception of access to information changed when a person approached him in a telecentre to ask how the price of silk changed on the international market: he was a union representative, negotiating with a company who wanted to cut worker salaries because of a decline in the market price. Without access to internet or the skills to use it, you don’t have the same confidence we have that such a question can be answered at all.

Then at OGDCamp, David Eaves reminded us that libraries were (partly) built before the majority of the population knew how to read, as an essential part of the infrastructure to promote literacy and culture 1.

Telecenters fulfil a role in underdeveloped communities as modern-day libraries, providing both access as well as the skills to access information and communication tools via the internet.

But we don’t have "open data libraries" or an infrastructure to promote "open data literacy" yet.

How open source software did it

It shouldn’t be necessary for people to become data managers just to benefit from open data sets. Intermediaries can develop applications and services to answer the needs of specific target groups based on linked open data, much as librarians help make information findable and accessible.

There are also parallels with open source software. Not every user needs to become a developer in order to use it. Although it is still to think otherwise sometimes, the open source movement has managed to provide easier interfaces to work with the collective work of developers.

The open data movement can identify a few next steps by looking at how the open source movement evolved.

Open Source

Open Data

Software packages (operating systems, word processors, graphics editors, and so on) are developed independently. Each software package can choose the programming language, development tools, the standards and best practices they use.

Data sets (budget overviews, maps, incident reports) are produced independently as well. The data formats and delivery methods can be chosen freely, and there are various emerging standards and best practices.

Communities around software packages usually set up mailing lists, chat channels and bug trackers for developers and users to inform each other about new releases, problems, and the roadmap for new versions. The mantra is "many eyes make all bugs shallow": let more people study the behaviour or the code of software, and errors and mistakes will be found and repaired more easily.

Data sets mainly are published. As Tim Davies noted in one of the conversations, there don’t seem to be mailing lists or release notes around data sets yet. To deliver the promise of a "wisdom of the crowds", users of data sets should have more and better ways to provide feedback and report errors.

Open source software is mostly used via distributions like Debian, Redhat, Ubuntu, separating producers and integrators. A distribution is a set of software packages, compiled and integrated in a way that makes them work well together, thereby lowering the barrier of entry to use the software. Distributions each have a different focus (free software, enterprise support, user-friendliness) and thus make different choices on quality, completeness, and interfaces.

Perhaps the current data sets released by governments could be considered "distributions", although the producer (a department) and the integrator (the portal manager) usually work for the same institution. could be considered a distribtion as well, although it does not (yet?) make clear choices on the type and the quality of data sets it accepts.

Software distributions make it possible to pool resources to make software interoperable, set up large-scale infrastructure, and streamline collaboration between "upstream" and "downstream". The open character stimulates an ecosystem where volunteers and businesses can work together, essential to create new business models.

Towards a "Debian for Development Data"

To sum up several concerns around open data for development:

  • Open data is currently mainly advocated for by developers and policy makers, without a strong involvement of other stakeholders (most noteworthy: those we like to benefit in underdeveloped communities). It tends to be driven mostly by web technology and is mostly focused on transparency of spending. It does not take into account (political) choices on why activities were chosen, and also lacks a lot in recording the results.

  • Data sets and ontologies are hard to find, not very well linked, with few generic applications working across data sets, and examples of good use of multiple data sets. Once you want to make data sets available, it is hard to promote the use of your data, provide feedback loops for improvements, administer dependencies, and keep track of what was changed along the way and why.

  • There are hardly any structural social components around current open data sets, repositories and registries.

So why don’t we start a "Debian for Development Data"?

  • A Social Contract and Open Data Guidelines like those for Debian can capture essential norms and values shared by community members, and inform decisions to be made. The contract can for instance value "actionable opportunties" over financial accountability. The Agile Manifesto is another example to draw from.

  • The community should set up basic communication facilities such as a mailing list, website, and issue tracker, to ease participation. Decision-making is essentially based on meritocracy: active participants choose who has the final say or how to reach consensus.

  • The data sets should be accompanied by software and documentation, to take away the problem of integration for most end users. Each data set and tool should have at least one "maintainer", who keeps an eye on updates and quality, and is the liaison for "upstream" data set publishers, offering a feedback loop from end-users to producers.

  • The CKAN software (powering the website mentioned before) draws on the lessons from distributions like Debian for its mechanisms to keep track of dependencies between data sets, and has version control, providing some support to track changes.

  • Ubuntu divides packages in categories like “core”, “non-free” and “ restrcited” to deal with license issues, and to express commitment of the community towards maintaining quality.

We stimulate the social component by providing more stakeholders a point of entry to get involved through socio-technical systems. We stimulate literacy by offering the stakeholders ways to get open data, publish their own, experiment with applications, and learn from each other. And we circumvent the tendency towards over-standardisation by doing this in parallel with other initiatives with sometimes overlapping goals and often different agendas.

1A quick check on Wikipedia indicates this seems to have mainly been the case in North-America, though.


Pushing the envelope

Guest - September 19, 2010 in External, Open Data, WG Development, Working Groups

The following guest post is from Francis Bacon, member of the Open Knowledge Foundation’s Working Group on Open Knowledge in Development and blogger at Pop Goes the Weasel.

A few weeks ago I received a large brown envelope in the post. It contained a letter, written in reply to a complaint I had made. And it contained a series of printed tables, showing how £154m of international aid funds have been spent.

The letter was from Oxfam, the international development charity. The tables included the name, location and amount spent for every overseas project funded by Oxfam in 2009-10. To my knowledge Oxfam are the first large British NGO to have released information about their spending in such detail.

The tables are interesting. Some of the projects listed have odd titles like “A Feminist look at Guatemalan reality” (£24,890). Others are poignant and sad: “Vulnerable Group Feeding in Midlands” (Zimbabwe, £2,959,757). Wherever you look you think 'what is this project?', 'why was so much spent on that?', 'were any of them effective?'.

Open aid data is important because it’s the first step to answering these questions. Organisations like Publish What You Fund and aidinfo are piecing together the incredibly complex jigsaw of how aid flows around the world. On its journey from the pocket of a rich tax-payer or charity-giver to a project in the developing world, aid funds pass through a multitude of organisations and funding bodies. These paths criss-cross and intertwine, resulting in an entangled web that is difficult to unpick.

The most damaging result of this complexity is that there can be no feedback loop between the individual donor and aid recipient. How can the public in rich countries hold aid agencies to account if they don’t even know where the money was spent or what it has been spent on?

Governments and multilateral organisations have at last started to engage with the transparency agenda. Last year the UK’s Department for International Development started publishing details of all their projects (although I have a long-running Freedom of Information request for some of the information they missed out). The World Bank and other governments are also starting to play ball.

But until now NGOs have declined to publish their data. In July 2010 I asked 8 large international development charities, including Oxfam, to provide a breakdown of their spending for the previous year. They all initially refused, on the grounds that it would be too costly to produce the data. I pointed out to some of them that, by refusing, they were breaching their own Open Information Policies. To their credit, Oxfam acknowledged this and subsequently changed tack.

Oxfam have shown that it isn’t overly costly to produce expenditure data and that doing so needn’t put staff at risk. We should encourage other NGOs to follow their example.

Get Updates