Support Us
Open Knowledge Foundation Open Knowledge
Foundation Blog

You are browsing the archive for Open Government Data.

Avatar of Kenneth Okwaroh

by Kenneth Okwaroh

Africanising the Open Government Partnership

June 17, 2013 in Open Government Data

This is cross-posted with permission from the Development Initiatives blog

“OGP will be real, only when it starts to make sense to the citizens out there” (Robert Hunja – World Bank)

The government of Kenya recently hosted the first ever OGP regional forum in Africa. The event aimed to establish guidelines for OGP activities for African countries; track and take stock of progress on the agenda to date and to think about how to Africanise the global OGP movement further. It attracted a range of delegates from across the continent involved and interested in the pursuit of open governance – government representatives, civil society actors, academia, the media, private sector, and multilateral institutions (see link for details).

Open governance is built on the principle of the universal right to access to information on the conduct of government and it places its value in enhancing effective public oversight. It aims to open up government affairs, (previously kept secret) to make it easier to scrutinise public officers and hold them to account. The argument is that access to government information enhances public participation and facilitates the audit of government actions. The Open Government Partnership (OGP) is a new multilateral global governance and transparency initiative formed in 2011. It aims to secure commitments from governments to promote transparency, empower citizens and harness new technologies to strengthen governance. The OGP outlines a set of principles augmented by a declaration that form the basis of the open government. To date 45 countries (three African) have endorsed the declaration, 11 others (two African) are currently processing commitments and many more across the globe are working towards attaining eligibility.

At the heart of the event was the question of how different partners could together put in place a strategy to strengthen open governance in the continent and bring forth an African perspective to the global OGP movement. Participants shared their experiences on the push for open and transparent institutions, accountable to the people, and brainstormed ideas on how the global OGP could respond. Discussions focused on: the role of big data, public statistics, analytics and technology’s role in improving service delivery; managing extractive industries to ensure processes are open, transparent, participatory and accountable; and how to leverage technology and the media to bolster citizen engagement and enhance public integrity.

Key issues emerging from the discussions included:

  • Inadequate understanding of the concept of open governance in Africa
  • The absence of strong champions for the agenda;
  • The impact of technocratic language used by OGP practitioners and how it fails to resonate with people at local level;
  • The lack of engagement among African leaders/governments and failure to share experiences;
  • The apparent overlap in governance monitoring mechanisms (OGP, African Union and United Nations, African Peer Review Mechanism etc ) and how this could be stifling progress
  • The need to explore both supply and demand side issues in scaling up OGP
  • The quality of information coming through existing open governance platforms
  • The inadequate engagement of African legislatures and private sector who have great influence, political muscle and interest in good governance in the continent.

The Civil Society/Government disconnect

Participants made the point that success of the OGP depends largely on trust and cooperation between government and civil society. Nonetheless, deliberations over the two day conference illustrated the clear tension between government and CSOs that must be addressed if we are to achieve meaningful progress.

There is currently a clear disconnect between the motivations, intentions and expectations of civil society and that of government, despite both working towards the common good of the citizen. Much work related to open governance appears to be taking place in silos with little coordination, mutual awareness and strategy. Though the need for effective partnerships between government and CSOs was emphasised, participants cautioned against very cosy relationships that create complacency and that could potentially jeopardise the watchdog role played by civil society.

“The era of sloganeering, CSO obstruction, political activism is gone [...] CSOs must reorient and repackage their engagement with government [... they] must begin to perceive themselves as partners with government”

– (Fred Matiang’i, Cabinet Secretary for ICT – Kenya)

It is our view that the African OGP steering committee must support states and civil society to collaborate with sufficient space for objective and constructive CSO monitoring and feedback in the role of ‘critical friend’. This will ensure better delivery and further progress in open governance.

Transforming commitments into action

It is encouraging that three African states are already members of the OGP, two others are processing commitments and a couple of others are eligible and looking to submit applications for membership. However, real success of the OGP will depend on implementation, monitoring and evaluation of the action plans developed by participating countries. Otherwise OGP membership risks being viewed as an end (utilised to score political mileage in the global political economy) rather than a means towards a greater goal of realising good governance and transparent and accountable government institutions.

We think that the OGP support unit must play an active role in monitoring participants to ensure they invest sufficiently in open governance, set realistic goals in their action plans, implement and accurately report on progress. Further, it must safeguard against monopolisation of the process by government (capable of overstating progress and being overambitious in planning) and minimising dialogue between CSOs and government The African experience must, in turn, inform the wider movement.

No Comments »

Avatar of Rufus Pollock

by Rufus Pollock

G8 countries must work harder to open up essential data

June 14, 2013 in Featured, News, Open Government Data

PRESS RELEASE. Cambridge, UK, 14th June 2013.

Also available in Chinese, German, Russian, Spanish and other languages here.

Open data and transparency will be one of the three main topics at the G8 Summit in Northern Ireland next week. Today transparency campaigners released preview results from the global Open Data Census showing that G8 countries still have a long way to go in releasing essential information as open data.

The Open Data Census is run by the Open Knowledge Foundation, with the help of a network of local data experts around the globe. It measures the openness of data in ten key areas including those essential for transparency and accountability (such as election results and government spending data), and those vital for providing critical services to citizens (such as maps and transport timetables). Full results for the 2013 Open Data Census will be released later this year.

g8 census

The preview results show that while both the UK and the US (who top the table of G8 countries) have made significant progress towards opening up key datasets, both countries still have work to do. Postcode data, which is required for almost all location-based applications and services, remains a major issue for all G8 countries except Germany. No G8 country scored the top mark for company registry data. Russia is the only G8 country not to have published any of the information included in the census as open data. The full results for G8 countries are online at: http://census.okfn.org/g8/

Rufus Pollock, Founder of the Open Knowledge Foundation, said:

“We’re delighted that many G8 countries have indicated their support for open data but today’s results show that progress is lagging behind promise. We call upon them to make good on their commitments and take a leading role in opening up the world’s data, to enable real transparency and accountability.”

Andrew Stott, former UK government Director for Transparency and Digital Engagement, who currently sits on the UK’s Public Sector Transparency Board, said:

“This is excellent work by the Open Knowledge Foundation’s community on measuring the reality of open data for the most important datasets. It shows that good progress has been made in recent years. However it also shows that there is more for all countries to do in order to deliver the open data vision and it gives each country a clear agenda for further improvement.”

Chris Taggart of OpenCorporates, the largest openly licensed database of companies in the world, said:

“Company registers are the fundamental public record of the creation and existence of companies. Today we live in a world where large corporations can consist of opaque networks of thousands of interlinked companies, avoiding scrutiny and competition. Criminals, money launderers, corrupt officials and fraudsters routinely use networks of front companies to hide and move money. In this context it is essential that access to the statutory information is not just freely available, but available under an open licence and as machine-readable data. Today’s results from the Open Data Census show that this message hasn’t yet got through to many of the world’s largest nations.”

David Eaves, co-founder of Open Data Day – which this year saw participants in 100 cities – and Openness advisor to the Mayor of Vancouver, said:

At a moment when many G8 members are trying to find ways to make government data more accessible the G8 Open Data Census could not be more timely. As a tool it offers a simple, easy to understand and clear way of evaluating how the different G8 countries are performing. I hope the leaders of each of those countries look at the census and use it as a way to learn from their peers and drive further change within their own government.

Charles Arthur, co-founder of the Free Our Data campaign and technology editor at the Guardian newspaper said:

As co-founder of the Free Our Data campaign, I’d say that it’s more important than ever for governments to make data available to us all so that we can understand and improve the world around us.

ENDS

To join the conversation about the Open Data Census, sign up to the Open Data Census mailing list

CONTACT
For enquiries please contact: press@okfn.org / +44 (0) 7795 176976 The Open Knowledge Foundation, St John’s Innovation Centre, Cowley Road, Cambridge, CB4 0WS UK.

NOTES FOR EDITORS

  • The G8 countries are meeting in County Fermanagh, Northern Ireland, UK, from June 17–18, 2013. The G8 countries are Canada, France, Germany, Italy, Japan, Russia, the United Kingdom, and the United States. Open data and transparency are one of the three main topics for this year’s event. See: https://www.gov.uk/government/topical-events/g8-2013

  • The Open Knowledge Foundation is a global movement to open up the world’s data and see it used and useful, empowering citizens with new knowledge and insights, and enabling fair and sustainable societies. The Foundation catalyses activities which promote and build on freely reusable open data and open content – including public information, publicly funded research and public domain cultural content. See: http://okfn.org/

  • The Open Data Census is coordinated by the Open Knowledge Foundation, using a network of local data experts around the globe to audit the levels of openness in each country. Full results for the 2013 Open Data Census will be released later this year. The datasets in the census are: Election Results; Company Register; National Map; Government Budget (by sector); Government Budget (transactional level data); Legislation; National Statistical Office Data (economic and demographic information); National Postcode/ZIP database; Public Transport Timetables; and Environmental Data on major sources of pollutants. For further information about the census, see this blog post. The preview results for the Open Data Census for G8 countries are available online at: http://census.okfn.org/g8/

  • The Open Definition sets out the principles which define “openness” in relation to data and content, to ensure that it can be freely used, reused and redistributed, and that it is interoperable with other open materials. Open materials must be freely usable and distributable by anyone, anywhere, for any purpose. To ensure this, the Open Definition requires that open material is accessible, in a suitable format, and has an appropriate open licence associated with it. See: http://opendefinition.org/

  • The United Kingdom and the United States both say that open data is a priority issue for their countries and for the world. US President Barack Obama and UK Prime Minister David Cameron have both made strong, explicit commitments to opening up official data. In May 2013 Obama released an Executive Order “making open and machine readable the new default for government information”, which was widely heralded as a major step. Over the past few years, the UK has released some of the most detailed spending information released by any government.

  • The Open Government Partnership is an international initiative to gain multilateral action on government openness, founded in 2011. Fifty-nine countries have subscribed to the Partnership so far. Half of the G8 countries are members of the Open Government Partnership (Canada, Italy, UK, US) and half are currently not (France, Germany, Japan, Russian Federation). Russia withdrew from the Open Government Partnership in May. See: http://www.opengovpartnership.org/

2 Comments »

Avatar of Christian Villum

by Christian Villum

Announcing the Global Open Data Initiative

June 11, 2013 in Featured, Open Government Data, Policy

The Global Open Data Initiative (GODI) is a new initiative led by the Open Knowledge Foundation, Open Institute, Fundar, Sunlight Foundation and the World Wide Web Foundation. It mission is to share principles and resources for governments and societies on how to best harness the opportunities created by opening government data.

The initiative is intended to provide a roadmap of policies and institutions that countries can use to build meaningful new open data reforms and initiatives, informed by the successes of others. Through GODI and associated work, we intend to expand awareness of open data and related issues, stimulate the development of the global open data community, provide a leading vision for how governments approach open data – as well as gather, expand, and amplify the evidence base for open data in general. Specifically:

Serve as a global guiding voice on open data issues

Civil society groups who focus on open data have often been isolated to single national contexts, despite the similar challenges and opportunities repeating themselves in countries across the globe. GODI is intended to help share valuable resources, guidance and judgment, and to clarify the potential for government open data globally.

Provide a leading vision for how governments approach open data

Open data commitments are among the most popular commitments for countries participating in the Open Government Partnership. GODI recommendations and resources will help guide open data initiatives and others as they seek to design and implement strong, effective open data initiatives and policies. GODI resources will also help civil society actors who will be evaluating government initiatives.

Increase awareness of open data

GODI will work to advance the understanding of open data issues, challenges, and resources by promoting best practices, engaging in online and offline dialogue, and supporting networking between organizations both new and familiar to the open data arena.

Support the development of the global open data community

Civil society organizations (CSOs) have a key role to play as suppliers, intermediaries, and users of open data, though at present, few organizations are engaging with open data and the opportunities it can make available for their work. Most CSOs lack the awareness, skills and support needed to be active users and providers of open data in ways that can help them meet their goals. GODI intends to help CSOs to engage with open data and use it as a key tool of their strategic programmes and activities in addressing issues such as climate change, democratic rights, land governance or financial reform. In addition, we hope to integrate the work of those CSOs already engaged in open data issues and usage, and to provide a map of the landscape of these and related actors.

Amplify and broaden the evidence base for open data

While the broader community of open data experts has come a long way in developing a research agenda and related resources, there is still a significant gap between the questions that can empower an open data agenda and those with the capacity to help answer them. GODI will work to identify areas for further study and development and will help build evidence-based policies and resources. We will also examine how open data can inform and strengthen decision-making processes within government.

Gather and strengthen existing resources

Open data resources and analysis have been explored and prepared by a number of different organizations, though this work is often scattered across the Internet and difficult to find. GODI seeks to aggregate these existing resources for inclusion with the resources we, too, will develop to create a central point of reference for governments and CSOs interested in open government data.

Find out more

You can find out more about the Global Open Data Initiative on its website and you also sign you to the mailing list.

Image: open government data (scrabble) by justgrimes on flickr, CC-BY-SA

1 Comment »

Avatar of Katleen Janssen

by Katleen Janssen

From PSI to open data – LAPSI is ready for a new round of legal questions

June 10, 2013 in Open Government Data, WG EU Open Data, WG Open Government Data

In February, 23 partners kicked off the LAPSI 2.0 thematic network on the legal aspects of public sector information in Leuven, Belgium. The network, consisting of academic institutions and stakeholders from 15 countries, will continue where the previous LAPSI network left off, and look at the remaining legal barriers hindering the full and open availability of public sector information in Europe. The network will enable knowledge exchange between stakeholders; showcase good practice on how Member States and public bodies deal with PSI issues; and provide policy recommendations on how the European legal framework can support open data.

This European legal framework is currently being challenged by the emerging open data ecosystem. PSI is gradually being replaced by open data in people’s minds, throwing up a lot of new questions. For instance, over the years, many efforts have been made by national policy makers and public authorities to create more transparency in licensing procedures and to develop standard licences (although more transparency would still be very welcome!). However, this has led – somewhat counter-productively – to a proliferation of licence models, even among the open licences. Therefore, the LAPSI 2.0 network is focusing its attention in the first year of activities on the ‘legal interoperability’ of licences. What strategies can help to prevent conflicting (open) standardised licensing models from arising, and how can existing problems due to a lack of interoperability be addressed?

Another layer of complication with licenses comes from the shift from the provision of data via bulk downloads to the creation of web services, requiring the combination of a data approach with what is traditionally known as terms of service or service level agreements. Moreover, the one-source, one-way delivery of information from the public sector to the users is increasingly being replaced by participatory data sharing, the introduction of feedback loops and the integration of PSI with user generated content. It is questionable if the current legal framework is ready for this.

The LAPSI 2.0 network will also be working hard to embed PSI and open data in the institutional culture of the public sector, and – if this does not work – on the enforcement of the rules on PSI and open data through efficient and effective redress mechanisms. While many public bodies have embraced open data, there are still many more that need to be convinced about the benefits for economic growth, participation and accountability.

Whatever LAPSI 2.0 recommends, it will have to function against the background of the new Directive on re-use of PSI, which is due this summer. While the new directive is definitely a step in the right direction, its exact impact can currently only be guessed at by the rumours that are seeping through about the trialogue process. We anxiously await the final version of the directive, and look forward to playing a role in the translation of the text into Member States’ domestic law.

Over the next two years, LAPSI 2.0, in cooperation with other projects and initiatives, will organise two conferences and a number of workshops on the legal aspects of PSI and open data. Our first conference is already planned: on October 24th, we hope to see you in Ljubljana for a great day on “The new PSI directive: what’s next?”. We are also planning workshops at the Samos Summit in July and you can find us at all the important open data events, including the OKCon in Geneva.

If you are interested in knowing more about the network and our activities, check out our website or register for the stakeholders newsletter.

No Comments »

Avatar of Christian Villum

by Christian Villum

Open knowledge at the Open Government Partnership conference in Mombasa, Africa

June 7, 2013 in Events, Open Government Data

Last week, the Open Knowledge Foundation had the pleasure of attending the Open Government Partnership conference in Mombasa, Kenya. Participants from all over Africa as well as the rest of the world convened to discuss transparency, citizen engagement and open knowledge – including open data – in this inspiring event to set the course for an open Africa.

The Open Government Partnership (OGP) is a multi-stakeholder coalition of leading governments and civil society organizations working to advance transparency and accountability in government with the goals of increasing the responsiveness of government to citizens, countering corruption, promoting economic efficiencies, harnessing innovation, and improving the delivery of services.

At the Open Knowledge Foundation we support these principles and attended the conference that attracted over 100 government and civil society leaders from Kenya, Tanzania, Ghana, Sierra Leone, Uganda, Mozambique, Liberia, South Africa, Malawi, Tunisia and other countries. There was also representation from other agencies such as the OGP Support Unit, the Independent OGP Civil Society Coordinator, the World Bank Institute, the African Commission on Human and Peoples’ Rights, UNESCO, APRM Support Unit and The World Bank.

The meeting secured agreement on several milestones, which are covered in detail in the summary on the Open Government Partnership blog, but most notably the event saw an increasingly visible and trustful relationship between government and civil society representatives – as well as the highlighting of open data in talks across several sessions as a central means to pursue the ideals and goals of the partnership.

To see a photo gallery from the event, visit the Kenya Open Data Facebook-page.


1 Comment »

Avatar of Laura James

by Laura James

Shakespeare review: analysis

May 15, 2013 in Access to Information, News, Open Data, Open Government Data

We welcome the Shakespeare review as a time to reflect, coming as it does at a time of great growth in open data in government and the public sector.

The UK has lead the way with government taking a pioneering stance on open data policy in recent years, and this report sets out key recommendations for how to best take forward this work.

It is particularly good to see acknowledgement that there is a “difference between a commitment to transparency and a true National Data Strategy for economic growth” as it is clear that many of the benefits of open public sector information will go beyond the economic.

As the Open Knowledge Foundation has long emphasized:

The best thing to be done with your data will be thought of by someone else

Shakespeare recognises this with the comment that “we cannot always predict where the greatest value lies but know there are huge opportunities across the whole spectrum of PSI.”

Getting more data released quickly, without agonising over quality concerns, is an excellent recommendation and we look forward to seeing this in practice. Alongside this we welcome the demand for high quality information in the National Core Reference Data plan, including key entity data; such reference data, following clear open standards, will transform what can be done with UK data. The request that Trading Funds should remove restrictive PSI licensing and work towards releasing all raw data for use and reuse is particularly warmly welcomed.

We are pleased to see consideration being given to privacy and confidentiality issues; our definition of open data has always excluded personally-identifiable information, but with greater data collection than ever before, we acknowledge the challenges this can bring for data publishers. The demand for realistic and pragmatic consideration of privacy and confidentiality is welcomed, and best practice guidelines will be very helpful in assisting data publishers here. In addition we hope to see key security and privacy sector experts engaged in this as there are tough technical challenges around anonymisation, aggregation and sandbox use, and deep technical understanding is needed to fully appreciate the risks and limits of such systems, and to create sensible guidelines.

We are also delighted to see open access mentioned in the report; open access to publicly-funded research data and papers has been a long-standing tenet of the Open Knowledge Foundation’s work. Shakespeare notes that “even today, access to academic research that has been paid for by the public is deliberately denied to the public, and to many researchers, by commercial publishers, aided by university lethargy, and government reluctance to apply penalties; thereby obstructing scientific progress.” We can, and must, do better here.

We applaud the call for more data scientists and greater statistical skills at all levels; stronger data awareness and skills are critical for all the benefits of open data to be realised. In particular, the recognition that interactive and workshop methods can be most effective at teaching data skills is well aligned with our own School of Data and long standing culture of hackathons and developer engagement. The more teaching and training around data, alongside other key STEM areas including maths and technology, the better.

Finally, it is great to see that the economic value of open data will be assessed through research and audit, but at the same time it is vital to be realistic about the timescales for significant change and impact in this field. We think on a timescale of decades to see the full benefits and effects of the new open approaches to creation, sharing and reuse of knowledge, and government and others must be realistic about what will be achieved and how quickly, to avoid disappointment.

Open data is valuable to us socially and culturally as well as commercially, but it is only one part of a solution, and we need to work on the other key elements, including institutional change, tools, skills and awareness, which are also necessary conditions to realise the full benefits of openness. These other elements may be harder, and more expensive, than the release of data – we should still release more open data, and we are glad to see this report affirming this and encouraging data skills alongside – but the journey is far from over.

As Shakespeare puts it:

“It is now time to build on the very positive start we have made on open data with a more directed, more predictable engineering of usable information. Obstacles must be cleared, structures defined, and progress audited, so that we have a purposeful, progressive strategy that we can trust to deliver the full benefits to the nation.”

If you’re interested in open data and you’d like to join our global community of open government data advocates, you can join our open-government mailing list:

Comments Off

Avatar of Mark Wainwright

by Mark Wainwright

Announcing CKAN 2.0

May 10, 2013 in CKAN, Featured, Featured Project, News, OKF Projects, Open Data, Open Government Data, Releases, Technical

CKAN is a powerful, open source, open data management platform, used by governments and organizations around the world to make large collections of data accessible, including the UK and US government open data portals.

Today we are very happy and excited to announce the final release of CKAN 2.0. This is the most significant piece of CKAN news since the project began, and represents months of hectic work by the team and other contributors since before the release of version 1.8 last October, and of the 2.0 beta in February. Thank you to the many CKAN users for your patience – we think you’ll agree it’s been worth the wait.

[Screenshot: Front page]

CKAN 2.0 is a significant improvement on 1.x versions for data users, programmers, and publishers. Enormous thanks are due to the many users, data publishers, and others in the data community, who have submitted comments, code contributions and bug reports, and helped to get CKAN to where it is. Thanks also to OKF clients who have supported bespoke work in various areas that has become part of the core code. These include data.gov, the US government open data portal, which will be re-launched using CKAN 2.0 in a few weeks. Let’s look at the main changes in version 2.0. If you are in a hurry to see it in action, head on over to demo.ckan.org, where you can try it out.

Summary

CKAN 2.0 introduces a new sleek default design, and easier theming to build custom sites. It has a completely redesigned authorisation system enabling different departments or bodies to control their own workflow. It has more built-in previews, and publishers can add custom previews for their favourite file types. News feeds and activity streams enable users to keep up with changes or new datasets in areas of interest. A new version of the API enables other applications to have full access to all the capabilities of CKAN. And there are many other smaller changes and bug fixes.

Design and theming

The first thing that previous CKAN users notice will be the greatly improved page design. For the first time, CKAN’s look and feel has been carefully designed from the ground up by experienced professionals in web and information design. This has affected not only the visual appearance but many aspects of the information architecture, from the ‘breadcrumb trail’ navigation on each page, to the appearance and position of buttons and links to make their function as transparent as possible.

[Screenshot: dataset page]

Under the surface, an even more radical change has affected how pages are themed in CKAN. Themes are implemented using templates, and the old templating system has been replaced with the newer and more flexible Jinja2. This makes it much easier for developers to theme their CKAN instance to fit in with the overall theme or branding of their web presence.

Authorisation and workflow: introducing CKAN ‘Organizations’

Another major change affects how users are authorised to create, publish and update datasets. In CKAN 1.x, authorisation was granted to individual users for each dataset. This could be augmented with a ‘publisher mode’ to provide group-level access to datasets. A greatly expanded version of this mode, called ‘Organizations’, is now the default system of authorisation in CKAN. This is much more in line with how most CKAN sites are actually used.

[Screenshot: Organizations page]

Organizations make it possible for individual departments, bodies, groups, etc, to publish their own data in CKAN, and to have control over their own publishing workflow. Different users can have different roles within an Organization, with different authorisations. Linked to this is the possibility for each dataset to have different statuses, reflecting their progress through the workflow, and to be public or private. In the default set-up, Organization user roles include Members (who can read the Organization’s private datsets), Editors (who can add, edit and publish datasets) and Admins (who can add and change roles for users).

More previews

In addition to the existing image previews and table, graph and map previews for spreadsheet data, CKAN 2.0 includes previews for PDF files (shown below), HTML (in an iframe), and JSON. Additionally there is a new plugin extension point that makes it possible to add custom previews for different data types, as described in this recent blog post.

[Screenshot: PDF preview]

News feeds and activity streams

CKAN 2.0 provides users with ways to see when new data or changes are made in areas that they are interested in. Users can ‘follow’ datasets, Organizations, or groups (curated collections of datasets). A user’s personalised dashboard includes a news feed showing activity from the followed items – new datasets, revised metadata and changes or additions to dataset resources. If there are entries in your news feed since you last read it, a small flag shows the number of new items, and you can opt to receive notifications of them via e-mail.

Each dataset, Organization etc also has an ‘activity stream’, enabling users to see a summary of its recent history.

[Screenshot: News feed]

Programming with CKAN: meet version 3 of the API

CKAN’s powerful application programming interface (API) makes it possible for other machines and programs to automatically read, search and update datasets. CKAN’s API was previously designed according to REST principles. RESTful APIs are deservedly popular as a way to expose a clean interface to certain views on a collection of data. However, for CKAN we felt it would be better to give applications full access to CKAN’s own internal machinery.

A new version of the API – version 3 – trialled in beta in CKAN 1.8, replaced the REST design with remote procedure calls, enabling applications or programmers to call the same procedures as CKAN’s own code uses to implement its user interface. Anything that is possible via the user interface, and a good deal more, is therefore possible through the API. This proved popular and stable, and so, with minor tweaks, it is now the recommended API. Old versions of the API will continue to be provided for backward compatibility.

Documentation, documentation, documentation

CKAN comes with installation and administration documentation which we try to keep complete and up-to-date. The major changes in the rest of CKAN have thus required a similarly concerted effort on the documentation. It’s great when we hear that others have implemented their own installation of CKAN, something that’s been increasing lately, and we hope to see even more of this. The docs have therefore been overhauled for 2.0. CKAN is a large and complex system to deploy and work on improving the docs continues: version 2.1 will be another step forward. Where people do run into problems, help remains available as usual on the community mailing lists.

… And more

There are many other minor changes and bug fixes in CKAN 2.0. For a full list, see the CKAN changelog.

Installing

To install your own CKAN, or to upgrade an existing installation, you can install it as a package on Ubuntu 12.04 or do a source installation. Full installation and configuration instructions are at docs.ckan.org.

Try it out

You can try out the main features at demo.ckan.org. Please let us know what you think!

Comments Off

Avatar of Martin Tisne

by Martin Tisne

Follow the Money, Follow the Data

May 3, 2013 in Ideas and musings, Open Data, Open Government Data, Open Spending

The following guest post from Martin Tisné was first published on his personal blog.

Money tunnel by RambergMediaImages, CC-BY-SA on Flickr

Some thoughts which I hope may be helpful in advance of the ‘follow the data‘ hack day this week-end:

The open data sector has quite successfully focused on socially-relevant information: fixing potholes a la http://www.fixmystreet.com/, adopting fire hydrants a la http://adoptahydrant.org/. My sense is that the next frontier will be to free the data that can enable citizens, NGOs and journalists to hold their governments to account. What this will likely mean is engaging in issues such as data on extractives’ transparency, government contracting, political finance, budgeting etc. So far, these are not the bread and butter of the open data movement (which isn’t to say there aren’t great initiatives like http://openspending.org/). But they should be:

At its heart, this agenda revolves around ‘following the money’. Without knowing the ‘total resource flow’:

  • Parents’ associations cannot question the lack of textbooks in their schools by interrogating the school’s budget
  • Healthcare groups cannot access data related to local spending on doctors, nurses
  • Great orgs such as Open Knowledge Foundation or BudgIT cannot get the data they need for their interpretative tools (e.g. budget tracking tool)
  • Investigative journalists cannot access the data they need to pursue a story

Our field has sought to ‘follow the money’ for over two decades, but in practice we still lack the fundamental ability to trace funding flows from A to Z, across the revenue chain. We should be able to get to what aid transparency experts call ‘traceability’ (the ability to trace aid funds from the donor down the project level) for all, or at least most fiscal flows.

Open data enables this to happen. This is exciting: it’s about enabling follow the money to happen at scale. Up until now, instances of ‘following the money’ have been the fruit of the hard work of investigative journalists, in isolated instances.

If we can ensure that data on revenues (extractives, aid, tax etc), expenditures (from planning to allocation to spending to auditing), and results (service delivery data) is timely, accessible, comparable and comprehensive, we will have gone a long way to helping ‘follow the money’ efforts reach the scale they deserve.

Follow the Money is a pretty tangible concept (if you disagree, please let me know!) – it helps demonstrate how government funds buy specific outcomes, and how/whether resources are siphoned away. We need to now make it a reality.

1 Comment »

Avatar of Adi Eyal

by Adi Eyal

Opening Public Data in South Africa

April 11, 2013 in OKFN Local, Open Government Data

Cape Town City Hall, Felix Gottwald

It seems somewhat absurd to me that publicly funded institutions in South Africa should be allowed to copyright data produced using public funds. Of course, it is reasonable to expect that physical assets such as buildings, vehicles or machinery should appear on their balance sheets and be reserved for their exclusive use. But knowledge? I’m not so sure. Assuming that the information in question cannot reasonably be considered a state secret, the revelation of which would harm national interests, I would expect that it cannot be owned or its use restricted to taxpayers.

Here’s an recent example:

municipal demarcation board The Municipal Demarcation Board is an independent organisation tasked by government with determining the borders between municipalities in South Africa. They are publicly funded, having received R38.5m ($4.3m) last year from the national treasury. Boundaries are published in the government gazette and are considered to be public knowledge. I am not a lawyer, but intuitively, this data cannot be considered to enjoy protection of copyright laws.

In order to clarify the situation, I contacted them and asked whether, as a citizen of South Africa, I was allowed to download the data from their site and use it for commercial purposes. The response that I received was:

“Unfortunately there is a lot of commitment to our data and it is copyright to us and we cannot allow you to use it commercially at all.”

As an exercise, I surveyed copyrights on the websites of 4 national agencies, including the national statistics agency and the electoral commission, as well as the websites of all 9 provinces and the 8 metropolitan municipalities. In some cases, licensing was unclear or missing. In the remaining cases, copyright was claimed and commercial use excluded, even for derived data products. This is stats sa reminiscent of the GPL licence but in reverse (all derived products should be restricted under the same rules). Most licences allowed for personal use and for reference purposes while some granted re-distribution rights (to third-parties for personal use or reference purposes).

It is of course possible that these copyrights have been applied as boilerplate by over-zealous lawyers, but our one data point – the response of the Municipal Demarcation Board – suggests that this is not the case.

As an exercise, I emailed each of these agencies requesting special permission for commercial rights (commercial is important here – in lieu of an actual open data policy, commercial rights give me the broadest possible scope for data use that I can hope for). I am eagerly awaiting their responses. You can follow the action on this spreadsheet as I update their policies as the replies come in.

Taking a step back, we should understand why this is important and not just a bunch of navel gazing. After all, we have a few really fantastic datasets “freely” available (free as in beer). The elections datasets (e.g. here and here) are unprecedented in their depth and detail. The City of Cape Town manages a very rich dataset on municipal valuations, and the national parliament hosts an intriguing database on gifts received by parliamentarians and other elected officials. However, aside from the fact that most of our data is not in machine-readable format (easily solved by web-scraping although this technology is itself encumbered by sticky legal questions), we are explicitly excluded from using this data for anything but personal use.

Open data has not yet percolated into the collective national consciousness. Some momentum has started to build through a number of niche groups of individuals seduced by the goings-on up north and even on the African continent. For this small sign of life to grow into a fully-fledged ecosystem of data consumers and producers, we need greater access to data under the stewardship of public entities. We need to be able to access that data for personal and for commercial use. We need to be able to download it and distribute derived products. And we should not have to ask for permission every time we do so.

Our post-apartheid constitution guarantees the right to access of information. It’s time that citizens of South Africa asserted that right.

10 Comments »

Avatar of Martin Virtel

by Martin Virtel

LobbyPlag – Who is really writing the law?

March 22, 2013 in Featured Project, Open Government Data

Sometimes, the band continues to play because the audience is enjoying the music so much. This is pretty much what happened to Lobbyplag. Our plan was to drive home a single point that outraged us: Some Members of the European Parliament were taking law proposals verbatim from lobbyists and trying to slip them into the upcoming EU privacy law. They actually copy-and-pasted texts provided by the likes of Amazon, Google, Facebook or some banking industry body. The fact itself was Max Schrems’ discovery. Max is a lawyer, and he sought the help of Richard Gutjahr and the data journalists and developers from OpenDataCity – to present his evidence to the public in form of a website called Lobbyplag. The name evokes memories of past projects where people had hunted down plagiarism in the doctoral theses of German politicians.

Lobbyplag – discover the copy&paste politicians from Martin Virtel on Vimeo.

A lovestorm of reactions ensued, not only from the usual consumer privacy advocates. The site struck a chord among lobbying-stressed lawmakers and outraged citizens alike. Wolfgang Thierse, the president of the German Parliament, called it “a meritorious endeavor”, and two European lawmakers pledged to disclose their sources. People started proposing other laws to look at, started sending us papers from lobbyists, and offered their help for finding more lobby-plagiarizing politicians.

What had happened? Looking into the details of Privacy Law is not normally a crowd-pleaser, and like most laws this one was being made out sight, watched over only by a few specialists. This is the norm especially for the EU parliament, which still doesn’t attract a level of public attention and scrutiny to match its real power. There had already been a lot of reports about the intense lobbying against the Privacy Law.

Lobbyplag made a difference because Lobbyplag set a different tone. We simply presented the proof of what was being done behind closed doors – and gave people the power to look it up for themselves. And they did. And they liked it. And asked for more.

lobbyplag-the-stats-from-the-imco-committee

At that point, we decided that this was to be more than a single issue website, this was a public utility in the making. We successfully completed a 8000€ crowdfunding campaign at Krautreporter.de, a fledgling German platform, and we are now building the tools that interested citizens (assisted by algorithms) will need to make the comparisons between lobbyist texts and law amendments, and draw the conclusions by themselves. Stefan’s Parltrack project, which provides APIs to the European Parliament’s paperwork, will provide the foundation, as it did for the first iteration of lobbyplag, and we’re looking at using the Open Knowledge Foundation’s pybossa, a microtasking framework (you can see it in action at crowdrafting.org).

Of course, the first round of money is only a start – we’re a team of volunteers – so we also submitted Lobbyplag to the Knight News Challenge, which this year fittingly is looking to support projects that improve the way citizens and governments interact – you can read more about the proposal and provide feedback on the Knight News page.

We think that making comparisons easy and bringing lobbying out into the light is a way to achieve that. There’s nothing inherently wrong with lawmakers relying on experts when they’re not experts themselves – you’d expect them to. But if they hide who they’ve been listening to, and if they only listen to one side, they contribute towards public distrust in their profession. Making the process of lawmaking and influencing lawmakers more transparent will result in better debate, better understanding and better laws.

There’s a saying that “Laws, like sausages, cease to inspire respect in proportion as we know how they are made” – but we think that is not true any longer. Citizens all over the world are not really willing to respect lawmakers unless they can trace what they are stuffing in there.

Comments Off

Please create an account to get started.

Sign up to the Open Knowledge Newsletter

Get Updates