Support Us
Open Knowledge Foundation Open Knowledge
Foundation Blog

You are browsing the archive for Open Data.

Avatar of Rufus Pollock

by Rufus Pollock

G8 Open Data Charter Highlights Open Data as Crucial for Governance and Growth

June 18, 2013 in Featured, Open Data, Policy

Today’s release of an Open Data Charter by the G8 is testimony to the growing importance of open data worldwide. The Charter recognizes the central role open data can play in improving government and governance and in stimulating growth through innovation in data-driven products and services. It endorses the principle of “open by default” — also supported in President Obama’s recent Executive Order on open data — and makes clear that open data must be open to all and usable by both machines and humans (as per the Open Definition).

As, Rufus Pollock, Founder of the Open Knowledge Foundation said: “We are delighted to see such high-level endorsement of the key principles of open data and transparency which the Open Knowledge Foundation, together with many others, have been campaigning for over a period of many years. At the same time, there is still much for the G8, and other countries, to do.”

The early results from Open Data Census reported last week show that, even for a small number of core datasets, G8 countries in many cases have a long way to go in opening up essential data. It is therefore good to see that the Charter recognizes a list of “high value datasets” which should be prioritized for release, though it is disappointing that there are no explicit commitments to release the types of data mentioned (and as the Census results showed there is much to be done in this area).

Moreover, as the Charter acknowledges with its 2nd principle on quality and quantity, it’s not just about open data but about quality data — the value of open data will be much diminished if the data turns out to be missing crucial information. In this area much of the devil is in the detail and it will take some detailed follow-up work to make sure this principle turns into practice.

As a concrete example of the quality point, in our work with open data on government finances in the OpenSpending project we’ve often been hampered by the lack of crucial identifiers (for example, for companies or departments), or by data that does not have the granularity to enable it to be used to answer key questions (such as how much was spent on project X), or by simple inaccuracy and unreliability.

Martin Tisne of the Omidyar Network said: “We need to benchmark what excellence means in open data and set a standard so that government reformers are empowered and civil society can engage and monitor. The Open Data Charter does this and offers its principles for consideration to other countries and initiatives. Open data is the most popular commitment out of hundreds put forward by close to 60 countries part of the Open Government Partnership. The Charter will be a great tool for these countries to develop ambitious and meaningful open data initiatives.”

Finally, Governments will have to think hard about how to turn transparency into accountability. This may involve both developing skills and innovations, as mentioned in the Charter, but also thought about the kinds of incentives, and changes in governance, that will make transparency actionable.

Open Data Charter

The Open Data Charter consists of a main section with 5 principles and a technical annexe which is described a “living set of guidelines” that flesh out best practices around the 5 principles. The principles are:

  • Open Data by Default
  • Quality and Quantity
  • Useable by All
  • Releasing Data for Improved Governance
  • Releasing Data for Innovation

Preamble

6) Open data can increase transparency about what government and business are doing. Open data also increase awareness about how countries’ natural resources are used, how extractives revenues are spent, and how land is transacted and managed. All of which promotes accountability and good governance, enhances public debate, and helps to combat corruption. Transparent data on G8 development assistance are also essential for accountability.

7). Providing access to government data can empower individuals, the media, civil society, and business to fuel better outcomes in public services such as health, education, public safety, environmental protection, and governance. Open data can do this by:

  • showing how and where public money is spent, providing strong incentives for that money to be used most effectively;
  • enabling people to make better informed choices about the services they receive and the standards they should expect.

…

Principles

9) We, the G8, agree that open data are an untapped resource with huge potential to encourage the building of stronger, more interconnected societies that better meet the needs of our citizens and allow innovation and prosperity to flourish.

10) We therefore agree to follow a set of principles that will be the foundation for access to, and the release and re-use of, data made available by G8 governments. They are:

  • Open Data by Default
  • Quality and Quantity
  • Useable by All
  • Releasing Data for Improved Governance
  • Releasing Data for Innovation

Lough Erne Declaration

The main Lough Erne Declaration dedicates to 5 of the 10 points to transparency and open data issues:

… Governments have a special responsibility to make proper rules and promote good governance. Fair taxes, increased transparency and open trade are vital drivers of this. We will make a real difference by doing the following:

…

2. Countries should change rules that let companies shift their profits across borders to avoid taxes, and multinationals should report to tax authorities what tax they pay where.

3. Companies should know who really owns them and tax collectors and law enforcers should be able to obtain this information easily.

…

5. Extractive companies should report payments to all governments – and governments should publish income from such companies.

…

7. Land transactions should be transparent, respecting the property rights of local communities.

…

10. Governments should publish information on laws, budgets, spending, national statistics, elections and government contracts in a way that is easy to read and re-use, so that citizens can hold them to account. [emphasis added]

Related Government Statements

Ensure that trustees of express trusts are obliged to obtain and hold adequate, accurate and current information on beneficial ownership regarding the trust.

Images: G8 Leaders Image CC-BY-NC on G8UK flickr

No Comments »

Avatar of Velichka Dimitrova

by Velichka Dimitrova

Second Open Economics International Workshop

June 5, 2013 in Events, Featured, Open Data, Open Economics, WG Economics, Workshop

Next week, on June 11-12, at the MIT Sloan School of Management, the Open Economics Working Group of the Open Knowledge Foundation will gather about 40 economics professors, social scientists, research data professionals, funders, publishers and journal editors for the second Open Economics International Workshop.

The event will follow up on the first workshop held in Cambridge UK and will conclude with agreeing a statement on the Open Economics principles. Some of the speakers include Eric von Hippel, T Wilson Professor of Innovation Management and also Professor of Engineering Systems at MIT, Shaida Badiee, Director of the Development Data Group at the World Bank and champion for the Open Data Initiative, Micah Altman, Director of Research and Head of the Program on Information Science for the MIT Libraries as well as Philip E. Bourne, Professor at the University of California San Diego and Associate Director of the RCSB Protein Data Bank.

The workshop will address topics including:

  • Research data sharing: how and where to share economics social science research data, enforce data management plans, promote better data management and data use
  • Open and collaborative research: how to create incentives for economists and social scientists to share their research data and methods openly with the academic community
  • Transparent economics: how to achieve greater involvement of the public in the research agenda of economics and social science

The knowledge sharing in economics session will invite a discussion between Joshua Gans, Jeffrey S. Skoll Chair of Technical Innovation and Entrepreneurship at the Rotman School of Management at the University of Toronto and Co-Director of the Research Program on the Economics of Knowledge Contribution and Distribution, John Rust, Professor of Economics at Georgetown University and co-founder of EconJobMarket.org, Gert Wagner, Professor of Economics at the Berlin University of Technology (TUB) and Chairman of the German Census Commission and German Council for Social and Economic Data as well as Daniel Feenberg, Research Associate in the Public Economics program and Director of Information Technology at the National Bureau of Economic Research.

The session on research data sharing will be chaired by Thomas Bourke, Economics Librarian at the European University Institute, and will discuss the efficient sharing of data and how to create and enforce reward structures for researchers who produce and share high quality data, gathering experts from the field including Mercè Crosas, Director of Data Science at the Institute for Quantitative Social Science (IQSS) at Harvard University, Amy Pienta, Acquisitions Director at the Inter-university Consortium for Political and Social Research (ICPSR), Joan Starr, Chair of the Metadata Working Group of DataCite as well as Brian Hole, the founder of the open access academic publisher Ubiquity Press.

Benjamin Mako Hill, researcher and PhD Candidate at the MIT and Berkman Center for Internet and Society at Harvard Univeresity, will chair the session on the evolving evidence base of social science, which will highlight examples of how economists can broaden their perspective on collecting and using data through different means: through mobile data collection, through the web or through crowd-sourcing and also consider how to engage the broader community and do more transparent economic research and decision-making. Speakers include Amparo Ballivian, Lead Economist working with the Development Data Group of the World Bank, Michael P. McDonald, Associate Professor at George Mason University and co-principle investigator on the Public Mapping Project and Pablo de Pedraza, Professor at the University of Salamanca and Chair of Webdatanet.

The morning session on June 12 will gather different stakeholders to discuss how to share responsibility and how to pursue joint action. It will be chaired by Mireille van Eechoud, Professor of Information Law at IViR and will include short statements by Daniel Goroff, Vice President and Program Director at the Alfred P. Sloan Foundation, Nikos Askitas, Head of Data and Technology at the Institute for the Study of Labor (IZA), Carson Christiano, Head of CEGA’s partnership development efforts and coordinating the Berkeley Initiative for Transparency in the Social Sciences (BITSS) and Jean Roth, the Data Specialist at the National Bureau of Economic Research.

At the end of the workshop the Working Group will discuss the future plans of the project and gather feedback on possible initiatives for translating discussions in concrete action plans. Slides and audio will be available on the website after the workshop. If you have any questions please contact economics [at] okfn.org

Comments Off

Avatar of Iain Hrynaszkiewicz

by Iain Hrynaszkiewicz

Open Knowledge may yet come to medicine – let’s help make it happen

May 20, 2013 in Campaigning, Open Data, Open Science

Today is International Clinical Trials Day. To mark the event, here’s a post from Iain Hrynaszkiewicz reviewing the current state of open knowledge in medicine. You can see an earlier version on F1000’s blog.

The European Medicines Agency (EMA), the organisation which approves drug license applications from the pharmaceutical industry in Europe, has made important progress towards more open science. They hope to release anonymised data from drug trials online, but are faced with widely divided opinions on how data sharing should happen, as well as legal challenges in making it happen. The Open Knowledge community has a chance to help produce better outcomes for the beneficiaries of medical research.

On 30 April 2013 the EMA published advice documents, which cover five different aspects of clinical data sharing and are designed to help the EMA craft their policy on data release. The advice was sourced from around 200 volunteers from across the drug industry, academic research, publishing, and patient advocacy communities. I’ll be the first point out this is not an open data policy – it’s a data sharing or data access policy. The EMA is, along with most medical research, a long way from implementing an Open Knowledge-compliant data policy – with data rapidly released in machine-readable formats to the public domain. But amongst the documents released there are some pertinent developments – worrying and promising in equal measures – that the open science community should recognise now.

Copyright and licenses

One suggestion in the EMA’s legal advisory group was that all data submitted to the EMA would be protected by copyright under the EU Database Directive. This seems unlikely, as it assumes all trial data are in a database. Data take many forms within and without databases. Whether copyright applies to data is a much debated issue depending on, amongst other things, the legal jurisdiction. However, Creative Commons CC0 was proposed to the EMA as possible solution to this problem. Data repositories Dryad and figshare were used as examples along with the journal F1000Research, which was the first journal to use the CC0 public domain dedication waiver for data it publishes.

Data formats and standards

Data standards breed efficiency – efficient reuse, sharing, understanding and computation. The advice to the EMA on data formats includes some promising recommendations. The advisory group was quick to recognise the importance of clinical data standards such as CDSIC and file formats that can be read with open source software. But to avoid delays in implementing the policy it seems likely that such standards will not be required and “any format shall be acceptable for all data until the policy is applied by stakeholders”. PDF, a format widely discouraged for data, was even recommended by some as a format for some types of data.

Many other issues were covered, and the documents are available with full version history.

Making more science data and research results available openly ultimately means faster progress in solving the most difficult problems facing the world. In medicine the benefits of doing more reliable science through open data are the most tangible. People’s health is improved. But much of the clinical research community are not even used to sharing or being able to share – publish – the reports of their work (papers in journals) let alone their raw data.

Publication bias, where positive trials are more frequently published than negative trials, has been found in more than 50 different treatments including widely prescribed antidepressants and anitvirals. A lack of available platforms is not the barrier. Many journals accept or encourage negative results – including F1000Research which just launched a fee waiver for negative results –and various repositories can accept negative data.

The EMA’s initiative comes at a time when there is unprecedented attention on access to information from medical research in the UK and EU. The UK Government’s Science and Technology Select Committee is reviewing large amounts of oral and written evidence on its recent inquiry on clinical trials. The Alltrials campaign for the reporting and registration of all trial results – an initiative of Sense About Science, BMJ and others and fronted by Dr Ben Goldacre – has amassed more than 50,000 signatures.

Medical research is finally moving, albeit slowly, to a new default of open. The open science and open knowledge community should support and guide the EMA and other interested parties in taking these important steps towards open data. And to mark International Clinical Trials Day, go sign the Alltrials petition! This is a real chance to change medical evidence for the better.

Comments Off

Avatar of Laura James

by Laura James

Shakespeare review: analysis

May 15, 2013 in Access to Information, News, Open Data, Open Government Data

We welcome the Shakespeare review as a time to reflect, coming as it does at a time of great growth in open data in government and the public sector.

The UK has lead the way with government taking a pioneering stance on open data policy in recent years, and this report sets out key recommendations for how to best take forward this work.

It is particularly good to see acknowledgement that there is a “difference between a commitment to transparency and a true National Data Strategy for economic growth” as it is clear that many of the benefits of open public sector information will go beyond the economic.

As the Open Knowledge Foundation has long emphasized:

The best thing to be done with your data will be thought of by someone else

Shakespeare recognises this with the comment that “we cannot always predict where the greatest value lies but know there are huge opportunities across the whole spectrum of PSI.”

Getting more data released quickly, without agonising over quality concerns, is an excellent recommendation and we look forward to seeing this in practice. Alongside this we welcome the demand for high quality information in the National Core Reference Data plan, including key entity data; such reference data, following clear open standards, will transform what can be done with UK data. The request that Trading Funds should remove restrictive PSI licensing and work towards releasing all raw data for use and reuse is particularly warmly welcomed.

We are pleased to see consideration being given to privacy and confidentiality issues; our definition of open data has always excluded personally-identifiable information, but with greater data collection than ever before, we acknowledge the challenges this can bring for data publishers. The demand for realistic and pragmatic consideration of privacy and confidentiality is welcomed, and best practice guidelines will be very helpful in assisting data publishers here. In addition we hope to see key security and privacy sector experts engaged in this as there are tough technical challenges around anonymisation, aggregation and sandbox use, and deep technical understanding is needed to fully appreciate the risks and limits of such systems, and to create sensible guidelines.

We are also delighted to see open access mentioned in the report; open access to publicly-funded research data and papers has been a long-standing tenet of the Open Knowledge Foundation’s work. Shakespeare notes that “even today, access to academic research that has been paid for by the public is deliberately denied to the public, and to many researchers, by commercial publishers, aided by university lethargy, and government reluctance to apply penalties; thereby obstructing scientific progress.” We can, and must, do better here.

We applaud the call for more data scientists and greater statistical skills at all levels; stronger data awareness and skills are critical for all the benefits of open data to be realised. In particular, the recognition that interactive and workshop methods can be most effective at teaching data skills is well aligned with our own School of Data and long standing culture of hackathons and developer engagement. The more teaching and training around data, alongside other key STEM areas including maths and technology, the better.

Finally, it is great to see that the economic value of open data will be assessed through research and audit, but at the same time it is vital to be realistic about the timescales for significant change and impact in this field. We think on a timescale of decades to see the full benefits and effects of the new open approaches to creation, sharing and reuse of knowledge, and government and others must be realistic about what will be achieved and how quickly, to avoid disappointment.

Open data is valuable to us socially and culturally as well as commercially, but it is only one part of a solution, and we need to work on the other key elements, including institutional change, tools, skills and awareness, which are also necessary conditions to realise the full benefits of openness. These other elements may be harder, and more expensive, than the release of data – we should still release more open data, and we are glad to see this report affirming this and encouraging data skills alongside – but the journey is far from over.

As Shakespeare puts it:

“It is now time to build on the very positive start we have made on open data with a more directed, more predictable engineering of usable information. Obstacles must be cleared, structures defined, and progress audited, so that we have a purposeful, progressive strategy that we can trust to deliver the full benefits to the nation.”

If you’re interested in open data and you’d like to join our global community of open government data advocates, you can join our open-government mailing list:

Comments Off

Avatar of Jonathan Gray

by Jonathan Gray

The future of open data in the UK: what we hope the Shakespeare review says

May 14, 2013 in Open Data, Policy

Tomorrow morning will see the release of a major new review on how to make the most out the UK’s Public Sector Information authored by Stephan Shakespeare, founder of opinion polls company YouGov.

Given our role in advocating open data in the UK for many years, we’re very keen to see what Shakespeare says. Here are a few of our thoughts about what we hope to see in the review.

Strong commitment to open data as the default for UK Public Sector Information.

On the back of President Obama’s recent Executive Order announced last week that says public information in the US should be “open and machine readable” by default, we hope Stephan Shakespeare’s review will urge similarly strong support for open data in the UK.

In particular we hope he pushes the UK to do more to open up raw data currently sold by trading funds, and key datasets like the Postcode Address File (which is a crucial part of any website or app which involves putting things on maps).

Supporting standards to make open data more usable and useful

We hope that the review contains recommendations about developing and promoting better standards to make open data more usable and useful.

From spending information, to carbon emissions data, to health data, we hope to see initiatives to ensure greater standardisation of public information across central and local government to lower barriers to reuse. (This point was also alluded to in a article on the Guardian published earlier today.)

We’ve started work in this direction with our Data Protocols, but we’d really like to see the UK government doing more in this area. Given its role in the Open Government Partnership, it could help to create standards that could be used not just in the UK, but around the world.

Open data isn’t just about money

While we expect the economic potential of open data to be a major focus area for tomorrow’s review, we hope Shakespeare recognises that open data is not just about money.

Making essential information easier to use can bring about many different kinds of value – not just those that can be directly measured in pounds and pence. For example, greater accountability, innovative digital public services for citizens, and new forms of civic participation, journalism and campaigning. The impact of open data isn’t a magical rapid increase in jobs and economic value, but in many cases will be over a longer term, and will include non-monetary gains such as fairness and equality.

Next month, the US’s National Day of Civic Hacking plans to mobilise over 5,000 developers to address a variety of different challenges. The Data Journalism Handbook is full of examples of how journalists are using data to improve the news.

The UK has been a world leader in using data to benefit society – from projects like mySociety‘s TheyWorkForYou, which enables people to track what their MPs say in parliament, to pioneering data driven reportage from the likes of the Guardian. We hope the review also recognises this, and recommends that the UK does more to support it.

If you’re interested in open data and you’d like to join our global community of open government data advocates, you can join our open-government mailing list:

Comments Off

Avatar of Jonathan Gray

by Jonathan Gray

We need open carbon emissions data now!

May 13, 2013 in Access to Information, Campaigning, Featured, Featured Project, Open Data, Policy, WG Sustainability, Working Groups

Last week the average concentration of carbon dioxide in the atmosphere reached 400 parts per million, a level which is said to be unprecedented in human history.

Leading scientists and policy makers say that we should be aiming for no more than 350 parts per million to avoid catastrophic runaway climate change.

But what’s in a number? Why is the increase from 399 to 400 significant?

While the actual change is mainly symbolic (and some commentators have questioned whether we’re hovering above or just below 400), the real story is that we are badly failing to cut emissions fast enough.

Given the importance of this number, which represents humanity’s progress towards tackling one of the biggest challenges we currently face – the fact that it has been making the news around the world is very welcome indeed.

Why don’t we hear about the levels of carbon dioxide in the atmosphere from politicians or the press more often? While there are regularly headlines about inflation, interest and unemployment, numbers about carbon emissions rarely receive the level of attention that they deserve.

We want this to change. And we think that having more timely and more detailed information about carbon emissions is essential if we are to keep up pressure on the world’s governments and companies to make the cuts that the world needs.

As our Advisory Board member Hans Rosling puts it, carbon emissions should be on the world’s dashboard.

Over the coming months we are going to be planning and undertaking activities to advocate for the release of more timely and granular carbon emissions data. We are also going to be working with our global network to catalyse projects which use it to communicate the state of the world’s carbon emissions to the public.

If you’d like to join us, you can follow #OpenCO2 on Twitter or sign up to our open-sustainability mailing list:

Image credit: Match smoke by AMagill on Flickr. Released under Creative Commons Attribution license.

5 Comments »

Avatar of Mark Wainwright

by Mark Wainwright

Announcing CKAN 2.0

May 10, 2013 in CKAN, Featured, Featured Project, News, OKF Projects, Open Data, Open Government Data, Releases, Technical

CKAN is a powerful, open source, open data management platform, used by governments and organizations around the world to make large collections of data accessible, including the UK and US government open data portals.

Today we are very happy and excited to announce the final release of CKAN 2.0. This is the most significant piece of CKAN news since the project began, and represents months of hectic work by the team and other contributors since before the release of version 1.8 last October, and of the 2.0 beta in February. Thank you to the many CKAN users for your patience – we think you’ll agree it’s been worth the wait.

[Screenshot: Front page]

CKAN 2.0 is a significant improvement on 1.x versions for data users, programmers, and publishers. Enormous thanks are due to the many users, data publishers, and others in the data community, who have submitted comments, code contributions and bug reports, and helped to get CKAN to where it is. Thanks also to OKF clients who have supported bespoke work in various areas that has become part of the core code. These include data.gov, the US government open data portal, which will be re-launched using CKAN 2.0 in a few weeks. Let’s look at the main changes in version 2.0. If you are in a hurry to see it in action, head on over to demo.ckan.org, where you can try it out.

Summary

CKAN 2.0 introduces a new sleek default design, and easier theming to build custom sites. It has a completely redesigned authorisation system enabling different departments or bodies to control their own workflow. It has more built-in previews, and publishers can add custom previews for their favourite file types. News feeds and activity streams enable users to keep up with changes or new datasets in areas of interest. A new version of the API enables other applications to have full access to all the capabilities of CKAN. And there are many other smaller changes and bug fixes.

Design and theming

The first thing that previous CKAN users notice will be the greatly improved page design. For the first time, CKAN’s look and feel has been carefully designed from the ground up by experienced professionals in web and information design. This has affected not only the visual appearance but many aspects of the information architecture, from the ‘breadcrumb trail’ navigation on each page, to the appearance and position of buttons and links to make their function as transparent as possible.

[Screenshot: dataset page]

Under the surface, an even more radical change has affected how pages are themed in CKAN. Themes are implemented using templates, and the old templating system has been replaced with the newer and more flexible Jinja2. This makes it much easier for developers to theme their CKAN instance to fit in with the overall theme or branding of their web presence.

Authorisation and workflow: introducing CKAN ‘Organizations’

Another major change affects how users are authorised to create, publish and update datasets. In CKAN 1.x, authorisation was granted to individual users for each dataset. This could be augmented with a ‘publisher mode’ to provide group-level access to datasets. A greatly expanded version of this mode, called ‘Organizations’, is now the default system of authorisation in CKAN. This is much more in line with how most CKAN sites are actually used.

[Screenshot: Organizations page]

Organizations make it possible for individual departments, bodies, groups, etc, to publish their own data in CKAN, and to have control over their own publishing workflow. Different users can have different roles within an Organization, with different authorisations. Linked to this is the possibility for each dataset to have different statuses, reflecting their progress through the workflow, and to be public or private. In the default set-up, Organization user roles include Members (who can read the Organization’s private datsets), Editors (who can add, edit and publish datasets) and Admins (who can add and change roles for users).

More previews

In addition to the existing image previews and table, graph and map previews for spreadsheet data, CKAN 2.0 includes previews for PDF files (shown below), HTML (in an iframe), and JSON. Additionally there is a new plugin extension point that makes it possible to add custom previews for different data types, as described in this recent blog post.

[Screenshot: PDF preview]

News feeds and activity streams

CKAN 2.0 provides users with ways to see when new data or changes are made in areas that they are interested in. Users can ‘follow’ datasets, Organizations, or groups (curated collections of datasets). A user’s personalised dashboard includes a news feed showing activity from the followed items – new datasets, revised metadata and changes or additions to dataset resources. If there are entries in your news feed since you last read it, a small flag shows the number of new items, and you can opt to receive notifications of them via e-mail.

Each dataset, Organization etc also has an ‘activity stream’, enabling users to see a summary of its recent history.

[Screenshot: News feed]

Programming with CKAN: meet version 3 of the API

CKAN’s powerful application programming interface (API) makes it possible for other machines and programs to automatically read, search and update datasets. CKAN’s API was previously designed according to REST principles. RESTful APIs are deservedly popular as a way to expose a clean interface to certain views on a collection of data. However, for CKAN we felt it would be better to give applications full access to CKAN’s own internal machinery.

A new version of the API – version 3 – trialled in beta in CKAN 1.8, replaced the REST design with remote procedure calls, enabling applications or programmers to call the same procedures as CKAN’s own code uses to implement its user interface. Anything that is possible via the user interface, and a good deal more, is therefore possible through the API. This proved popular and stable, and so, with minor tweaks, it is now the recommended API. Old versions of the API will continue to be provided for backward compatibility.

Documentation, documentation, documentation

CKAN comes with installation and administration documentation which we try to keep complete and up-to-date. The major changes in the rest of CKAN have thus required a similarly concerted effort on the documentation. It’s great when we hear that others have implemented their own installation of CKAN, something that’s been increasing lately, and we hope to see even more of this. The docs have therefore been overhauled for 2.0. CKAN is a large and complex system to deploy and work on improving the docs continues: version 2.1 will be another step forward. Where people do run into problems, help remains available as usual on the community mailing lists.

… And more

There are many other minor changes and bug fixes in CKAN 2.0. For a full list, see the CKAN changelog.

Installing

To install your own CKAN, or to upgrade an existing installation, you can install it as a package on Ubuntu 12.04 or do a source installation. Full installation and configuration instructions are at docs.ckan.org.

Try it out

You can try out the main features at demo.ckan.org. Please let us know what you think!

Comments Off

Avatar of Rufus Pollock

by Rufus Pollock

Government Data Open and Machine Readable by Default Announces President Obama

May 10, 2013 in News, Open Data, Policy

There was big news for open data yesterday with a new Executive Order announced by President Obama. The order lays out the general principles that open, machine readable, data are the “new default”. (We note the Open Definition already includes machine readability in the definition of open data). There will a new Open Data Policy which will require U.S. government agencies to conform to standards “to collect or create information in a way that supports downstream information processing and dissemination activities”. Below, we summarize the key points.

Open By Default

The order reiterates some of the key reasons for openness and makes clear that open is the default:

To promote continued job growth, Government efficiency, and the social good that can be gained from opening Government data to the public, the default state of new and modernized Government information resources shall be open and machine readable. Government information shall be managed as an asset throughout its life cycle to promote interoperability and openness …

Development of an “Open Data Policy”

Sec. 2. Open Data Policy. (a) The Director of the Office of Management and Budget (OMB), in consultation with the Chief Information Officer (CIO), Chief Technology Officer (CTO), and Administrator of the Office of Information and Regulatory Affairs (OIRA), shall issue an Open Data Policy to advance the management of Government information as an asset

Clear set of best practices and tools

[Section 3] (a) Within 30 days of the issuance of the Open Data Policy, the CIO and CTO shall publish an open online repository of tools and best practices to assist agencies in integrating the Open Data Policy into their operations in furtherance of their missions. The CIO and CTO shall regularly update this online repository as needed to ensure it remains a resource to facilitate the adoption of open data practices.

Build the policy into procurement

(b) Within 90 days of the issuance of the Open Data Policy, the Administrator for Federal Procurement Policy, … to identify and initiate implementation of measures to support the integration of the Open Data Policy requirements into Federal acquisition and grant-making processes

By building open data release as a default into procurement this should greatly simplify open publication and remove arguments against release based on cost and complexity.

Assessment

Implementation will be tracked and assessed:

[Section 3] (c) Within 90 days of the date of this order, the Chief Performance Officer (CPO) shall work with the President’s Management Council to establish a Cross-Agency Priority (CAP) Goal to track implementation of the Open Data Policy.

In a follow-up we’ll be providing more detailed analysis of the full Open Data Policy memorandum including plans for the new data.gov and its use of CKAN.

Comments Off

Avatar of Martin Tisne

by Martin Tisne

Follow the Money, Follow the Data

May 3, 2013 in Ideas and musings, Open Data, Open Government Data, Open Spending

The following guest post from Martin Tisné was first published on his personal blog.

Money tunnel by RambergMediaImages, CC-BY-SA on Flickr

Some thoughts which I hope may be helpful in advance of the ‘follow the data‘ hack day this week-end:

The open data sector has quite successfully focused on socially-relevant information: fixing potholes a la http://www.fixmystreet.com/, adopting fire hydrants a la http://adoptahydrant.org/. My sense is that the next frontier will be to free the data that can enable citizens, NGOs and journalists to hold their governments to account. What this will likely mean is engaging in issues such as data on extractives’ transparency, government contracting, political finance, budgeting etc. So far, these are not the bread and butter of the open data movement (which isn’t to say there aren’t great initiatives like http://openspending.org/). But they should be:

At its heart, this agenda revolves around ‘following the money’. Without knowing the ‘total resource flow’:

  • Parents’ associations cannot question the lack of textbooks in their schools by interrogating the school’s budget
  • Healthcare groups cannot access data related to local spending on doctors, nurses
  • Great orgs such as Open Knowledge Foundation or BudgIT cannot get the data they need for their interpretative tools (e.g. budget tracking tool)
  • Investigative journalists cannot access the data they need to pursue a story

Our field has sought to ‘follow the money’ for over two decades, but in practice we still lack the fundamental ability to trace funding flows from A to Z, across the revenue chain. We should be able to get to what aid transparency experts call ‘traceability’ (the ability to trace aid funds from the donor down the project level) for all, or at least most fiscal flows.

Open data enables this to happen. This is exciting: it’s about enabling follow the money to happen at scale. Up until now, instances of ‘following the money’ have been the fruit of the hard work of investigative journalists, in isolated instances.

If we can ensure that data on revenues (extractives, aid, tax etc), expenditures (from planning to allocation to spending to auditing), and results (service delivery data) is timely, accessible, comparable and comprehensive, we will have gone a long way to helping ‘follow the money’ efforts reach the scale they deserve.

Follow the Money is a pretty tangible concept (if you disagree, please let me know!) – it helps demonstrate how government funds buy specific outcomes, and how/whether resources are siphoned away. We need to now make it a reality.

1 Comment »

Avatar of Laura James

by Laura James

Open Knowledge: much more than open data

May 1, 2013 in Featured, Ideas and musings, Join us, OKF, Open Data, Our Work

Book, Ball and Chain

We’ve often used “open knowledge” simply as a broad term to cover any kind of open data or content from statistics to sonnets, and more. However, there is another deeper, and far more important, reason why we are the “Open Knowledge” Foundation and not, for example, the “Open Data” Foundation. It’s because knowledge is something much more than data.

Open knowledge is what open data becomes when it’s useful, usable and used. At the Open Knowledge Foundation we believe in open knowledge: not just that data is open and can be freely used, but that it is made useful – accessible, understandable, meaningful, and able to help someone solve a real problem. —Open knowledge should be empowering – it should be enabling citizens and organizations understand the world, create insight and effect positive change.

It’s because open knowledge is much more than just raw data that we work both to have raw data and information opened up (by advocating and campaigning) and also by making, creating the tools to turn that raw material into knowledge that people can act upon. For example, we build technical tools, open source software to help people work with data, and we create handbooks which help people acquire the skills they need to do so. This combination, that we are both evangelists and makers, is extremely powerful in helping us change the world.

Achieving our vision of a world transformed through open knowledge, a world where a vibrant open knowledge commons empowers citizens and enables fair and sustainable societies, is a big challenge. We firmly believe it can done, with a global network of amazing people and organisations fighting for openness and making tools and more to support the open knowledge ecosystem, although it’s going to take a while!

We at the Open Knowledge Foundation are committed to this vision of a global movement building an open knowledge ecosystem, and we are here for the long term. We’d love you to join us in improving the world through open knowledge; there will be many different ways you can help coming up during the months ahead, so get started now by keeping in touch – by signing up to receive our Newsletter, or finding a local group or meetup near you.

1 Comment »

Please create an account to get started.

Sign up to the Open Knowledge Newsletter

Get Updates