Open Knowledge Foundation Blog » Open Definition

Update on Open Source Initiative’s adoption of the Open Knowledge Definition

Jonathan Gray — Wed, 04 Aug 2010 15:00:43 +0000

A few weeks back we blogged about Russ Nelson’s proposals for the Open Source Initiative (OSI) to adopt the Open Knowledge Definition, our standard for openness in relation to content and data.

Russ has written back to us with some notes and questions from a session on this at OSCON:

Okay, so, as promised, here is my report on the “Open Data Definition” BOF held on Wednesday, July 21, at 7PM. There were about ten people present, which is a reasonable attendance, particularly when set against the Google Android Hands-on session at which they gave out free Nexus One phones.

Didn’t seem wise to me to start from scratch, especially given the good work done by the Open Knowledge Foundation on their Open Knowledge Definition: http://www.opendefinition.org/okd/. So we read through it section by section, by way of review. Here are the questions we arrived at (thanks to Skud aka Kirrily Robert for taking notes):

What happens with data that’s not copyrightable? 1a. What about data that consists of facts about the world and thus even a collection of it cannot be copyrighted, but the exact file format can be copyrighted? Many sub-federal-level governments in the US have to publish facts on demand but claim a copyright on the formatting.

What about data that’s not accessible as a whole, but only through an API?

We’re thinking that OKD #9 should read “execution of an additional agreement” rather than “additional license”.

Does OKD #4 apply to works distributed in a particular file format? Is a movie not open data if it’s encoded in a patent-encumbered codec? Does it become open data if it’s re-encoded?

What constitutes onerous attribution in OKD #5? If you get open data from somebody, and they have an attribution page, is it sufficient for you to comply with the attribution requirement if you point to the attribution page?

This serves as an invitation to discuss these issues on the new list open-data@opensource.org . Send subscription requests to open-data-subscribe@opensource.org . Unsubscribe by sending a request to open-data-unsubscribe@opensource.org .

If these issues are successfully resolved, then this committee will recommend to the OSI board that the OKD should be adopted as OSI approved. If they can’t be resolved by, say, the end of 2010, then we will give up on trying. Either way, the intent is to lay down the list by the end of this year unless the participants desire otherwise.

So if you’d like to join the conversation, please join the list! We’ve also created an Etherpad to gather responses to some of these issues:

http://okfnpad.org/okd-questions

Share This

Belarusian translation of the Open Knowledge Definition (OKD)

Daniel Dietrich — Wed, 28 Jul 2010 16:01:23 +0000

We’ve just added a Belarusian translation of the Open Knowledge Definition thanks to Patricia Clausnitzer!

http://www.opendefinition.org/okd/bielaruskaja/

If you’d like to translate the Definition into another language, or if you’ve already done so, please get in touch on our discuss list, or on info at the OKF’s domain name (okfn dot org).

Share This

Should the Open Source Initiative adopt the Open Knowledge Definition?

Jonathan Gray — Mon, 19 Jul 2010 16:34:45 +0000

Russ Nelson, License Approval Chair at the Open Source Initiative (OSI), recently proposed a session at OSCON about OSI adopting a definition for open data:

I’m running a BOF at OSCON on Wednesday night July 21st at 7PM, with the declared purpose of adopting an Open Source Definition for Open Data. Safe enough to say that the OSD has been quite successful in laying out a set of criteria for what is, and what is not, Open Source. We should adopt a definition Open Data, even if it means merely endorsing an existing one. Will you join me there?

Subsequently a bunch of people wrote to Russell letting him know about the Open Knowledge Definition that we created a few years ago:

The Open Knowledge Definition (OKD) sets out principles to define ‘openness’ in knowledge – that’s any kind of content or data ‘from sonnets to statistics, genes to geodata’. The definition can be summed up in the statement that “A piece of knowledge is open if you are free to use, reuse, and redistribute it — subject only, at most, to the requirement to attribute and share-alike.”

Russell suggested there was scope for the OSI to adopt the OKD, and emailed us a further blurb for the event:

Should the Open Source Initiative write its own definition of Open Data? Or is the Open Knowledge Foundation’s definition up to snuff? Come help us decide at OSCON next week. We have a BOF scheduled at 19:00 on 21 July 2010. We’ll present the results of our decision to the OSI for adoption at its next board meeting.

We’re excited at the prospect that the OKD might get adopted as an official open data definition by OSI, and would love to hear from folks who plan to attend the session!

Share This

Why Share-Alike Licenses are Open but Non-Commercial Ones Aren’t

Rufus Pollock — Thu, 24 Jun 2010 08:43:44 +0000

It is sometimes suggested that there isn’t a real difference in terms of “openness” between share-alike (SA) and non-commercial (NC) clauses — both being some restriction on what the user of that material can do, and, as such, a step away from openness.

This is not true. A meaningful distinction can be drawn between share-alike and non-commercial clauses (or any other clause that discriminates against a particular type of person or field of endeavour), with the former being “open” and the latter being not “open”.

This distinction is important. It has relevance, for example, as to why Open Data Commons should not provide NC licenses but will provide a share-alike one. As well as to Creative Commons whose set of licenses includes both share-alike and non-commercial options. As such, not all CC licenses are open and CC licenses are are not all mutually compatible. This is something of an irony as it means that Creative Commons provide a set of licenses that don’t, in fact, result in a commons.

What’s the Problem? Why Does This Matter?

What’s the problem with NC licenses, aren’t “SA” licenses a step away from open too? And if we debate this, don’t we just end up having a pointless license holy war?

The distinction between NC and SA licenses isn’t about “holy war” but something very practical: license compatibility and the integrity of the “open” commons. The core of a “commons” of data (or code) is that one piece of “open” material contained therein can be freely intermixed with other “open” material.

This interoperability is absolutely key to realizing the main practical benefits of “openness” which is the ease of use and reuse — which, in turn, mean more and better stuff getting created and used.

The Open Knowledge/Data Definition functions as a “standard” to ensure interoperability just in the same way as normal tech standards operate (but in this case for licenses rather than for a piece of hardware or software). The aim is to ensure that any license which complies with the definition will be interoperable with any other such license meaning that data or content under the one license can be combined with data or content under the other license.

Share-alike or attribution requirements are allowed within the definition precisely because they do not break this interoperability (and may even help promote the commons by ensuring material is “shared back”). Non-commercial provisions are not permitted because they fundamentally break the commons, not only through being incompatible with other licenses but because they overtly discriminate against particular types of users. (I should emphasize here that the definition is directly following the line set out in the original open source definition …)

Thus, there is a meaningful distinction between attribution and share-alike requirements and other such as non-commercial (NC), and it is a distinction that merits the description of share-alike licenses as being open but non-commercial licenses as not being open.

Isn’t It Just About Degree?

Yes, NC and especially ND are more restrictive, but stating that NC licenses aren’t open is wrong - they’re just not as open.

This is incorrect.

To reiterate: it is a mistake to view the set of licenses as some continuous spectrum of ‘openness’ with PD at one end and full rights reserved at the other — with the implication that all licenses in between are more or less open.

There are significant discontinuities and in particular we can meaningfully partition the set of licenses into open and not-open based on a) their interoperability b) the freedom they provide to all persons (and companies) to use, reuse and redistribute.

But You Can’t Trademark Openness …

it’s annoying that someone claims to be releasing data openly, but it turns out to be NC and no-compete and a bunch of other stuff. It would be nice to say to them - “you can’t claim to be open because you don’t meet this definition”. But unfortunately it would probably be difficult to get the trademark on the word “open”

It’s quite right that you can’t trademark openness — and no-one should want to! However, we can make an effort as a community to have a clear shared meaning for “open” in relation to data and content along the lines of http://opendefinition.org/ — just as the open source definition has done for code. By insisting on this meaning we are doing something valuable: creating a standard and maintaining interoperability.

Share This

Russian translation of the Open Knowledge Definition (OKD)

Lisa Evans — Tue, 27 Apr 2010 17:24:23 +0000

We’ve just added a Russian translation of the Open Knowledge Definition thanks to Maxim Dubinin.

http://www.opendefinition.org/okd/russkiy

If you’d like to translate the Definition into another language, or if you’ve already done so, please get in touch on our discuss list, or on info at the OKF’s domain name (okfn dot org).

Share This

Norwegian translation of the Open Knowledge Definition (OKD)

Lisa Evans — Thu, 22 Apr 2010 19:29:06 +0000

We are pleased to now have a Norwegian translation of the Open Knowledge Definition thanks to Svein-Magnus Sørensen, Harald Groven and Olav Anders Øvrebø.

http://www.opendefinition.org/okd/norsk_bokmaal/

If you’d like to translate the Definition into another language, or if you’ve already done so, please get in touch on our discuss list, or on info at the OKF’s domain name (okfn dot org).

Share This

Chinese translation of the Open Knowledge Definition (OKD)

Lisa Evans — Thu, 08 Apr 2010 18:59:29 +0000

We’ve just added a Chinese translation of the Open Knowledge Definition thanks to Mao, Ching-Chen at Fu Jen Catholic University, Taiwan.

http://www.opendefinition.org/okd/chinese

If you’d like to translate the Definition into another language, or if you’ve already done so, please get in touch on our discuss list, or on info at the OKF’s domain name (okfn dot org).

Share This

The cake test of freedom

jwalsh — Mon, 15 Mar 2010 18:51:56 +0000

At last week’s Jornadas SIG Libre in Girona, Ivan Sanchez of the Spanish OpenStreetmap community told me about the cake test of data freedom.

What is the cake test? Easy: geographic data, or a map, is open only if someone can make you a gift of a cake with your map on it.

The cake test is inspired by the dissident test and the desert island test used by the Debian community to gauge software freedom for packages to be included in a free and open distribution.

For data to pass the cake test, you must be able to freely share the data with someone (the baker) who can re-use it for a profitable activity (the baking of cakes) and is then freely able to redistribute the resulting derived work (the cake).

The cake test can apply to all kinds of information resources, not just geodata. A resource that passes the cake test will be open in the sense of the Open Knowlege Definition. You could print a research paper onto a cake, a chart based on a dataset, some code describing an algorithm. Obviously a map just looks prettier on a cake.

The objective of the Cake Test is quite simple:

If a layperson can’t decide if one can or cannot give away a cake, or cannot do this easily, then the data or the maps cannot be freely used.

And you could be sure that if two datasets each passed the cake test, then it should be fine to give someone a cake decorated with parts of both of them - that is the intention of the data makers.

Is it open data? Does the data pass the cake test?

Share This

Open Knowledge Web Buttons: Get Them Now

UK Government announces lots of new open data!

Jonathan Gray — Mon, 07 Dec 2009 13:22:08 +0000

This morning UK Prime Minister Gordon Brown announced plans to open up lots more UK Government data! His speech describes plans to put much more detailed information online under open licenses in 2010.

This includes:

public services performance data - including on crime, hospitals and schools
new transport data
geospatial data from Ordnance survey (as we recently blogged about)

We are very pleased that it looks like the new datasets will be:

Released in raw form - as OKF Director Rufus Pollock first blogged about two year ago last month, and alluded to by Sir Tim Berners-Lee at TED.
Released in a way which is compliant with the Open Knowledge Definition - i.e. free for anyone to use for any purpose, include commercial.

We’re also very proud that the new data.gov.uk site, the official registry of UK Government open datasets, is powered by CKAN (as we announced a couple of months ago). If you’re interested in following the latest development about this as they happen, please join the official mailing list.

The new Putting the Frontline First: Smarter Government initiative gives further detail on how the new data will be published. In particular, section 1.3. Radically opening up data and promoting transparency, gives a set of “public data principles”, which are as follows:

‘Public data’ are ‘government-held non-personal data that are collected or generated in the course of public service delivery’.

Our public data principles state that:

Public data will be published in reusable, machine-readable form

Public data will be available and easy to find through a single easy to use online access point (http://www.data.gov.uk/)

Public data will be published using open standards and following the recommendations of the World Wide Web Consortium

Any ‘raw’ dataset will be represented in linked data form

More public data will be released under an open licence which enables free reuse, including commercial reuse

Data underlying the Government’s own websites will be published in reusable form for others to use

Personal, classified, commercially sensitive and third-party data will continue to be protected.

This is fantastic news - and we’ve highlighted key parts of the Prime Minister’s speech below:

Information is the key. An informed citizen is a powerful citizen.

[...] We are determined to be among the first governments in the world to open up public information in a way that is far more accessible to the general public.

So I am grateful to Sir Tim Berners-Lee and Professor Nigel Shadbolt for leading a project to ‘make public data public’.

This has enormous potential. Already more than 1,000 active users of the internet have registered their interest in working with government on this, and we have so far made around 1,100 datasets accessible to them.

And there are many hundreds more that can be opened up - not only from central government but also from local councils, the NHS, police and education authorities.

[...] In this way people will no longer be passive recipients of services but, through dialogue and engagement, active participants - shaping, controlling and determining what is best for them.

And I can announce today that we will actively publish all public services performance data online during 2010 completing the process by 2011. Crime data, hospital costs and parts of the national pupil database will go on line in 2010. We will use this data to benchmark the best and the worst and drive better value for money.

It will have a direct effect on how we allocate resources. We will introduce next year NHS tariffs based on best practice on the ground not average price. And we will be benchmarking the whole of the prison and probation system by 2011.

And we will give our frontline services greater freedoms and flexibilities to respond innovatively to this data, reducing the number of ring fenced budgets, rationalising different central funding projects and joining-up capital funding within a local area.

Releasing data can and must unleash the innovation and entrepreneurship at which Britain excels - one of the most powerful forces of change we can harness.

When, for example, figures on London’s most dangerous roads for cyclists were published, an online map detailing where accidents happened was produced almost immediately to help cyclists avoid blackspots and reduce the numbers injured.

And after data on dentists went live, an iphone application was created to show people where the nearest surgery was to their current location.

And from April next year ordnance survey will open up information about administrative boundaries, postcode areas and mid-scale mapping.

All of this will be available for free commercial re-use, enabling people for the first time to take the material and easily turn it into applications, like fix my street or the postcode paper.

And I can further announce today that, again from next April, we will also release public transport data hitherto inaccessible or expensive and release significant underlying data for weather forecasts for free download and re-use.

We are currently working on a new project which will map open government data initiatives from around the world. We are also working on a guidance document for opening up government data, and starting a new working group on open government data to promote technical and legal standards, as well as to help document what open government data is out there. If you’re interested in any of this, we’d love to hear from you!

Share This

Open data on cities: an international round up

Jonathan Gray — Tue, 27 Oct 2009 10:30:52 +0000

Over the last few months there have been lots of exciting announcements about open data from cities around the world. We decided to take a look at what is currently out there - in particular taking note of:

Whether datasets are open as in the Open Knowledge Definition - i.e. whether they explicitly say that they can be used by anyone, for any purpose, without restriction (except perhaps attribution, integrity or sharealike requirements).
Whether or not there are facilities to download raw data in bulk - i.e. whether they easily allow users to directly download all the data in open, machine readable formats.

We’ve now got 16 packages with the city tag on CKAN, our open-source registry of open data:

http://ckan.net/tag/read/city

United States

Boston

Background: Boston’s Datahub was mentioned in this post on open data in different cities from October 2009.
Open?: There is no explicit notice saying the data is open.
Bulk download?: No bulk download available.
More information:
- http://www.ckan.net/package/city-boston-gis-datahub
- http://www.ckan.net/package/metroboston-datacommon

Chicago

Background: Mentioned in this blog post from May 2009, and this blog post from October 2009.
Open?: There is no explicit notice saying the data is open.
Bulk download?: Boundary data is available for download in bulk.
More information:
- http://ckan.net/package/city-chicago

New Orleans

Background: Mentioned in this post from May 2009.
Open?: No re-distribution allowed.
Bulk download?: No bulk download available.
More information:
- http://ckan.net/package/city-neworleans

New York

Background: Nyc.gov/data was reported in New York Times earlier this month.
Open?: Though not explicit, terms imply that data can be re-used.
Bulk download?: Data available for download on a single page.
More information:
- http://ckan.net/package/city-nyc-gov-data

Portland

Background: Announced on this post and reported in ReadWriteWeb in September 2009. At time of writing no open data appears to be published yet.
Open?: Not yet published.
Bulk download?: Not yet published.
More information:
- http://ckan.net/package/city-portland

San Francisco

Background: Reported in Guardian in October.
Open?: No explicit license or terms of use stating material can be reused.
Bulk download?: Direct links to datasets given on HTML pages.
More information:
- http://ckan.net/package/datasf

Washington, D.C.

Background: One of the earliest and best examples of publishing local government data online - publicised by Vivek Kundra, who went on to work on data.gov.
Open?: Users must notify the OCTO and redistribute a disclaimer.
Bulk download?: All datasets are linked to from main page.
More information:
- http://ckan.net/package/dc-octo

Canada

Calgary

Background: A draft motion to make the City of Calgary’s data open was reported in July 2009. At time of writing no open data appears to be published yet.
Open?: Not yet published.
Bulk download?: Not yet published.
More information:
- http://ckan.net/package/city-calgary

Nanaimo

Background: Reported by datalibre.ca in July 2009.
Open?: No explicit statement that data is open. Legal disclaimer states that permission must be sought.
Bulk download?: Datasets all linked to from main page.
More information:
- http://ckan.net/package/city-nanaimo

Toronto

Background: VisibleGovernment.ca report in April 2009 that toronto.ca/open is announced announced at Mesh 2009. ~~At time of writing no open data appears to be published yet.~~ On November 2 the open data catalogue was launched at http://www.toronto.ca/open/catalogue.htm
Open?: Bespoke terms and conditions compliant with Open Knowledge Definition
Bulk download?: Yes. See catalogue page.
More information:
- http://ckan.net/package/city-toronto

Vancouver

Background: Reported on ReadWriteWeb in May 2009.
Open?: Data is explicitly open.
Bulk download?: Data is available for download in bulk.
More information:
- http://ckan.net/package/vancouver-open-data-catalogue

UK

Birmingham

Background: Digital Birmingham announce their ‘Open City’ initiative to increase access to public datasets in April 2009. They host an event in August 2009, reported here. At time of writing no open data appears to be published yet.
Open?: Not yet published.
Bulk download?: Not yet published.
More information:
- http://ckan.net/package/city-birmingham

Lichfield

Background: Reported in the Guardian in October 2009.
Open?: Datasets cannot be used for commercial purposes.
Bulk download?: All datasets are available for download on a single page.
More information:
- http://ckan.net/package/city-lichfield

London

Background: Initiative to open up the City of London’s data was reported in the Guardian in October 2009. At time of writing some datasets are published.
Open?: . Currently no permission is granted to reuse data.
Bulk download?: Datasets cannot be downloaded in bulk.
More information:
- http://ckan.net/package/city-london

How to open up city data

There are some excellent examples of publishing open data on cities - in particular New York, Washington and Vancouver. However not all data is explicitly open, or made available in bulk. Below is our recipe for opening up city data:

Use a license or legal tool to make datasets legally open! - If you are using your own custom copyright notice, license, disclaimer or terms and conditions, make sure they are compliant with the Open Knowledge Definition. You can also use existing licenses and legal tools, such as:
- the PDDL, the ODbL, or CC0 for data
- and CC-BY or CC-BY-SA for content
Make the raw data available in bulk! - Publish data in open, machine readable formats in a way which makes it easy for users to automatically download it. This could mean directly linking to all files in a single HTML page, or putting files in a single publicly accessible directory. Don’t make it difficult for users to download material by only allowing access to data via a shiny interface. Keep it simple!

Get involved!

Does your city publish open data? Do any of the details above need to be amended or added to? If you would like to get involved we encourage you to:

add more open datasets on cities to CKAN, using the tag ‘city‘ tag, or let us know about them in the comments below…
update the packages for the datasets listed above - just click edit on the relevant CKAN pages!
use the Is it Open? service to request whether or not datasets in your city are legally open

Share This