Support Us

You are browsing the archive for Exemplars.

Libraries in Cologne open up bibliographic data!

Guest - March 15, 2010 in CKAN, Exemplars, External, OKF Projects, Open Data, Open Knowledge Foundation, WG Open Bibliographic Data

The following press release is reproduced with permission from Adrian Pohl and Felix Ostrowski, who are both at the North Rhine-Westphalian Library Service Center and who are both members of the Open Knowledge Foundation’s Working Group on Open Bibliographic Data – launched earlier this month. We’ve added a koeln-library-data package to the bibliographic data group on CKAN.

Cologne-based libraries and the Library Centre of Rhineland-Palatinate (LBZ) in cooperation with the North Rhine-Westphalian Library Service Center (hbz) are the first German libraries to adopt the idea of Open Access for bibliographic data by publishing their catalog data for free public use. The University and Public Library of Cologne (USB), the Library of the Academy of Media Arts Cologne, the University Library of the University of Applied Science of Cologne and the LBZ are taking the lead by releasing their data. The Public Library of Cologne has announced to follow shortly. The release of bibliographic data forms a basis for linking that data with data from other domains in the Semantic Web.

Libraries have been involved with the Open Access movement for a long time. The objective of this movement is to provide free access to knowledge to everybody via the internet. Until now, only few libraries have done so with their own data. Rolf Thiele, deputy director of the USB Cologne, states:

> Libraries appreciate the Open Access movement because they themselves feel obliged to provide access to knowledge without barriers. Providing this kind of access for bibliographic data, thus applying the idea of Open Access to their own products, has been disregarded until now. Up to this point, it was not possible to download library catalogues as a whole. This will now be possible. We are taking a first step towards a worldwide visibility of library holdings on the internet.

The library of the European Organization for Nuclear Research (CERN) has already published its data under a public domain license in January.

Public data is placed in the public domain The publication of the data enables anybody to download, modify and use it for any purpose. “In times in which publishers and some library organisations see data primarily as a source of capital, it is important to stick up for the traditional duty of libraries and librarians. Libraries have always strived to make large amounts of knowledge accessible to as many people as possible, with the lowest restrictions possible,” said Silke Schomburg, deputy director of the hbz. “Furthermore libraries are funded by the public. And what is publicly financed should be made available to the public without restrictions,” she continued.

Cooperation and data exchangie between libraries have been firmly established in the library world for more than 100 years. Freely supplying bibliographic data should not only further enhance cooperation among libraries but enable subsequent use by non-library institutions. “In the course of the internet’s development it became clear that many services can be greatly enhanced by catalog data. The German Wikipedia for example has been enriched with German National Library data for a long time. Such enrichment is often hindered and constricted by the data’s half open character,” Schomburg notes.

Data for the Semantic Web The North Rhine-Westphalian Library Service Center has recently begun evaluating the possibilities to transform data from library catalogs in such a way that it can become a part of the emerging Semantic Web. The liberalization of bibliographic data provides the legal background to perform this transformation in a cooperative, open, and transparent way. Currently there are discussions with other member libraries of the hbz library network to publish their data. Moreover, “Open Data” and “Semantic Web” are topics that are gaining perception in the international library world.

Further information and links to the published datasets are available at:

    *

Clear Climate Code, and Data

davidjones - January 28, 2010 in Exemplars, External, Open Data, Open Science

The following guest post is by David Jones who is, among other things, a curator of the climate data group on CKAN (the OKF’s open source registry of open data) and co-founder of Clear Climate Code (which we blogged about back in 2008).

Clear Climate Code have been working on ccc-gistemp, a project to reimplement in clear Python NASA’s GISTEMP. GISTEMP is a global historical temperature analysis, it produces, amongst other things, graphs like this, that tell you whether the Earth is getting warmer or cooler:

Official GISTEMP global anomaly.

Because this graph is important for studying the world’s climate (and determining the signature of global warming), there is a lot of public discussion about where this data comes from. The raw data underlying the graph is surface weather station temperature records. The raw data is processed to produce the data for the graph:

gistemp

The box in the middle, labelled “GISTEMP”, is a process that converts the raw station records into the data for the graph on the right, which is the global temperature anomaly. There are descriptions of this process available, for example Hansen and Lebedeff, 1987. A description is one thing, but it might not tell you everything you need to know. Perhaps the description is sufficiently clear and accurate for you to reproduce the process, perhaps not. The ultimate authority on the process is the source code that implements it, because It’s the source code that is executed in order to produce the processed data. So if you want to know exactly what the process involves, you have to get hold of the source code.

In effect it is the source code that adds value to the raw data to produce processed data. So in a sense, the value of the processed data is embodied in the source code. That’s what makes the source code important.

The source code for GISTEMP is written mostly in Fortran by scientists at NASA, and is available from them. This source code is the working code used by the NASA scientists, it is not necessarily the best source code for explaining how the process works (to an interested and competent member of the general public). There is the question of whether NASA, a publicly funded body, should be paying someone to write code that makes a better tool for communicating with the public (for example by writing better documentation, or writing it in a more exemplary style). I am not going to address that question. The source code NASA use is the source code we have right now.

Our goal at Clear Climate Code is to take this code and produce a new version that is clearer, but does the same thing. We have taken great steps forward towards this goal: We have recently released a version which is all in Python and which reproduces NASA’s results exactly. We think much of this code is already a great deal clearer than the starting material, but we continue to make it clearer. Of course we would welcome your support. If you want to help, please join our mailing list, or you can follow our progress at our blog and on twitter.

The reasons Clear Climate Code chose Python as the implementation language for ccc-gistemp are: accessibility, clarity, and familiarity. By accessible I mean that there is a large community of Python programmers, but also there are several tutorials and other materials for learning Python should you be motivated. Python is used to teach undergraduates programming. Python is relatively clear; it’s deliberately designed to be free of the clutter that imperils other programming languages. It’s certainly possible for people who are not professional programmers to create small programs in Python, and examine and modify existing Python programs. And lastly, it’s familiar; Nick Barnes and I already knew Python when we started the project. This seems like a trivial consideration, but in fact Clear Climate Code is an unpaid project and it’s pretty easy to come up with reasons to do something else instead, so the fact that we already knew Python was important.

Hopefully Clear Climate Code illustrates how both code and data are central to the public understanding of science. For an issue like global warming it is absolutely crucial that public are involved. CKAN’s climate data group is a place where non-specialists can access scientist’s data more easily, and hopefully use it to innovate, do their own hobby science, or create visualisations to better communicate with the public. I’m hoping to add more data sources to the climate data group in the near future, if you’re interested in adding more data to this group, please get in touch.

Data.gov.uk goes public – and its using CKAN!

Jonathan Gray - January 21, 2010 in CKAN, Exemplars, External, News, Open Data, Open Government Data, Open Knowledge Foundation, Releases

Data.gov.uk goes public today, and we’ve very proud that it is using CKAN, our open source registry of open data, to list official UK government datasets (as we announced in October):

  • We’ve been working closely with the Cabinet Office team to get this out the door, and over 2500 datasets have been released via the site!

In the Cabinet Office press release, Sir Tim Berners-Lee says:

> Making public data available for re-use is about increasing accountability and transparency and letting people create new, innovative ways of using it. Government data should be a public resource. By releasing it, we can unlock new ideas for delivering public services, help communities and society work better, and let talented entrepreneurs and engineers create new businesses and services.

The new launch has received lots of press coverage – even making the front page of the BBC news website! Below is a selection:

Data.gov.uk

Open Street Map community responds to Haiti crisis

Jonathan Gray - January 15, 2010 in Exemplars, External, Open Data, Open Geodata

There has recently been a flurry of activity in the Open Street Map community to improve maps of Haiti to assist humanitarian aid organisations responding to the recent earthquake.

In particular mappers and developers are scouring satellite images to identify collapsed and damaged buildings/bridges, spontaneous refugee camps, landslides, blocked roads and other damaged infrastructure – to help NGOs and international organisations respond more effectively to the crisis.

They have issued a call for assistance:

On January 12 2010, a 7.0 earthquake struck Port-au-Prince. The OpenStreetMap community can help the response by tracing Yahoo imagery and other data sources, and collecting existing data sets below. If you have connections with expat Haitian communities, consider getting in touch to work with them to enter place names, etc.

On Wednesday Mikel Maron wrote to the OSM talk list asking for help. Yesterday several companies authorised the OSM community to use their images.

There have been specific requests for up to date mapping information from humanitarian organisations on the ground. For example, on Wednesday, Nicolas Chavent of the Humanitarian OpenStreetMap Team wrote to the OSM talk list:

I am relaying a mapping requirement grounded in Haiti from GIS practitioners mapping there at the United Nations Office of Coordination of Humanitarian Affairs (UNOCHA): “NEED to map any spontaneous camps appearing in the imagery with size in area”

Recently generated data from Open Street Map has been used in maps by ITHACA (Information Technology for Humanitarian Assistance, Cooperation and Action) and the World Food Programme.

Yesterday evening Mikel Maron reported there had been over 400 edits since the earthquake. At the time of writing it looks like this has now more than doubled to over 800 edits since 12th January.

The following two images – before and after the earthquake – give you an impression of how much the OSM community have been doing!

haiti.osm.pre-event

haiti.osm.20090114180900

For more see:

The Open Knowledge Foundation is seeking an Editor for Open Text Book!

Jonathan Gray - October 31, 2009 in Exemplars, News, OKF Projects, Open Knowledge Foundation, Open Textbooks

We are seeking an Editor for Open Text Book, one of the highest ranked sites on the web for finding textbooks that you can freely use, reuse and redistribute:

This is a volunteer position requiring a one to two day a month commitment. If you are interested in contributing to the world of open education in general and open text books in particular just get in touch.

Open Text Book Editor

More Information

The Open Knowledge Foundation is looking for an Editor for its Open Text Book project. The project was launched in 2007 after Steve Coast of Open Street Map donated us the domain name. It aims to be a curated one stop shop for open textbooks – that is textbooks anyone is free to access, redistribute, reuse and build upon.

Recently there has been a sharp rise in interest in open textbooks. Earlier this month, a bill was proposed to make all Federally funded textbooks in the State of California available under an open license. Last year saw the start of a student led campaign to make textbooks open – which is currently supported by over 2000 college professors. There now are a plethora of open textbook projects around the world – at different educational levels, for a variety of different subjects. Its an exciting time for open textbooks!

The Open Text Book project aims to be, in the first instance, a simple registry to make it easy to locate open textbooks from many different sources. We have also begun to archive copies of some of the books in a repository. There is plenty of room for expanding the project in the future.

Open Text Book Editor

Responsibilities

We anticipate the Editor will spend one to two days a month on the project. This is a volunteer position and the Editor can be based anywhere in the world. The Editor will be responsible for:

  • Adding new textbooks to the registry on a monthly basis, and curating the repository of mirrored textbooks;
  • Checking the legal status of the textbooks to see that they are compliant with the Open Knowledge Definition;
  • Attending virtual meetings with the Working Group on Open Textbooks;
  • Giving input on the design of the Open Text Book website, and on the future of the Open Text Book project.
Open Text Book Editor

Get in touch!

If you are interested in the position, please get in touch, and let us know:

  • Your name, affiliation, and website (if you have one!)
  • Why you think you’d make a good Editor
  • Your ideas about the future of the Open Text Book project

If you know anyone who you think might be interested to hear about the position – please point them to this post! You can also help spread the word by microblogging the following Identi.ca and Twitter posts:

Open Text Book

OpenFlights data released under Open Database License (ODbL)

Jonathan Gray - October 14, 2009 in Exemplars, External, News, Open Data, Open Data Commons, Open Geodata

OpenFlights

OpenFlights is a site for “flight logging, mapping, stats and sharing”.

We’re very pleased to hear they’ve just released their data under the Open Database License (ODbL):

One of OpenFlights‘ most popular features is our dynamic airport and airline route mapping, and today, we’re proud to release the underlying data in an easy-to-use form, up to date for October 2009. Behold 56749 routes between 3310 airports on 669 airlines spanning the globe.

The data can be downloaded from our Data page and is free to use under the Open Database License.

See also the OpenFlights package on CKAN:

Fields of Gold: short film about open data on European farm subsidies

Jonathan Gray - September 25, 2009 in Exemplars, External, Open Data

Fields of Gold

Farm Subsidy have recently released a short film called Fields of Gold: Lifting the Veil on Europe’s Farm Subsidies.

The film tells the story of a campaign to open up data about where money from the Common Agricultural Policy goes – from national Freedom of Information requests from the likes of Jack Thurston and Nils Mulvad, to the construction of FarmSubsidy.org, a website which hosts cleaned up and aggregated European CAP data. It looks at the history of European farming policies, as well as news headlines resulting from the disclosure of where money goes – putting the data into context.

Some of it was shot at the European Open Data Summit (you can catch a glimpse of the European Open Data Inventory on CKAN at around 5:45!) – and there is an emphasis on the potential of new forms of collaboration between journalists and data analysts. As an example, it looks at an investigate report by the International Herald Tribune, which built on Farm Subsidy’s findings.

The film discusses the value and importance of making data open. Journalist Brigitte Alfter argues that the public have a right to know where public funds are spent. European policy analyst David Osimo talks about how making data open allows it to be aggregated, analysed and visualised by third parties – which can facilitate richer and more meaningful exploration. Finally the film alludes to Siim Kallas‘s broader drive towards transparency in European institutions, and talks about how Farm Subsidy paves the way for more open access to official European datasets.

Farm Subsidy

Where’s my nearest postbox? Open data for UK postbox locations

Jonathan Gray - August 28, 2009 in Exemplars, External, Open Data, Open Government Data

UK post box by Andrew Dunn

Where is your nearest postbox, and when is the post collected from it? Now you can get open data showing the locations and collection times of over 116,000 postboxes in the UK. You can browse relevant datasets on CKAN at:

The story behind this data reads like an inverted version of The Little Red Hen. Instead of nobody helping out, and nobody eating the bread except the Little Red Hen, numerous people have helped to request, reformat, clean up and add to this data – and now, as its open, anyone can re-use it!

Last year Tom Taylor made a Freedom of Information request using the What Do They Know? service (developed by the good folks at mySociety) resulting in the publication of PDF documents containing information about UK postboxes.

Edward Betts of the Open Library cleaned and re-published this data in Tab Separated Value (TSV) format:

Abi Broom and Peter Chamberlin made further FOI requests using the What Do They Know? service, resulting in the publication of more postbox locations and collection times:

Unfortunately the geographic information provided by the Royal Mail was not very detailed, so Matthew Somerville has developed a service to locate them more accurately:

Help locate unlocated postboxes – the Royal Mail supplied a list of every postbox’s location, but unfortunately, it did not have useful co-ordinates, only postcodes or sub-postcodes and some textual data. So I wrote this site: look up the postboxes near you by entering the first half of your postcode, locate one whose location you know on the map, pick which postbox you’ve located, and submit. The pages also include postbox last collection times, if we know them.

See the uk-locating-postboxes package:

If you live in the UK, you too can help improve the data by confirming the locations of your local postboxes!

Do you know of datasets containing the locations of postboxes in other countries? If so, please let us know by adding them to CKAN, or by leaving a comment below!

Opengov.se – a registry of open government data in Sweden

Jonathan Gray - August 25, 2009 in CKAN, Exemplars, External, Open Data, Open Government Data, Policy, WG EU Open Data

Opengov.se

Opengov.se is a registry of open government data in Sweden:

> Opengov.se is an initiative to highlight available public datasets in Sweden. It contains a commentable catalog of government datasets, their formats and usage restrictions.

It makes a note of what percentage of the datasets are fully open – i.e. material that is free for anyone to re-use and re-distribute without restriction, and that is in an open format. This is currently at 16%, or 5 out of a total of 30 datasets. The open datasets can be viewed at:

  • The site has received favourable press coverage in Sweden, and has been feature on Swedish national television and in a national newspaper.

Peter Kranz, who runs the site, has been contacted by both civil servants that want help with open data plans and politicians that want advice on how legislation should change to increase the amount of open government data.

He became interested in open government data, after starting to build eurlex.nu, a commentable semantic web version of European Union Legal Information. He was frustrated by the state of existing official websites, so decided to build a new version of the site – but found that he’d have to pay 10,000 Euros to access the data plus 3,000 EUR for each additional language.

As with other European countries, many agencies in Sweden charge for access to raw data. For example, SMHI, the Swedish Meteorological and Hydrological Institute, charges for access to weather data. As a result, people using weather information in their applications get data from the Norwegian authorities instead.

Peter decided to build Opengov.se after getting involved in the W3C EGovernment Interest Group and the US based Open Gov Data group – and realising that there weren’t any open goverment data initiatives in Sweden.

He is currently planning to scrape proposed bills and public consultations from the Swedish central government website – to create something like a Govtrack “light”.

We’ve added an se-opengov package to CKAN – and you can see the ‘country-sweden‘ for other Swedish open documents and datasets:

  • *

If you know of other open data from or about Sweden, please let us know by adding them to CKAN, or by adding them in a comment below!

Where is the nearest bus stop? UK Department for Transport adds NaPTAN data to Open Street Map

Jonathan Gray - August 20, 2009 in Exemplars, External, News, Open Data, Open Geodata, Open Government Data

Bus stop by Ti.mo on Flickr

The UK’s Department for Transport (DfT) has recently released data from the National Public Transport Access Node (NaPTAN) database to be put on Open Street Map (OSM).

As it says on the NaPTAN website:

> NaPTAN provides a unique identifier for every point of access to public transport in the UK, together with meaningful text descriptions of the stop point and its location.

The NaPTAN page on the Open Street Map wiki says the data contains:

> [...] details of some 350,000 public transport access points in Great Britain including bus stops, railway stations, tram stops and ferry terminals. This data includes a name, geocode, official code and other information useful to the project. The data set includes both the physical points of access to transport (Platforms, bus stops, airport gateways, etc), the interchanges (Stations, Airports, Ports, Clusters, etc ) and the the entrances to the interchanges from the street or public thoroughfare [...]

While the main NaPTAN database has restrictions on commercial use (see the naptan package page on CKAN, added at our Workshop on Public Information last autumn), under a special arrangement with the DfT and Traveline, the Open Street Map Foundation has been given access to the database to import useful and relevant data to Open Street Map to be made open under the terms of their license, the Creative Commons Attribution Sharealike license.

An email from Roger Slevin at the DfT earlier this year placates the concerns that Ordnance Survey may claim rights in the data:

> I am conscious that some concern has been expressed about whether OS has any rights to the NaPTAN data (or NPTG) – and I can assure the OSM community that the Department for Transport has been assured by Ordnance Survey that they do not claim any rights over NaPTAN location data – and it is a matter of record that Department for Transport is the owner of the NPTG database. Both NaPTAN and NPTG are maintained by DfT as national databases, collating data from all local transport authorities in England, Wales and Scotland.

Further details of the import agreement are available at:

  • The NaPTAN data is currently being converted to OSM format, imported by county and merged with OSM data. The first county, West Midlands, was uploaded at the end of March and data for Greater London was uploaded on Monday. It is planned to have data for the whole of the UK by the end of the year. This will mean that Open Street Map should have bus stops and other public transport points for the whole of Great Britain!

A full list of NaPTAN data added to OSM is available at:

  • We’ve added a package page to CKAN at:

  • *

If you are based in the UK and interested in helping out – you can check that the data in your local area is correct, as there are some ghost stops in the data, and duplicates where transport access points were previously added to OSM!

This is excellent news – and big kudos to the DfT for donating the data! We hope that other departments consider following suit and adding their geodata to OSM!

The import was supported by Ideas in Transit, which is “a five-year project that applies User Innovation to the transport challenges faced by individuals and society”. For more on their Open Street Map related activity, you can see the Ideas in Transit page on the OSM wiki.

Detail of OSM showing transport access points

Get Updates