Support Us
Open Knowledge Foundation Open Knowledge
Foundation Blog

You are browsing the archive for Technical.

Avatar of Mark Wainwright

by Mark Wainwright

Announcing CKAN 2.0

May 10, 2013 in CKAN, Featured, Featured Project, News, OKF Projects, Open Data, Open Government Data, Releases, Technical

CKAN is a powerful, open source, open data management platform, used by governments and organizations around the world to make large collections of data accessible, including the UK and US government open data portals.

Today we are very happy and excited to announce the final release of CKAN 2.0. This is the most significant piece of CKAN news since the project began, and represents months of hectic work by the team and other contributors since before the release of version 1.8 last October, and of the 2.0 beta in February. Thank you to the many CKAN users for your patience – we think you’ll agree it’s been worth the wait.

[Screenshot: Front page]

CKAN 2.0 is a significant improvement on 1.x versions for data users, programmers, and publishers. Enormous thanks are due to the many users, data publishers, and others in the data community, who have submitted comments, code contributions and bug reports, and helped to get CKAN to where it is. Thanks also to OKF clients who have supported bespoke work in various areas that has become part of the core code. These include data.gov, the US government open data portal, which will be re-launched using CKAN 2.0 in a few weeks. Let’s look at the main changes in version 2.0. If you are in a hurry to see it in action, head on over to demo.ckan.org, where you can try it out.

Summary

CKAN 2.0 introduces a new sleek default design, and easier theming to build custom sites. It has a completely redesigned authorisation system enabling different departments or bodies to control their own workflow. It has more built-in previews, and publishers can add custom previews for their favourite file types. News feeds and activity streams enable users to keep up with changes or new datasets in areas of interest. A new version of the API enables other applications to have full access to all the capabilities of CKAN. And there are many other smaller changes and bug fixes.

Design and theming

The first thing that previous CKAN users notice will be the greatly improved page design. For the first time, CKAN’s look and feel has been carefully designed from the ground up by experienced professionals in web and information design. This has affected not only the visual appearance but many aspects of the information architecture, from the ‘breadcrumb trail’ navigation on each page, to the appearance and position of buttons and links to make their function as transparent as possible.

[Screenshot: dataset page]

Under the surface, an even more radical change has affected how pages are themed in CKAN. Themes are implemented using templates, and the old templating system has been replaced with the newer and more flexible Jinja2. This makes it much easier for developers to theme their CKAN instance to fit in with the overall theme or branding of their web presence.

Authorisation and workflow: introducing CKAN ‘Organizations’

Another major change affects how users are authorised to create, publish and update datasets. In CKAN 1.x, authorisation was granted to individual users for each dataset. This could be augmented with a ‘publisher mode’ to provide group-level access to datasets. A greatly expanded version of this mode, called ‘Organizations’, is now the default system of authorisation in CKAN. This is much more in line with how most CKAN sites are actually used.

[Screenshot: Organizations page]

Organizations make it possible for individual departments, bodies, groups, etc, to publish their own data in CKAN, and to have control over their own publishing workflow. Different users can have different roles within an Organization, with different authorisations. Linked to this is the possibility for each dataset to have different statuses, reflecting their progress through the workflow, and to be public or private. In the default set-up, Organization user roles include Members (who can read the Organization’s private datsets), Editors (who can add, edit and publish datasets) and Admins (who can add and change roles for users).

More previews

In addition to the existing image previews and table, graph and map previews for spreadsheet data, CKAN 2.0 includes previews for PDF files (shown below), HTML (in an iframe), and JSON. Additionally there is a new plugin extension point that makes it possible to add custom previews for different data types, as described in this recent blog post.

[Screenshot: PDF preview]

News feeds and activity streams

CKAN 2.0 provides users with ways to see when new data or changes are made in areas that they are interested in. Users can ‘follow’ datasets, Organizations, or groups (curated collections of datasets). A user’s personalised dashboard includes a news feed showing activity from the followed items – new datasets, revised metadata and changes or additions to dataset resources. If there are entries in your news feed since you last read it, a small flag shows the number of new items, and you can opt to receive notifications of them via e-mail.

Each dataset, Organization etc also has an ‘activity stream’, enabling users to see a summary of its recent history.

[Screenshot: News feed]

Programming with CKAN: meet version 3 of the API

CKAN’s powerful application programming interface (API) makes it possible for other machines and programs to automatically read, search and update datasets. CKAN’s API was previously designed according to REST principles. RESTful APIs are deservedly popular as a way to expose a clean interface to certain views on a collection of data. However, for CKAN we felt it would be better to give applications full access to CKAN’s own internal machinery.

A new version of the API – version 3 – trialled in beta in CKAN 1.8, replaced the REST design with remote procedure calls, enabling applications or programmers to call the same procedures as CKAN’s own code uses to implement its user interface. Anything that is possible via the user interface, and a good deal more, is therefore possible through the API. This proved popular and stable, and so, with minor tweaks, it is now the recommended API. Old versions of the API will continue to be provided for backward compatibility.

Documentation, documentation, documentation

CKAN comes with installation and administration documentation which we try to keep complete and up-to-date. The major changes in the rest of CKAN have thus required a similarly concerted effort on the documentation. It’s great when we hear that others have implemented their own installation of CKAN, something that’s been increasing lately, and we hope to see even more of this. The docs have therefore been overhauled for 2.0. CKAN is a large and complex system to deploy and work on improving the docs continues: version 2.1 will be another step forward. Where people do run into problems, help remains available as usual on the community mailing lists.

… And more

There are many other minor changes and bug fixes in CKAN 2.0. For a full list, see the CKAN changelog.

Installing

To install your own CKAN, or to upgrade an existing installation, you can install it as a package on Ubuntu 12.04 or do a source installation. Full installation and configuration instructions are at docs.ckan.org.

Try it out

You can try out the main features at demo.ckan.org. Please let us know what you think!

No Comments »

Avatar of Rufus Pollock

by Rufus Pollock

Frictionless Data: making it radically easier to get stuff done with data

April 24, 2013 in Featured, Ideas and musings, Labs, Open Data, Open Standards, Small Data, Technical

Frictionless Data is now in alpha at http://data.okfn.org/ – and we’d like you to get involved.

Our mission is to make it radically easier to make data used and useful – our immediate goal is make it as simple as possible to get the data you want into the tool of your choice.

This isn’t about building a big datastore or a data management system – it’s simply saving people from repeating all the same tasks of discovering a dataset, getting it into a format they can use, cleaning it up – all before they can do anything useful with it! If you’ve ever spent the first half of a hackday just tidying up tabular data and getting it ready to use, Frictionless Data is for you.

Our work is based on a few key principles:

  • Narrow focus — improve one small part of the data chain, standards and tools are limited in scope and size
  • Build for the web – use formats that are web “native” (JSON) and work naturally with HTTP (plain-text, CSV is streamable etc)
  • Distributed not centralised — designed for a distributed ecosystem (no centralized, single point of failure or dependence)
  • Work with existing tools — don’t expect people to come to you, make this work with their tools and their workflows (almost everyone in the world can open a CSV file, every language can handle CSV and JSON)
  • Simplicity (but sufficiency) — use the simplest formats possible and do the minimum in terms of metadata but be sufficient in terms of schemas and structure for tools to be effective

We believe that making it easy to get and use data and especially open data is central to creating a more connected digital data ecosystem and accelerating the creation of social and commercial value. This project is about reducing friction in getting, using and connecting data, making it radically easier to get data you need into the tool of your choice. Frictionless Data distills much of our learning over the last 7 years into some specific standards and infrastructure.

What’s the Problem?

Today, when you decide to cook, the ingredients are readily available at local supermarkets or even already in your kitchen. You don’t need to travel to a farm, collect eggs, mill the corn, cure the bacon etc – as you once would have done! Instead, thanks to standard systems of measurement, packaging, shipping (e.g. containerization) and payment, ingredients can get from the farm direct to my local shop or even my door.

But with data we’re still largely stuck at this early stage: every time you want to do an analysis or build an app you have to set off around the internet to dig up data, extract it, clean it and prepare it before you can even get it into your tool and begin your work proper.

What do we need to do for the working with data to be like cooking today – where you get to spend your time making the cake (creating insights) not preparing and collecting the ingredients (digging up and cleaning data)?

The answer: radical improvements in the “logistics” of data associated with specialisation and standardisation. In analogy with food we need standard systems of “measurement”, packaging, and transport so that its easy to get data from its original source into the application where you can start working with it.

Frictionless DAta idea

What’s Frictionless Data going to do?

We start with an advantage: unlike for physical goods transporting digital information from one computer to another is very cheap! This means the focus can be on standardizing and simplifying the process of getting data from one application to another (or one form to another). We propose work in 3 related areas:

  • Key simple standards. For example, a standardized “packaging” of data that makes it easy to transport and use (think of the “containerization” revolution in shipping)
  • Simple tooling and integration – you should be able to get data in these standard formats into or out of Excel, R, Hadoop or whatever tool you use
  • Bootstrapping the system with essential data – we need to get the ball rolling

frictionless data components diagram

What’s Frictionless Data today?

1. Data

We have some exemplar datasets which are useful for a lot of people – these are:

  • High Quality & Reliable

    • We have sourced, normalized and quality checked a set of key reference datasets such as country codes, currencies, GDP and population.

  • Standard Form & Bulk Access

    • All the datasets are provided in a standardized form and can be accessed in bulk as CSV together with a simple JSON schema.

  • Versioned & Packaged

    • All data is in data packages and is versioned using git so all changes are visible and data can becollaboratively maintained.

2. Standards

We have two simple data package formats, described as ultra-lightweight, RFC-style specifications. They build heavily on prior work. Simplicity and practicality were guiding design criteria.

Frictionless Data: package standard diagram

Data package: minimal wrapping, agnostic about the data its “packaging”, designed for extension. This flexibility is good as it can be used as a transport for pretty much any kind of data but it also limits integration and tooling. Read the full Data Package specification.

Simple data format (SDF): focuses on tabular data only and extends data package (data in simple data format is a data package) by requiring data to be “good” CSVs and the provision of a simple JSON-based schema to describe them (“JSON Table Schema”). Read the full Simple Data Format specification.

3. Tools

It’s early days for Frictionless Data, so we’re still working on this bit! But there’s a need for validators, schema generators, and all kinds of integration. You can help out – see below for details or check out the issues on github.

Doesn’t this already exist?

People have been working on data for a while – doesn’t something like this already exist? The crude answer is yes and no. People, including folks here at the Open Knowledge Foundation, have been working on this for quite some time, and there are already some parts of the solution out there. Furthermore, much of these ideas are directly borrowed from similar work in software. For example, the Data Packages spec (first version in 2007!) builds heavily on packaging projects and specifications like Debian and CommonJS.

Key distinguishing features of Frictionless Data:

  • Ultra-simplicity – we want to keep things as simple as they possibly can be. This includes formats (JSON and CSV) and a focus on end-user tool integration, so people can just get the data they want into the tool they want and move on to the real task
  • Web orientation – we want an approach that fits naturally with the web
  • Focus on integration with existing tools
  • Distributed and not tied to a given tool or project – this is not about creating a central data marketplace or similar setup. It’s about creating a basic framework that would enable anyone to publish and use datasets more easily and without going through a central broker.

Many of these are shared with (and derive from) other approaches but as a whole we believe this provides an especially powerful setup.

Get Involved

This is a community-run project coordinated by the Open Knowledge Foundation as part of Open Knowledge Foundation Labs. Please get involved:

  • Spread the word! Frictionless Data is a key part of the real data revolution – follow the debate on #SmallData and share our posts so more people can get involved

6 Comments »

Avatar of Pierre-Yves Vandenbussche

by Pierre-Yves Vandenbussche

Announcing: Linked Open Vocabularies (LOV), enabling the vocabulary commons

July 10, 2012 in Featured Project, Linked Open Data, News, Open Data, Our Work, Technical

We are delighted to announce that Linked Open Vocabularies is now being hosted on Open Knowledge Foundation servers and is now officially an Open Knowledge Foundation project.

LOV Project in 5 points

  • LOV is about vocabularies (aka. metadata element sets or ontologies) in OWL / RDFS used to describe linked data.
  • LOV provides a single-stop access to the Vocabulary Commons ecosystem
  • LOV helps to improve vocabularies understanding, visibility, usability, synergy, sustainability and overall quality
  • LOV promotes a technically and socially sustainable management of the Vocabulary Commons ecosystem
  • LOV is a community and open project. You are welcome to join the team of gardeners of the Vocabulary Commons!

Project context

The LOV project is borned in the framework of the Datalift project which aims at providing a platform to lift data from semi-structured formats (csv, xls, etc.) to linked data. Part of this project under Mondeca‘s company responsibility was focused on vocabulary selection and re-use. The LOV project purpose goes now far beyond this original catalogue. The LOV dataset is maintained by Bernard Vatant and Pierre-Yves Vandenbussche.

Project purposes

  • To identify vocabularies used or usable to express linked data in RDF
  • To harvest or create metadata and links between vocabularies
  • To suggest to vocabulary curators some vocabulary description improvements
  • To foster sustainable and responsible behavior of vocabulary creators and publishers
  • To provide advanced search features among vocabulary ecosystem elements

Project features

Among the various features of the LOV project, you can explore the vocabularies dataset using an intuitive UI. You can also access directly to an RDF dump via a file or an endpoint.

Lov 1

For every vocabulary, as much metadata as possible is harvested (gathered in the RDF file, in the documentation or via interaction with authors). For example, the links between a particular vocabulary and the ecosystem are shown as well as its different versions.

Lov 2

One may search for a particular vocabulary element using the LOV search feature, filtering results by domain, type, or vocabulary. This feature is enabled thanks to the LOV-bot which monitor all the vocabularies on a daily basis.

Lov 3

OKFN support and the future of Vocabulary Commons

Along with a sustainable and resilient future for vocabularies, we believe the LOV project should live far beyond the Datalift research project in which it is born. In that perspective, the Open Knowledge Foundation agreed to support our project for the future years. We are really delighted by this support, which strengthens our belief that heritage organizations will play a major role in vocabularies preservation.

LOV and Vocabulary Commons future belongs to its community. You are therefore, as an individual or organization, most welcome to participate in the future of LOV in many ways:

Comments Off

Avatar of Sam Leon

by Sam Leon

Introducing PyBossa – the open-source micro-tasking platform

June 8, 2012 in Featured, Featured Project, OKF Projects, Our Work, PyBossa, Technical, WG Open Data in Science, Working Groups

PyBossa Logo

For a while now our network has been working on applications, tools and platforms for crowd-sourcing and micro-tasking. At the end of last year, we posted about a cute little app developed at a hackday called the Data Digitizer that was being used to transcribe Brazillian budgetary data.

In recent months we’ve been working closely with the Citizen Cyberscience Center on an exciting new platform called PyBossa. In a nut-shell, PyBossa is a free, open-source crowd-sourcing and micro-tasking platform. It enables people to create and run projects that utilise online assistance in performing tasks that require human cognition such as image classification, transcription, geocoding and more. PyBossa is there to help researchers, civic hackers and developers to create projects where anyone around the world with some time, interest and an internet connection can contribute.

There is already a wealth of such projects, including long-running ones such as FreeBMD – a huge effort to transcribe the Civil Registration of births, marriages and deaths in the UK – as well as more recent ones such as GalaxyZoo – a hugely successful project based on volunteer efforts to classify photographs of galaxies taken by the Hubble telescope.

With PyBossa we want to make the creation of such potentially transformative projects as easy as possible and so PyBossa is different to existing efforts:

  • It’s a 100% open-source
  • Unlike, say, “mechanical turk” style projects, PyBossa is not designed to handle payment or money — it is designed to support volunteer-driven projects.
  • It’s designed as a platform and framework for developing deploying crowd-sourcing and microtasking apps rather than being a crowd-sourcing application itself. Individual crowd-sourcing apps are written as simple snippets of Javascript and HTML which are then deployed on a PyBossa instance (such as PyBossa.com). This way one can easily develop custom apps while using the PyBossa platform to store your data, manage users, and handle workflow.

You can read more about the architecture in the PyBossa Documentation and follow the step-by-step tutorial to create your own apps.

Demos

PyBossa currently comes with several demo applications that showcase two types of projects:

Flickr Person shows how easily you can create a project where you have a set of photos or figures that need a classification or a description of the photo. In this demo application, the latest 20 published public photos from Flickr are used as input for the volunteers where they will have to answer a simple question: Do you see a human in this photo?

PyBossa Baby

The demo project Melanoma comes from an idea conceived by a team at Sage Bionetworks. Melanoma is one of the most life-threatening forms of cancer and its incidence is on the rise. It is often difficult for medical professionals to determine if a skin lesion is cancerous or not, but if diagnosed early patients have a 95% chance of survival. Advances in computer-aided image manipulation have improved the diagnostic process, but the hope is that the combination of these techniques and crowd-sourcing will improve these techniques further making early diagnosis more common.

In the demo you are asked to say if a skin lesion shows signs of being cancerous, and are taken through the various key questions: is it asymmetrical?, are its borders blurred?, is its colour uneven? and is it bigger than 6mm in diameter?. The plan is to extend this demo into a project that will help citizens recognise the early signs of skin cancer and also enable scientists to evaluate the role of crowd-sourcing in medical diagnosis.

PyBossa Melanoma

Urban Parks is a rather different kind of project. It shows a web mapping tool where volunteers are asked to locate an urban park for a given city. The goal is to show how web mapping tools can be used to address tasks like geo-locating items in a map.

PyBossa Urban Parks

If you want to try the demos and PyBossa, go to PyBossa.com and get clicking. If you are interested in the framework you can download the source code from the Github repo and access the documentation here.

The Future

The focus on PyBossa has initially been on online citizen science projects, but it could have important applications in a host of other domains. For one, PyBossa could be used to help transcribe handwritten manuscripts of historical significance and contribute to existing efforts to make more of our shared cultural heritage available for free online and in a structured form.

We have no doubt that there are hundreds of other use-cases for PyBossa which we haven’t conceived of yet, and we’re looking forward to seeing the unexpected projects that emerge from it.

Call to action

Does PyBossa sound like something you’d like to get involved in? If so…

For any questions that you would like to address directly to the development team please use info [at] pybossa.com

1 Comment »

Avatar of Rufus Pollock

by Rufus Pollock

Introducing the DataStore

March 27, 2012 in CKAN, Our Work, Technical

A major new feature in the DataHub is good news for data wranglers. The DataStore allows users to store and load structured data into a database, where it can be queried, filtered, or accessed from other programs via a rich data API.

The API is also used by CKAN’s inbuilt Recline Data Explorer, giving in-page previews of the data with full text search, filtering, sorting and graphing, as in the screenshots below:

[IMG: Sorting] [IMG: Graph]

These new DataHub capabilities are powered by the recently enhanced DataStore and Data API functionality of our open-source CKAN data management system, which as well as powering the DataHub runs many other data portals including data.gov.uk.

An introduction to the DataStore and Data API



Comments Off

Avatar of Naomi Lillie

by Naomi Lillie

Open Biblio launches BibSoup in beta

February 13, 2012 in Bibliographic, Metadata, Our Work, Releases, Technical, WG Open Bibliographic Data

BibSoup is here! And it’s going to revolutionise how you work with bibliographic metadata.

bibsoup_screenshot

The team has been coding and blogging and bugfixing for a while now on the BibServer software, and we’ve mentioned in passing that our own instance has been up and running under the name of BibSoup… Now we are officially launching for beta fun, and asking the community to come and have a go and let us know what you think.

To get involved:

Setting up a Bibserver and Faceted Browsing (Mark MacGillivray) from Bibsoup Project on Vimeo.

Share your collection with the world

If you are already used to managing your bibliography in tools like Zotero, Bibsonomy, Mendeley – or even in text editors using Bibtex or RIS – then you already have a collection and a management strategy, and probably even a way to collaborate with other people on it. But then, what do you do with it? At BibSoup you can upload it and create an interactive web page to share with anyone.

Search and faceted browse

Once your collection is up on BibSoup, you can search across all the records and filter by any value in your collection. You can also customise the view of your collection to show the most useful filters by default, or perform advanced searches using powerful query syntax.

Graph the vital statistics

Want to know which year most of your publications were published? Which author is most common in your collection? Which journal publishes the most relevant articles? Just use the visualise option to create a great bubble or bar chart based on one of your filter values, and quickly find and click the one you want.

Improve your collection

You can use your BibServer display to quickly spot errors in your records; then you can export and make improvements. Soon you will also be able to edit directly in the BibServer, and share collection management with others.

Use your records on the internet

Once your collection is in BibServer, it is even possible to use the content directly as references within pages that you create on the web. There is a full RESTful API and every page returns JSON (just add .json or &format=json if have URL parameters).

Export as BibJSON

BibServer uses a particular flavour of JSON that we call BibJSON; JSON formats are really useful for using on the web, and can quite easily be exported and used for fun things like the visualisations we demonstrate.

Learn more

We are continuing to work hard on the code – making the records editable is our current focus, along with further development of the BibJSON conventions.

Upload from BibTex or RIS sources is available, which means you can import data from most major bibliographic tools already; you can even use the parsers programmatically if you like. We would be happy to talk to anyone interested in developing more parsers too.

bibserver_screenshot

The aim of this project is to show how Open Bibliography enables scholarship; to show our community what we are missing if we do not commit to Open Bibliography; and to show that Open Bibliography is a fundamental requirement of a community committed to discovery and dissemination of ideas.

This project is run by the Open Knowledge Foundation Open Bibliographic Working Group, with funding from JISC via the Open Biblio 2 project, and with Cambridge University Library as lead partners.

We are building on the large collections – and the goodwill of the providers of those collections – that we received last year, to demonstrate how the content of such collections can be useful to the individual or small group, by enabling them to identify and share records of value to them (earlier BibServer work was funded with partial support from U.S. National Science Foundation Award 0835773).

We hope you like BibSoup and find all this useful; do add your collections so we can share them with the rest of the world, too. Also if you would like your own BibServer, go ahead and download the code, or contact us for help / support options.

Tags: , 1 Comment »

Avatar of Theodora Middleton

by Theodora Middleton

Major new CKAN release: v1.5!

November 14, 2011 in CKAN, News, OKF Projects, Open Data, Technical

The following post is by David Read, on behalf of the CKAN team.

We’re proud to announce a major new release of CKAN!

Version 1.5 brings major improvements including:

  • Major user experience upgrades around dataset publication and access plus a new theme
  • Integrated structured and blob data storage, with associated with data previewing and visualization
  • Extended catalog API providing ability to access every piece of the CKAN system
  • Documentation overhaul and extension including a new administrator and development manual at http://docs.ckan.org/
  • Easier installation and deployment, specifically via new debian / ubuntu packages of CKAN — CKAN installation and deployment can now be less than 5 minutes

CKAN, the Open Knowledge Foundation’s data hub and catalogue software, has now been deployed in over 20 countries around the world, providing Open Data hubs for governments and communities. Started five years ago, CKAN has gradually increased in momentum – the development team is now at 6 full-time developers. Originally developed to power the community data site http://thedatahub.org/ (previously http://ckan.net) which can be freely used by anyone in the open data communities, the Foundation has also now been involved in assisting governments and other public organisations make use of CKAN through the development of customisations and new features as well as the provision of hosted solutions.

For more about this release, see: http://ckan.org/2011/11/09/ckan-1-5-release/

We’d also like to take this opportunity to thank the amazing on-line community around CKAN for their continued ideas, suggestions and support in the development of this open source data hub software.

Comments Off

Avatar of Jonathan Gray

by Jonathan Gray

What open data catalogs are there in your country?

October 14, 2011 in CKAN, Metadata, OKF Projects, Open Data, Open Government Data, Technical, WG EU Open Data, WG Open Government Data, Working Groups

The following post is from Jonathan Gray, Community Coordinator at the Open Knowledge Foundation.

Earlier this year at OKCon 2011 in Berlin we launched datacatalogs.org to make a comprehensive list of open data catalogs around the world.

We’ve just upgraded it to run on the latest version of CKAN, our open source data hub software, which powers data.gov.uk, the IATI Registry and numerous other data catalogs.

We’ve also just enabled groups, which means that it is possible to create curated lists of data catalogs for different purposes. For example, you can see lists of:

If you’re interested in curating lists of data catalogs in your country, or internationally, please sign up to our new data-catalogs mailing list and say hello!

1 Comment »

Avatar of Jonathan Gray

by Jonathan Gray

Release of DataCatalogs.org to map open data around the world

June 30, 2011 in CKAN, News, OKF Projects, Open Data, Open Government Data, Our Work, Press, Releases, Technical, WG EU Open Data, WG Open Government Data, Working Groups

The following post is from Jonathan Gray, Community Coordinator at the Open Knowledge Foundation.

We’re very pleased to announce an alpha version of datacatalogs.org, a website to help keep track of open data catalogues from around the world. The project is being launched to coincide with our annual conference, OKCon 2011. You can see the site here:

This is just the beginning of what we hope will become an invaluable resource for anyone interested in finding, using or having an overview of data catalogues from around the world. We have lots of ideas about improvements and features that we’d like to add. If you have anything you think we should prioritise, please let us know in comments below, or on the ckan-discuss list!

Below is a press release for the project (and here in Google Docs). If you know anyone who you think might be interested in this, we’d be most grateful for any help in passing it on!

> ### PRESS RELEASE: Mapping open data around the world

> BERLIN, 30th June 2011 – Today a broad coalition of stakeholders are launching DataCatalogs.org, a new project to keep track of open data initiatives around the world.

> Governments are beginning to recognise that opening up public information can bring about a wide variety of social and economic benefits – such as increasing transparency and efficiency, creating jobs in the new digital economy, and enabling web and mobile developers to create new useful applications and services for citizens.

> But it can be difficult to keep up with the pace of developments in this area. Following on from the success of initiatives like the Obama administration’s data.gov and the UK government’s data.gov.uk, nearly every week there is a new open data initiative from a local, regional or national government somewhere around the world – from Chicago to Torino, Morocco to Moldova.

> A group of leading open data experts are helping to keep DataCatalogs.org updated, including representatives from international bodies such as the World Bank, independent bodies such as the W3C and the Sunlight Foundation, and numerous national governments.

> Neil Fantom, Manager of the World Bank’s Development Data Group, says: “Open data is public good, but only if you can find it – we’re pleased to see initiatives such as DataCatalogs.org giving greater visibility to public information, allowing easier discovery of related content from different publishers and making open data more valuable for users.”

> Beth Noveck, who ran President Obama’s open government programme and is now working with the UK Government says: “This project is a simple but important start to bringing together the community of key open data stakeholders. My hope is that DataCatalogs.org grows into a vibrant place to articulate priorities, find and mash up data across jurisdictions and curate data-driven tools and initiatives that improve the effectiveness of government and the lives of citizens.”

> Cathrine Lippert, of the Danish National IT and Telecom Agency says: “DataCatalogs.org is a brilliant guide to keeping track of all the data that is being opened up around the world. In addition to our own national data catalogue, we can now point data re-users to DataCatalogs.org to locate data resources abroad.”

> Andrew Stott, former Director of Digital Engagement at the UK’s Cabinet Office says: “This initiative will not only help data users find data in different jurisdictions but also help those implementing data catalogues to find good practice to emulate elsewhere in the world.”

> ### Notes for editors

> The Open Knowledge Foundation (okfn.org) is a not-for-profit organisation founded in 2004. It has played a significant role in supporting open data around the world, particularly in Europe, and helps to run the UK’s national data catalogue, data.gov.uk.

> DataCatalogs.org is being launched at the Open Knowledge Foundation’s annual conference, OKCon 2011 (okcon.org) which brings together developers, designers, civil servants, journalists and NGOS for a week of planning, coding and talks.

> For further details please contact Jonathan Gray, Community Coordinator at the Open Knowledge Foundation on jonathan.gray@okfn.org.


Comments Off

Avatar of Lucy Chambers

by Lucy Chambers

The Annotator – Preview

May 12, 2011 in Annotator, News, Releases, Technical

In November 2010, Rufus Pollock announced the Annotator project on the OKFN blog. Since this initial release the project has been developed into a fully fledged product.

The Annotator is a JavaScript widget that can be added to any webpage to allow inline annotation of its contents. Combined with a storage system, such as AnnotateIt, this allows online collaboration on all forms of HTML documentation, literature and articles.

Annotator 1 Annotator 3 Annotator 4

Features

Updated interface

Making use of the latest web technologies, we’ve refined the interface to be more intuitive for the user. Annotations will always try and display inside the window and the comment editor can now be resized and repositioned to suit.

Simple to install

Once the Annotator library code has been included, the Annotator can be added to your website in just one more line of JavaScript:

var content = jQuery(‘#content-to-annotate’).annotator();

Pluggable architecture

The Annotator itself is a just a small piece of code that manages the annotations. It can then be extended with plugins to enhance its functionality.

A broad range of plugins can be created. Examples bundled with the Annotator include tagging, permissions and a store for interfacing with an external database to save annotations.

Add plugins to your webpage with just one more line of code:

content.annotator(‘addPlugin’, ‘Tags’).annotator(‘addPlugin’, ‘Store’);

Store

The Annotator Store plugin can be tailored to talk to any backend but to make things really easy we’ve created AnnotateIt, a simple web service for storing your annotations. This is currently a closed Beta and you can sign up for an account to test it out here.

On the other hand if you’re feeling creative you can build your own store. The Annotator uses a very simple JSON based protocol that can be implemented in any language. The AnnotateIt source code is available on GitHub for guidance.

Bookmarklet

AnnotateIt also allowed us to build a bookmarklet that allows you to inject the Annotator into any webpage and begin annotating immediately. The next time you visit the webpage simply reload the bookmarklet to have all your annotations restored. This makes it easy to separate editorial control over content from control over commentary.

Find out more

If we’ve piqued your interest please head on over to annotateit.org and sign up for a free account.

We’re very excited to see what can be done with the project and are seeking feedback to continue to improve the product. To get in touch just drop us an email at annotator [at] okfn [dot] org.

If you’re interested in adding the Annotator to your own site or developing a plugin then our GitHub page is the place to go. Here you can download the latest Annotator source code and browse the wiki for details on getting started.

Useful Links

1 Comment »

Please create an account to get started.

Sign up to the Open Knowledge Newsletter

Get Updates