Support Us

You are browsing the archive for Technical.

CKAN 0.9 Released

August 13, 2009 in CKAN, News, OKF, OKF Projects, Releases, Technical

We are pleased to announce the release of CKAN version 0.9! CKAN is the Comprehensive Knowledge Archive Network, a registry of open knowledge packages and projects.

Changes include:

  • Add version attribute for package
  • Fix purge to use new version of Versioned Domain Model (vdm) (0.4)
  • Link to changed packages when listing revision
  • Show most recently registered or updated packages on front page
  • Bookmarklet to enable easy package registration on CKAN
  • Usability improvements (package search and creation on front page
  • Use external list of licenses from license repository
  • Convert from py.test to nosetests

There are now over 560 packages in the registry – which means that on average we’ve been adding a package a day since version 0.8 was released in May!

Open Data and the Semantic Web Workshop, London, 13th November 2009

August 5, 2009 in Events, OKF, Open Data, Technical

Linking Open Data cloud

We’re currently organising a workshop on ‘open data and the semantic web’, which will take place in London this autumn. Details are as follows:

  • When: Friday 13th November 2009, 1000-1800
  • Where: London Knowledge Lab, 23-29 Emerald Street, London, WC1N 3QS. (See map)
  • Wiki: http://wiki.okfn.org/SemanticWeb
  • Participation: Attendance is free. If you are planning to come along please add your name to the wiki.
  • Microbloggers: See notices on identi.ca and Twitter

Further details:

Semantic web technologists and advocates are increasingly beginning to see the value of ‘open data’ for the data web. Tim Berners-Lee has spoken about the importance of open data, and being able to access raw data in easy to use formats, and the Linking Open Data project demonstrates what can be done by linking together a rich variety of publicly re-usable datasets.

This informal, hands-on workshop will bring together researchers, technologists, and people interested in open data and the semantic web from both public and private sector organisations for a day of talks and discussions.

Themes will include:

  • Linking Open Data
  • Legal tools for open data
  • Finding open data

Data.gov goes live!

May 22, 2009 in News, Open Data, Policy, Technical

Data.gov

The US governments new data.gov site (which we blogged about last month) is now live!

There are currently a selection of core datasets available – from information about World Copper Smelters to results from the Residential Energy Consumption Survey. Raw data is available in XML, Text/CSV, KML/KMZ, Feeds, XLS, or ESRI Shapefile formats. As well as exploring and downloading the data that is available you can also suggest datasets that you’d like to be added!

From the site:

As a priority Open Government Initiative for President Obama’s administration, Data.gov increases the ability of the public to easily find, download, and use datasets that are generated and held by the Federal Government. Data.gov provides descriptions of the Federal datasets (metadata), information about how to access the datasets, and tools that leverage government datasets. The data catalogs will continue to grow as datasets are added. Federal, Executive Branch data are included in the first version of Data.gov.

Data.gov is a major milestone in the Obama administration’s Open Government Initiative. To mark the occasion, Sunlight Labs, Google, O’Reilly Media, and TechWeb have launched Apps for America 2 – inviting proposals for open source mashups, visualisations or other innovative re-uses of material from data.gov.

You can watch a video of Vivek Kundra, the US’s CIO, talking about data.gov on YouTube.

Great news for open government data – and open data in general!

Launch of Open Data Grid

May 13, 2009 in News, OKF, OKF Projects, Open Data, Technical

storage facility 37 by sevensixfive

In the last couple of months we’ve had several threads on the okfn-discuss list about distributed storage for open data (see here and here).

Last month we started a distributed storage project, aiming to provide distributed storage infrastructure for OKF and other open knowledge projects.

After researching various technical options, we’ve launched an Open Data Grid based on Allmydata’s open-source “Tahoe” system at:

Anyone can store open data on the grid, or start running a storage node. For more details see the readme. If you’d like to comment on the service feel free to post on the okfn-discuss list!

Avatar of admin

by admin

CKAN 0.8 Released

May 12, 2009 in CKAN, News, OKF Projects, Technical

A new release of CKAN is now out together with a new, and substantially improved versioned domain model library. Changes include:

  • View information about package history (ticket:53)
  • Basic datapkg integration (ticket:57)
  • Show information about package openness using icons (ticket:56)
  • One-stage package create/registration (r437)
  • Reinstate package attribute validation (r437)
  • Upgrade to vdm 0.4

The CKAN code is available from:

The data is available from:

We’ve now over 500 packages — an almost 100% increase in the last six months. If you come across a large dataset or substantial collection, please consider registering it on CKAN!

5th Communia Workshop: Post-Event Information + Statement

April 23, 2009 in COMMUNIA, Events, News, OKF, Open Data, Policy, Talks, Technical

Participants at 5th COMMUNIA Workshop

The 5th Communia Workshop took place last month at the London School of Economics. It brought together researchers, policy-makers, stakeholders and representatives from across Europe, the United States and Australia for two days of talks and discussions about reusing public sector content and data.

In the afternoon of the first day, participants co-drafted a simple statement. If you support the statement, we encourage you to sign – regardless of whether or not you attended the workshop:

In addition many of the speakers made suggestions for policy recommendations, which are available at:

Documentation, including audio, video and slides, will be published at:

Material published so far includes:

You can also see:

Open Knowledge Conference (OKCon) 2009: Saturday 28th March

March 22, 2009 in Events, Free Culture, News, OKF, Open Access, Open Data, Open Science, Public Domain, Talks, Technical

Open Knowledge Conference (OKCon) 2009 will take place next Saturday 28th March – less than a week away!

If you plan to attend, and haven’t registered yet – we encourage you to book your ticket now as space is limited!

OKCon 2009: speakers and sessions

There will be two main sessions on ‘open knowledge for development’ and ‘open data and the semantic web’. In addition there will be plenty of open space – which currently includes talks ranging from visualising historical data to public domain fashion. Speakers will include:

Full details are available at the programme page and the provisional open space schedule.

Second Open Visualisation Workshop, Saturday 21st June 2008, Trampoline Systems

June 13, 2008 in Events, OKF, Talks, Technical

At the first Open Visualisation Workshop last month, we found there was significant demand for regular workshops. Hence, the second Open Visualisation Workshop will take place a week from Saturday! Details are as follows:

Once again, the event should be a good opportunity for people new to this area to learn a bit more about the range of open source visualisation software packages that are out there – and for open source visualisation veterans to showcase their work and to exchange their experiences.

This time we hope to get stuck into to tinkering around with several software packages, and possibly start work on a new project visualising some open material!

Avatar of admin

by admin

Versioned Domain Model v0.2 with Support for SQLAlchemy Released

May 30, 2008 in News, OKF Projects, Technical

v0.2 of versioned domain model is now finally done — it was 95% complete in Feb but it has taken another 3 months to iron out the last bugs and polish up!

Apart from being a concrete implementation of a system for versioning data(bases) — and therefore important for efforts to do more collaborative development of data — it is also a crucial piece of infrastructure for various of our current and future projects such as CKAN, Microfacts/Weaving History, and Public Domain Works.

Versioned Domain Model (v0.2) with Support for SQLAlchemy

Versioned Domain Model (vdm) is a package which allows you to ‘version’ your domain model in the same way that source code version control systems such as subversion allow you version your code. In particular, versioned domain model versions a complete model and not just individual domain objects (for more on this distinction see below).

At present the package is provided as an extension to SQLAlchemy and SQLObject (with an extension to Elixir in progress).

Getting It

Either via easy_install::

$ easy_install vdm

Or from subversion at::

http://knowledgeforge.net/ckan/svn/vdm/trunk

A Full Versioned Domain Model

To permit ‘atomic’ changes involving multiple objects at once as well as to facilitate domain object traversal it is necessary to introduce an explicit ‘Revision’ object to represent a single changeset to the domain model.

One also needs to introduce the concept of ‘State’. This allows us to make (some) domain objects stateful, in particular those which are to be versioned (State is necessary to support delete/undelete functionality as well as to implement versioned many-to-many relationships).

For each original domain object that comes versioned we end up with 2 domain objects:

  • The ‘continuity’: the original domain object.
  • The ‘version/revision’: the versions/revisions of that domain object.

Often a user will never need to be concerned (explicitly) with the version/revision object as they will just interact with the original domain object, which will, where necessary, ‘proxy’ requests down to the ‘version/revision’.

To give a flavour of all of this here is a pseudo-code example::

# NB: Book and Author are domain objects which has been made
# versioned using the vdm library
# 
# we need a session of some kind to track which objects have been changed
# each session then has a single revision
rev1 = Revision(author='me')
session.revision = rev1
# typo!
b1 = Book(name='warandpeace', title='War and Peacee')
b2 = Book(name='annakarenina', title='Anna')
b3 = Book(name='warandpeace')
a1 = Author(name='tolstoy')
# this is just shorthand for ending this revision and saving all changes
# this may vary depending on the implementation
rev1.commit()
timestamp1 = rev1.timestamp

# some time later

rev2 = Revision(author='me')
session.revision = rev2
b1 = Book.get(name='warandpeace')
# correct typo
b1.title = 'War and Peace'
# add the author
a1 = Author.get(name='tolstoy')
b1.authors.append(a1)
# duplicate item so delete
b3.delete()
rev2.commit()

# some time even later
rev1 = Revision.get(timestamp=timestamp1)
b1 = Book.get(name='warandpeace') 
b1 = b1.get_as_of(rev1)
assert b1.title == 'War and Peacee'
assert b1.authors == []
# etc

More Information

For more information see the documentation in the package repository at:

http://knowledgeforge.net/ckan/svn/vdm/trunk/

Please create an account to get started.

Sign up to the Open Knowledge Newsletter

Get Updates