Momentum building for open government data in Norway

The following guest post is from Olav Anders
Øvrebø, Assistant Professor at
the University of Bergen, and member of the Open Knowledge
Foundation‘s Working Group on EU Open
Data. This text was first published as a European Public Sector Information Platform Topic Report on ePSIplatform.eu.

A series of promising new initiatives gives reasons to be a lot more optimistic about government data reuse in Norway today than anyone could have been a year ago. The right tools will hopefully soon be available. Now convincing examples of reuse are needed.

When the application deadline expired at midnight we had received 135 applications to Nettskap! We are very pleased!

This exuberant tweet was published on the Ministry of Government Administration’s official account on May 10. The idea competition Nettskap 2.0, a Norwegian version of the Apps for Democracy or Show Us a Better Way contests, had been announced just a month earlier, which makes the quantity of applications quite impressive. Nettskap applicants were not required to place government data reuse at the heart of their projects, but the Ministry had encouraged this. It worked – 90 projects, two out of three, are based on the reuse of data.

Winners of the competition are to be announced in June, and will receive funding totaling 2,5 Million Kroner (around 320.000 Euro).

Nettskap is just one of several pieces of good news for government data enthusiasts coming out of Norway lately. In April, the same Ministry unveiled plans for a Norwegian datastore. The URL has been chosen – data.norge.no – and work is now underway on the project details and tender. In the meantime, a blog is set up at data.norge.no where government data questions are discussed.

Two reports on public data reuse have been published already this year, helping to raise interest in the topic. The first was written by a group at the University of Bergen (led by me). The second was produced by an expert group put together by the Norwegian Board of Technology, a consultative office to Parliament.

Norway has long had the potential to become a very interesting place for government data reuse. The public sector is, relatively speaking, considered well-run and efficient, and lots of quality data are collected. There is a deep-rooted tradition of transparency in government, supported by a Freedom of Information Act (FOIA). And the population is more tech-savvy than most, always quick to adopt new communication technology and services.

But despite these favourable conditions, Norway has not been in the data reuse vanguard so far. The accelerating activity and increased attention given to government data issues during the last months might change that. In this article I will consider three factors which I believe will decide the outcome:

Infrastructure: The coming datastore needs to be supplemented by an open license for data reuse and practical initiatives to encourage data publishing.
Principles: The PSI directive has been implemented in the FOIA. However, the fundamental principle of division of labour between the public and private sector (and civil society) in data reuse is still not generally acknowledged.
Reuse in practice: Some public sector agencies are already doing an excellent job making data available, while others haven’t even started. Convincing examples of good reuse are quickly needed for recent positive developments to take hold.

Building the infrastructure
The “data.govs” in the US, UK and other datastores give much-needed ammunition to government data advocates everywhere. In a list of recommendations that concluded our report at the University of Bergen, the swift creation of a Norwegian data.gov was in first place, and many others have voiced the same demand. After first indicating that it was under consideration, the Minister of Government Administration, Rigmor Aasrud, announced the URL data.norge.no in early April.

Not content with waiting for a datastore, several private initiatives were taken during last autumn to start a crowdsourced collection of public sector data sources. This was an important part of our own Bergen project, where we started with an open Googledoc spreadsheet. Some 130 data sources were registered there in a couple of months. And one IT engineer started a wiki called datakilder.no, with essentially the same goal.

The experiences from these experiments are available to the Ministry as it now plans its official datastore, and I believe they could be valuable. Moreover, through a cooperation with the Open Knowledge Foundation, we recently exported the Googledoc data to a Norwegian language instance of the foundation’s Comprehensive Knowledge Archive Network registry (the same software that is used in data.gov.uk).

We are currently considering how to continue this work now that data.norge.no has been announced. Personally, I think an independent datastore run by the open data community has an important role to play in pressing government agencies to release data (here we can be inspired by our German and Canadian colleagues. We have no guarantee that the official datastore will be filled with interesting data instantly – probably it will not. Also, an independent datastore can be used for experiments – such as including international and private sector data sources, and promoting and testing the principles behind linked open data.

Besides the datastore question, the need to define standard licenses for data reuse has been a key request from the open data community. At a conference held by the Ministry in Oslo last December, Opera Software CTO Håkon Wium Lie said that coming up with licenses should be priority number one for the Ministry. Lie, who helped organize a citizen campaign for opening up government data back in 1994, said that in principle there were only two reasons for not publishing data: To protect personal data and national security.

Licenses are certainly sorely needed. Today, practice varies greatly between government agencies. Some attach explicit conditions to their data, others do not inform at all. Some only allow non-commercial reuse. Typically, these differing conditions are confusing to potential re-users. Last December, the Norwegian Mapping Authority released map “tiles” for non-commercial reuse (but not the underlying vector data). It soon emerged that the definition of non-commercial was flawed. For example, would embedding map tiles in a for-profit news website count as commercial use? The Mapping Authority had to edit its terms and conditions.

It remains to be seen how open, or how restrictive, the licenses will be. This is certainly one of the crucial questions now.

During interviews and conversations with civil servants for the University of Bergen report, we noticed great discrepancies in knowledge and interest. We found agencies that had been working professionally with their data sources for years, while others hadn’t given the issue much thought at all. The need is clearly there not only to define licenses, but also to produce a comprehensive and practical guide to opening up data for reuse. We have tried to contribute by translating Ton Zijlstra and James Burke’s government data flowchart into Norwegian.

Adding to the recent positive developments, the Norwegian Ministry has made it clear that it is working on such a guide. I believe the open data community outside government should be involved here as well. This could be done in a number of ways. One possibility is to have everyone comment on a draft of the guide, but the community should also contribute with its own initiatives. Setting up a wiki collecting examples of good practices is one option.

Implementing the principles
Government agencies should make data available for reuse in relevant formats, then leave it to the private sector and civil society groups to present the data in creative and interesting ways (for-profit or not). This assumption of a division of labour is central to the whole idea of government data reuse.

In practice, government agencies will present data as well, and it is hard to see that it would make sense to stop them doing that – as long as the data are released for reuse at the same time. However, the government data debates in Norway show that the fundamental principle is not accepted by everyone. Now and again you hear the argument that it is “unfair” that private companies should earn money on reusing the data collected by a government agency.

Another typical argument against publishing data is that they can be misinterpreted. In a survey of state agencies for the University of Bergen report, 43 percent of respondents agreed that “private businesses and individuals can misunderstand data and disseminate misleading information”. They are probably right – it is impossible to guarantee against misinterprations. However, today it is a very difficult argument to make that information should be kept from the public sphere because of such concerns (how much information would we not have to withdraw if that were a guiding principle?).

An easily discernible tendency that is noted with frustration by those seeking access to government data, is the public sector agency that spends resources on producing its own presentation of data, without releasing the “raw” data. In the worst case, this is the spin doctor’s data presentation.

The good news is that the open data community – from entrepreneurs to journalists to activists – have been handed a new and potentially powerful tool to persuade agencies to make data available.

The EU’s PSI directive was implemented in Norwegian law through the changes in the Freedom of Information Act which came into force January 1, 2009. This is the new section 9:

Right to request access to a collation of information from databases:
Any person may request access to a collation of information that is electronically stored in the databases of an administrative agency insofar as the collation can be done using simple procedures.

In the regulations, the Norwegian Mapping Authority has been permitted to continue its policy of charging for access to map data. Given the importance of map data for so many types of applications, the Mapping Authority’s pricing regime has been heavily criticized for years, dominating the limited discourse on government data.

That debate will predictably continue. Less well known, but now beginning to seep through to the open data community, are the possibilities created by the preparatory documents to the revised FOIA. In the Ministry of Justice’s guide to the Act, the right to copies of digitally stored data is extended to copies in “all existing formats”. It remains to be seen how this will be interpreted in concrete cases. But it is already clear that the unpopular practice in many government agencies of publishing data sets in the pdf format will have to change. At the least, agencies will have to make spreadsheet versions of those data available.

Having data reuse principles “merged” with the FOIA will probably have positive consequences for those of us who work for better access to government data. The FOIA carries a lot of weight in Norway. It gives the open data community a much stronger foundation for seeking access to data than arguments about innovation and economic growth can.

Adding to this is the well-established routines among journalists of using the FOIA as a tool in their daily work. For over a decade, media organizations have had online access to the correspondence register of ministries, and agencies on other government levels have implemented their own public registers. On basis of the information in the register (sender, receiver, letter headline etc), journalists claim to see copies of the correspondence. The press associations follow up by lobbying the ministries and agencies, trying to influence them to adopt liberal practices in releasing documents.

In this environment, claiming access to data just becomes an extension of an existing practice and tradition, not something entirely new and unproven. Civil servants are used to handling FOIA requests. In theory at least, this should work in the open data community’s favour.

Gaining real access
In our research for the Bergen project, we found that only one third of state agencies inform about data sources on their website homepage. Moreover, it is difficult to say if the information is complete (in most cases it probably isn’t). FOIA or not, with no information about data sources it is exceedingly difficult for potential reusers to locate the data. Hence, even with infrastructure and principles in place, there will still be ways to “hide” data.

This means that the open data community will have to continue lobbying and struggling for access to data. Hopefully the government will come up with incentives for agencies to make data available. But pressure from the community will be essential.

That pressure will have much greater chance for success if there are convincing examples of creative reuse to point to. Here contests such as Nettskap are important – they will speed up the production of good reuse cases. The community can do a lot in this area as well, by highlighting intelligent reuse of data at home and abroad.

In the best case scenario, user groups build “coalitions” with government agencies. There is a potential for this organized around location – for example, journalists, IT consultants, scientists and activists can team up with the city government with the common aim to improve a region’s competitiveness and transparency. Or such coalitions can be organized around interests – such as environment/climate data, where there is both commercial and civil society interest.

A government data champion?
Norway and the other Scandinavian countries are potential open data champions. Progress has been made in several key areas during the last year, as I have tried to show in this article – more progress than could have been anticipated a year ago.

The Norwegian open data community that I have mentioned repeatedly, is in fact still emerging and very fragmented. One reason is that the interest for open data unites people with very different backgrounds – in the IT sector (business development and programming), in universities, in the media, in the public sector agencies.

Hence, an important challenge will be to bring people closer together and start building more efficient coalitions around the common interest of reusing data for different purposes. To accomplish this, the power of the convincing example cannot be overestimated.

Even if this succeeds, the community will still need help. Political leadership is essential, as only a clear sign that the government gives the question of data reuse priority will give the great majority of agencies the incentive to act. Unlike in the US and UK, we are still waiting for open data to be promoted from the highest political level.