Open for Business

The following post is from Professor Nigel Shadbolt, who is on the UK Government’s Public Sector Transparency Board. This article was originally published in Think Quarterly and is reproduced here with permission from the author.

The first decade of the twenty-first century has been defined by our insatiable demand for information. It has led to the emergence of the ‘open data’ movement, whose powerful advocates include politicians and government officials. In January 2010, Tim Berners-Lee and I unveiled data.gov.uk, providing a single point of access to thousands of UK government datasets, from detailed local and national spending data to street-level crimes and hospital infection rates.

Why is there a growing momentum behind open government data (OGD)? What are the benefits of making non-personal public data freely available? And what does it mean for both businesses and ordinary citizens?

Open data provides a platform on which innovation and value generation can flourish. If governments publish their data and get out of the way, the applications that people want will emerge. In the UK, services like FixMyStreet reduce the pain of reporting local problems like dog fouling and broken streetlights by allowing the public to share their complaints online. Who’s Lobbying helps keep track of the special interests influencing government ministers. Schooloscope makes school performance information useable. SpotlightOnSpend shows not just how various councils are spending our money, but which companies are profiting. And there are dozens of apps like TravelOptions that make finding your way around London easier. All are powered by open data.

The really cool thing is that OGD can be the agent of its own improvement. In the UK, there has been a crowd-sourced effort to improve the Department of Transport’s database, which, amongst other things, details the precise location of the nation’s bus stops. Or at least it purports to – about 18,000 aren’t where they’re supposed to be, so the public has been busy bringing the database into alignment with reality.

The lesson is to appreciate the larger economic and social prize – letting the data go enables value to be built at scale.

And let’s not forget that detailed information about spending, education, transport, energy, environment, crime and health enables citizens to be better informed and hold public service providers to account. If we really do believe in evidence-based policy then it is essential to have data that is open to scrutiny and debate.

For data.gov.uk, it wasn’t enough just to establish a single point of access and then populate it with datasets. We had to draw up the Open Government Licence (OGL), which grants blanket permission to re-use the majority of government data. Developers won’t use data if it is ring-fenced by restrictions and limitations. We established the UK’s Public Data Principles to determine the ‘what’ and ‘how’ for publishing government data on the web. We set out a simple ‘five stars of openness’ test for judging how open and re-useable data is, from simply putting data on the web under an open licence, to linking it to other data to enrich and give it context. But there is plenty more work to be done.

The challenges are organisational and cultural. Persuading state departments to publish non-personal public data necessitates a significant change in attitude. We have to show the benefits so that the advantages of publication are clear. Often our public services are operated by the private sector on a franchised, regulated or subsidised basis. The UK government will be looking to extend its open data principles to these organisations, too. If a private company is in receipt of public funds to run a public service then the data it uses to run these services should be open.

As governments grapple with these challenges, the next question is how open data works for business.

An increasing number of companies are selling added-value services that build on OGD. From business intelligence to spending analysis and data-driven journalism as practised by the Guardian in the UK and The Texas Tribune in the US, there is value in data – whether it is a paid-for app built from now-open UK mapping data, or the latest free travel app that makes its developer money through advertising.

But what about the deeper question of whether businesses’ own data might be better exploited if it was open? There was a time when bookshops regarded their inventory a trade secret. They wouldn’t tell anyone else – customers, competitors or their supply chain – exactly what stock they held. This is now inconceivable: you expect to know what the online bookshop carries and when you can expect to receive your order. Price comparison and product aggregation sites are a good example of how companies can’t afford to hide their information. Whether you’re the cheapest offer or the more expensive one with additional features, your product or service data needs to be seen.

The airline industry demonstrates how opening up data can help a business, while also helping the industry overall. Only a few years ago, you had to go to an airline’s website to find a flight, visiting more than one in order to make a decision on what to book. Then we saw the emergence of flight search engines such as Kayak and Skyscanner. They started scraping the airlines’ sites for timetables, prices and search results to help prospective travellers make a choice. The airlines fought this – blocking and banning the harvesting of their sites. But over time they came to realise that having their flight data on more sites and in more searches meant more business. They have started to recognise the value of making their data more openly available.

We know that better information makes better markets. Lack of access to information about demand and supply makes it difficult for both suppliers and traders to plan, economise and improve their activities.

Open data offers the prospect of instant connectivity between partners, as in open supply chains, where businesses source from places they might never have considered or even suspected could be a source. Open data can reduce integration costs, improve transparency and harness the innovation of others. If you release your data then others will develop applications that make best use of it – providing new services that benefit you directly, like all of those free travel apps that the travel companies didn’t have to write, but which nevertheless drive people onto the transportation network.

Of course, in the world of abundant data, where we see significant amounts being made freely available – or available at marginal cost – there is a clear business challenge. The question is what sort of data provider you’ll become. The world wants high-quality data with a good provenance. Data authorities like Google, Engadget or the Internet Movie Database do well out of their ‘trusted brand’ status.

So what should you do? An ‘open data assay’ is a first step. Ask yourself: what information do you hold? Can any of it be published freely to improve transparency or enhance brand reputation? Is there data which, if published, could make your business more efficient, or generate value, whether directly or indirectly? Take the OGD checklist and frame it in a company context. Think hard about where value is generated in our new information environments. We know how to architect and set up open data portals – so the next steps are ones we can take together. Think about registering data.YOURCO.com or data.YOURSECTOR.com – you might need it sooner than you think.

The OGD revolution is important. Viewed as a precursor to a wider open data movement, it could be as important as any we have seen in the web era.