Cross-posted from scraperwiki.com This post is a write-up of the talk I gave at Strata London in May 2015 called “Sharing humanitarian data at the United Nations”. You can find the slides on that page. The Humanitarian Data Exchange (HDX) is an unusual data hub. It’s made by the UN, and is successfully used by […]
This is a guest post by Francis Irving (@frabcus), CEO of ScraperWiki, who has made several of the world’s first civic websites such as TheyWorkForYou and WhatDoTheyKnow. This is the third in a series of posts about the UK Government’s Performance Platform. Part 1 introduced why the platform is exciting, and part 2 described how […]
The possibilities of open data have been enthralling us for 10 years. I came to it through wanting to make Government really usable, to build sites like TheyWorkForYou. But that excitement isn’t what matters in the end. What matters is scale – which organisational structures will make this movement explode? Whether by creating self-growing volunteer […]
The following post is by Francis Irving, CEO of ScraperWiki. Can you think of an open data curation project where the people who work on it come from multiple commercial companies? In the mid 1990s, as open source code began to boom, the equivalent was commonplace. Geeks working at ISPs would together patch the Apache webserver into shape. Startups like […]
This is a joint blog post by Francis Irving, CEO of ScraperWiki, and Rufus Pollock, Founder of the Open Knowledge Foundation. It’s being cross-posted to both blogs. Content Management Systems, remember those? It’s 1994. You haven’t heard of the World Wide Web yet. Your brother goes to a top university. He once overheard some geeks […]
The following post is by Francis Irving, CEO of ScraperWiki. Now it seems almost normal that red in tooth and claw competitors, like Microsoft and Google, are both major contributors to the latest version of a popular open source operating system kernel. Businesses are gradually realising they can share the costs of anything based on […]
The following post is from Francis Irving, CEO of ScraperWiki. ‘Should Britain flog off the family silver to cut our national debt?‘ — that’s the question the UK current affairs documentary Dispatches tackled last Monday. ScraperWiki worked with Channel 4 News and Dispatches to make two supporting data visualisations, to help viewers understand what assets […]
If you’d asked me back in 2005, I’d have told you that the 2010 election would be the first online election. It turned out not to be. For example, the YouTube and Facebook leaders debate was much less important than the Television debates. However, there are a few places relating to data where the Internet […]
When data.gov.uk was launched, I had a quick browse around the data, to get a feel for what was in it. Most data sets that I randomly looked at were from statistics.gov.uk (from the Office for National Statistics). Today, I decided to investigate, and work out some basic statistics about the source of the data. […]
If starting a new, public interest, organisation, there are three obvious principles you might like to have. Finance – have all bank transactions automatically public in real time. Plus accounts. Software – all software made by the organisation to be open source. Information – voluntarily subscribe to some sort of FOI law. The software one […]
GISTEMP is a crucial open data set, because it contains the historical global temperature record. Not very important right now, but in the medium term absolutely vital for the continuing functioning of our society given the likelihood of adverse climate change. Stations that measure temperature naturally do so at specific points in space, and the […]
Writing in Times Online in April 2006 the eminent Professor Richard Susskind, legal tech guru and adviser to the great and good, spelt out his vision for a “Wikipedia of English law”: This online resource could be established and maintained collectively by the legal profession; by practitioners, judges, academics and voluntary workers. If leaders in […]
Free Our Bills! is a campaign led by a cheeky platypus, just escaping from the portcullis of Parliament. Sign up now, or read on… Sometimes data being free isn’t good enough – it needs to be released in a properly structured format. If you want to reproduce the text of Bills (proposed new laws in […]
Hot off the press – the UK’s Department for Business, Enterprise and Regulatory Reform has published a review of the economics of trading funds. The review follows (I think) recommendation 9 of the Power of Information review: Recommendation 9. By Budget 2008, government should commission and publish an independent review of the costs and benefits […]
In a speech today, the Leader of the Opposition in the UK said some interesting things about freeing up Government information. He begins with some general open information philosophy: But look at our Government at home. It’s still bureaucratic, still top-down and still old-world. It still thinks it knows best and that it should keep […]
Sometimes you find some data whose lack of freedom is totally mysterious from a commercial point of view. At mySociety, we recently released made some travel time maps which help you work out where you should live that would have the quickest commute to your place work. Interactive flash sliders to balance this delicate equation […]
My last post here, Google vs. Facebook, was about how our own personal knowledge, that you’ve put into social networks like Facebook or MySpace, should also be open. By this I mean that you should have control of it, and it should be encoded in open formats with open protocols. The last week there’s been […]
Facebook has striken fear not only into the hearts of incumbent dot-com billionaires, but also into the hearts of open data freaks. It’s terrifying me – not least, because I use it. I give the most personal, sensitive information that I have to a private US corporation. I have no way of getting the data […]
There’s a buzz at GUADEC, an open source computer desktop conference in Birmingham right now, about the idea of the Online Desktop. Increasingly we all use web services rather than local applications, and store our own personal knowledge in other people’s proprietary formats and software. GMail rather than Outlook, Flickr rather than iPhoto. Just as […]
Yesterday, the UK’s Cabinet Office published an important new review of Public Sector Information. It’s a fat document to read through, but it oozes with ideas and stories (many no doubt from readers of this blog) about the importance of open data. The obvious question is, what will the Government do with it at all […]
Today the Department for Constitutional Affairs’ long awaited Statute Law Database project has launched, free at point of use for anyone. It’s super. Last week, access to consolidated versions of the law of the UK wasn’t possible without paying lots of money. Now it is free. There are some down sides – 40 acts are […]
“The OFT’s market study into the commercial use of public information has found that more competition in public sector information could benefit the UK economy by around £1billion a year.” Summary and full report here.
Public Whip and TheyWorkForYou have been around for a few years now, grabbing data from the UK Parliament website and making it more useful, and accessible with better navigation, search and email alerts. It’s little known, but we also make all that open data available in a separate screen scraping project called parlparse. Rufus first […]
In many countries, such as the United States, laws are published as open data, which anybody can copy and reproduce. In the United Kingdom, only the changes to the law (patches, in computer science terms) are published freely. You can find them on the Office of Public Sector Information (OPSI) website. They are known as […]
This work is licensed under a Creative Commons Attribution 4.0 International License.