Bounties for scrapers: a new approach to opening global data

This is a guest post by Chris Taggart, co-founder of OpenCorporates.com and member of the Open Knowledge Foundation’s Working Group on Open Government Data.

On Friday we at OpenCorporates announced an innovative (and frankly untested!) way for the open data community to work together in helping opening up one of the most important datasets there is: company numbers and names.

Full details are on the OpenCorporates blog, but basically we’re working with the superb ScraperWiki to open up the company names and numbers in a consistent way. And because we’d like to use that info, we’re offering small bounties for each jurisdiction that’s added, with a total pot of £2,500 (it’s worth stressing that neither the scrapers nor the data will belong to OpenCorporates).

We’ve already had a few scrapers written in response to the challenge, but are plenty more territories to do, and we’re particularly keen to see the opening up of the data for those countries where the system is a little newer, such as those in Eastern Europe, north Africa, or Asia. And then, of course there’s the huge task of the US states (we’ve done Michigan and DC).

It’s worth saying that many of these registers have distinctly un-open licences. This is in part why we’re just asking for the most basic and non-contentious information: the company name, number and possibly status or company type. However moving forward we need to open up the whole register, and we’ve already had positive discussions with some countries for doing this. Till then, happy scraping.