This is a guest blog post by OpenCorporates.
OpenCorporates is now 3 years old. Looking back our first blog on the Open Knowledge (Foundation) blog about reaching 20 million companies, it is heartening to see that we have come a long way. We now have over 70 million companies in 80 jurisdictions worldwide making us the world’s largest open database of companies. The success story of OpenCorporates is not that of a tiny team but that of the whole open data community because it has always been a community effort thanks to the efforts of Open Knowledge and others. From writing scrapers to alerting us when new data is available, deciphering language issues or helping us grow our reach – the open data community has been the driver behind OpenCorporates.
Yet, while our core target of a URL for every single company in the world is making great progress, there’s a bigger goal here – of de-siloing all the government data that relates to companies and connecting it to those companies. In fact, one of the most frequent questions has been “How can I help get data into OpenCorporates?” Now, we have an answer to that. Not just an answer – a brand new platform, that makes it possible for the community to help us get company-related data into OpenCorporates.
To start this new era of crowdscraping – we launched a #FlashHacks campaign which aims to get 10 million datapoints in 10 days. With your help, we are confident we can smash the target.
Why is this important?
Information about public and private sector is of monumental importance to understanding and changing the world we live in. Transnational corporations can wield unprecedented influence on politics and economy and we have a limited capacity to understand this when we don’t know what these legal entities look like. The influence of these companies can be good or bad and we don’t have a clear picture of this.
Company information is often not available and when it is, it is buried under hard-to-use websites and PDFs. Fortunately, the work of the open data and transparency community has brought a tide of change. With the introduction of Open Government Partnership and G8 Open Data Charter, governments are committing to make this information easily and publicly available. Yet, action on this front remains slow. And that’s why scraping is at the heart of the open data movement! Where would the open data community be if it had not been for bot-writers spending time deciphering formats and writing code to release data?
We want to use #FlashHacks as a celebration of the commitment of bot-writers and invite others to join us in changing the world through open data.
#FlashHacks at OKFestival
The last day of the campaign coincides with the last day of OKFestival, probably, the biggest gathering of the open data community. So, we will be putting on three #FlashHacks in partnership with Open Knowledge Germany, Code for Africa and Sunlight Foundation.
The OKF Germany #FlashHack will be releasing German data. Sign up here.
The Sunlight Foundation #FlashHack will be releasing political lobbying data. Sign up here.
The Code for Africa #FlashHack will be releasing African data. Sign up here.
How you can join the crowdscraping movement if you can’t make it to OKFest?
- If you can code in Ruby and/or Python, join http://missions.opencorporates.com and sign up!
- Have a look at the datasets we have listed on the Campaign page! If there is a dataset you think we should include in this, please put that down here.
- Sign up to a mission! Send a tweet pledge to say you have taken on a mission.
- Write the bot and submit on the platform.
- Tweet your success with the #FlashHacks tag! Don’t forget to upload the FlashHack design as your twitter cover photo and facebook cover photo to get more people involved.
Any problems – you can post on our Google Group.
This post is by a guest poster. If you would like to write something for the Open Knowledge Foundation blog, please see the submissions page.
