Support Us

You are browsing the archive for Open Data.

UK Crime Data: Feeling is Believing

Meg Foulkes - July 1, 2015 in Featured, Legal, Open Data, Open Government Data, open knowledge

Latest crime data shows that the UK is getting significantly more ‘peaceful’. Last month, the Institute for Economics and Peace published the UK Peace Index, revealing UK crime figures have fallen the most of all EU countries in the past decade. Homicide rates, to take one indicator, have halved over the last decade.

Crime Scene by Alan Cleaver, Flickr, CC-BY

Crime Scene by Alan Cleaver, Flickr, CC-BY

But the British public still feels that crime levels are rising. How can opening up crime data play a part in convincing us we are less likely to experience crime than ever before?

The ‘Perception Gap’

The discrepancy between crime data and perceptions of the likelihood of crime is particularly marked in the UK. Although it has been found that a majority of the public broadly trust official statistics, the figures are markedly lower for those relating to crime. In one study, 85% of people agreed that the Census accurately reflects changes in the UK, but only 63% said the same of crime statistics.

Credibility of Police Data

Police forces have been publishing crime statistics in the UK since 2008, using their own web-based crime mapping tools or via the national crime mapping facility (http://maps.police.uk/ and http://www.police.uk). This has been purportedly for the purpose of improving engagement with local communities alongside other policy objectives, such as promoting transparency. But allegations of ‘figure fiddling’ on the part of the police have undermined the data’s credibility and in 2014, the UK Statistics Authority withdrew its gold-standard status from police figures, pointing to ‘accumulating evidence’ of unreliability.

The UK’s open data site for crime figures allows users to download street-level crime and outcome data in CSV format and explore the API containing detailed crime data and information about individual police forces and neighbourhood teams. It also provides Custom CSV download and JSON API helper interfaces so you can more easily access subsets of the data.

Crime map from data.police.co.uk

But the credibility of the data has been called into question. Just recently, data relating to stop-search incidents for children aged under-12 was proved ‘inaccurate’. The site itself details many issues which call the accuracy of the data into question: inconsistent geocoding policies in police forces; “Six police forces we suspect may be double-reporting certain types of incidents“; ‘siloed systems’ within police records; and differing IT systems from regional force to force.

In summary, we cannot be sure the ‘data provided is fully accurate or consistent.’

The Role the Media Plays: If it Bleeds, it Leads

In response to persistent and widespread public disbelief, the policies of successive UK governments on crime have toughened: much tougher sentencing, more people in prison, more police on the streets. When the British public were asked why they think there is more crime now than in the past, more than half (57%) stated that it was because of what they see on television and almost half (48%) said it was because of what they read in newspapers [Ipsos MORI poll on Closing the Gaps. One tabloid newspaper, exclaimed just recently: “Rape still at record levels and violent crime rises” and “Crime shows biggest rise for a decade“. As the adage goes, If it Bleeds, it Leads.

Crime Data and Mistrust of the Police

Those engaged in making crime figures meaningful to the public face unique challenges. When Stephen Lawrence was murdered in 1993, and the following public inquiry found institutional racism to be at the heart of the Met police, public trust towards the police was shattered. Since then, the police have claimed to have rid their ranks of racism entirely.

Police by Luis Jou García, Flickr, CC BY-NC 2.0

Police by Luis Jou García, Flickr, CC BY-NC 2.0

But many remain less than convinced. According to official statistics, in 1999-2000, a black person was five times more likely than a white person to be stopped by police. A decade later, they were seven times more likely. One criminologist commented: “Claims that the Lawrence inquiry’s finding of institutional racism no longer apply have a hollow ring when we look at the evidence on police stops.” [Michael Shiner reported in the Guardian].

Equally, the police distrust the public too. The murder of two young, female police officers in Manchester in 2012 ignited the long-rumbling debate over whether the police should be armed. So the divide between the police and the public is a serious one.

A Different Tack?

In 2011, a review was undertaken by the UK Statistics Authority into Crime Data. Its recommendations included:

  • Improving the presentation of crime statistics to make them more authoritative
  • Reviewing the availability of local crime and criminal justice data on government websites to identify opportunities for consolidation
  • Sharing of best practice and improvements in metadata and providing reassurance on the quality of police crime records.

It’s clear that the UK police recognise the importance of improving their publication of data. But it seems that opening data alone won’t fix the shattered trust between the public and the police, even if the proof that Britons are safer than ever before is there in transparent, easily navigable data. We need to go further back in the chain of provenance, scrutinise the reporting methods of the police for instance.

But this is about forgiveness too, and the British public might just not be ready for that yet.

Putting Open at the Heart of the Digital Age

Rufus Pollock - June 5, 2015 in Featured, Open Data, open knowledge

Introduction

I’m Rufus Pollock.

In 2004 I founded a non-profit called Open Knowledge

The mission we set ourselves was to open up all public interest information – and see it used to create insight that drives change.

What sort of public interest information? In short, all of it. From big issues like how our government spends our taxes or how fast climate change is happening to simple, everyday, things like when the next bus is arriving or the exact address of that coffee shop down the street.

For the last decade, we have been pioneers and leaders in the open data and open knowledge movement. We wrote the original definition of open data in 2005, we’ve helped unlock thousands of datasets. And we’ve built tools like CKAN, that powers dozens of open data portals, like data.gov in the US and data.gov.uk in the UK. We’ve created a network of individuals and organizations in more than 30 countries, who are all working to make information open, because they want to drive insight and change.

But today I’m not here to talk specifically about Open Knowledge or what we do.

Instead, I want to step back and talk about the bigger picture. I want to talk to you about digital age, where all that glitters is bits, and why we need to put openness at its heart.

Gutenberg and Tyndale

To do that I first want to tell you a story. Its a true story and it happened a while ago – nearly 500 years ago. It involves two people. The first one is Johannes Gutenberg. In 1450 Gutenberg invented this: the printing press. Like the Internet in our own time, it was revolutionary. It is estimated that before the printing press was invented, there were just 30,000 books in all of Europe. 50 years later, there were more than 10 million. Revolutionary, then, though it moved at the pace of the fifteenth century, a pace of decades not years. Over the next five hundred years, Gutenberg’s invention would transform our ability to share knowledge and help create the modern world.

The second is William Tyndale. He was born in England around 1494, so he grew up in world of Gutenberg’s invention.

Tyndale followed the classic path of a scholar at the time and was ordained as a priest. In the 1510s, when he was still a young man, the Reformation still hadn’t happened and the Pope was supreme ruler of a united church across Europe. The Church – and the papacy – guarded its power over knowledge, forbidding the translation of the bible from Latin so that only its official priests could understand and interpret it.

Tyndale had an independent mind. There’s a story that he got into an argument with a local priest. The priest told him:

“We are better to be without God’s laws than the Pope’s.”

Tyndale replied:

“If God spare my life ere many years, I will cause the boy that drives the plow to know more of the scriptures than you!”

What Tyndale meant was that he would open up the Bible to everyone.

Tyndale made good on his promise. Having fled abroad to avoid persecution, between 1524 and 1527 he produced the first printed English translation of the Bible which was secretly shipped back to England hidden in the barrels of merchant ships. Despite being banned and publicly burnt, his translation spread rapidly, giving ordinary people access to the Bible and sowing the seeds of the Reformation in England.

However, Tyndale did not live to see it. In hiding because of his efforts to liberate knowledge, he was betrayed and captured in 1534. Convicted of heresy for his work, on the 6th October 1536, he was strangled then burnt at the stake in a prison yard at Vilvoorden castle just north of modern day Brussels. He was just over 40 years old.

Internet

So let’s fast forward now back to today, or not quite today – the late 1990s.

I go to college and I discover the Internet.

It just hit me: wow! I remember days spent just surfing around. I’d always been an information junkie, and I felt like I’d found this incredible, never-ending information funfair.

And I got that I was going to grow up in a special moment, at the transition to an information age. We’d be living in this magical world, where the the main thing we create and use – information – could be instantaneously and freely shared with everyone on the whole planet.

But … why Openness

So, OK the Internet’s awesome …

Bet you haven’t heard that before!

BUT … – and this is the big but.

The Internet is NOT my religion.

The Internet – and digital technology – are not enough.

I’m not sure I have a religion at all, but if I believe in something in this digital age, I believe in openness.

This talk is not about technology. It’s about how putting openness at the heart of the digital age is essential if we really want to make a difference, really create change, really challenge inequity and injustice.

Which brings me back to Tyndale and Gutenberg.

Tyndale revisited

Because, you see, the person that inspired me wasn’t Gutenberg. It was Tyndale.

Gutenberg created the technology that laid the groundwork for change. But the printing press could very well have been used to pump out more Latin bibles, which would then only have made it easier for local priests to be in charge of telling their congregations the word of God every Sunday. More of the same, basically.

Tyndale did something different. Something so threatening to the powers that be that he was executed for it.

What did he do? He translated the Bible into English.

Of course, he needed the printing press. In a world of hand-copying by scribes or painstaking woodcut printing, it wouldn’t make much difference if the Bible was in English or not because so few people could get their hands on a copy.

But, the printing press was just the means: it was Tyndale’s work putting the Bible in everyday language that actually opened it up. And he did this with the express purpose of empowering and liberating ordinary people – giving them the opportunity to understand, think and decide for themselves. This was open knowledge as freedom, open knowledge as systematic change.

Now I’m not religious, but when I talk about opening up knowledge I am coming from a similar place: I want anyone and everyone to be able to access, build on and share that knowledge for themselves and for any purpose. I want everyone to have the power and freedom to use, create and share knowledge.

Knowledge power in the 16th century was controlling the Bible. Today, in our data driven world it’s much broader: it’s about everything from maps to medicines, sonnets to statistics. Its about opening up all the essential information and building insight and knowledge together.

This isn’t just dreaming – we have inspiring, concrete examples of what this means. Right now I’ll highlight just two: medicines and maps.

Example: Medicines

Everyday, millions of people around the world take billions of pills, of medicines.

Whether those drugs actually do you good – and what side effects they have – is obviously essential information for researchers, for doctors, for patients, for regulators – pretty much everyone.

We have a great way of assessing the effectiveness of drugs: randomized control trials in which a drug is compared to its next best alternative.

So all we need is all the data on all those trials (this would be non-personal information only – any information that could identify individuals would be removed). In an Internet age you’d imagine that that this would be a simple matter – we just need all the data openly available and maybe some way to search it.

You’d be wrong.

Many studies, especially negative ones, are never published – the vast majority of studies are funded by industry who use restrictive contracts to control what gets published. Even where pharmaceutical companies are required to report on the clinical trials they perform, the regulator often keeps the information secret or publishes it as 8,000 page PDFs each page hand-scanned and unreadable by a computer.

If you think I’m joking I’ll give just one very quick example which comes straight from Ben Goldacre’s Bad Pharma. In 2007 researchers in Europe wanted to review the evidence on a diet drug called rimonabant. They asked the European regulator for access to the original clinical trials information submitted when the drug was approved. For three years they were refused access on a variety of grounds. When they did get access this is what they got initially – that’s right 60 pages of blacked out PDF.

We might think this was funny if it weren’t so deadly serious: in 2009, just before the researchers finally got access to the data, rimonabant was removed from the market on the grounds that it increased the risk of serious psychiatric problems and suicide.

This situation needs to change.

And I’m happy to say something is happening. Working with Ben Goldacre, author of Bad Pharma, we’ve just started the OpenTrials project. This will bring together all the data, on all the trials and link it together and make it open so that everyone from researchers to regulators, doctors to patients can find it, access it and use it.

Example: Maps

Our second example is maps. If you were looking for the “scriptures” of this age of digital data, you might well pick maps, or, more specifically the geographic data on which they are built. Geodata is everywhere: from every online purchase to the response to the recent earthquakes in Nepal.

Though you may not realize it, most maps are closed and proprietary – you can’t get the raw data that underpins the map, you can’t alter it or adapt it yourself.

But since 2004 a project called OpenStreetMap has been creating a completely open map of the planet – raw geodata and all. Not only is it open for access and reuse use the database itself is collaboratively built by hundreds of thousands of contributors from all over the world.

What does this mean? Just one example. Because of its openness OpenStreetMap is perfect for rapid updating when disaster strikes – showing which bridges are out, which roads are still passable, what buildings are still standing. For example, when a disastrous earthquake struck Nepal in April this year, volunteers updated 13,199 miles of roads and 110,681 buildings in under 48 hours providing crucial support to relief efforts.

The Message not the Medium

To repeat then: technology is NOT teleology. The medium is NOT the message – and it’s the message that matters.

The printing press made possible an “open” bible but it was Tyndale who made it open – and it was the openness that mattered.

Digital technology gives us unprecedented potential for creativity, sharing, for freedom. But they are possible not inevitable. Technology alone does not make a choice for us.

Remember that we’ve been here before: the printing press was revolutionary but we still ended up with a print media that was often dominated by the few and the powerful.

Think of radio. If you read about how people talked about it in the 1910s and 1920s, it sounds like the way we used to talk about the Internet today. The radio was going to revolutionize human communications and society. It was going to enable a peer to peer world where everyone can broadcast, it was going to allow new forms of democracy and politics, etc. What happened? We got a one way medium, controlled by the state and a few huge corporations.

Look around you today.

The Internet’s costless transmission can – and is – just as easily creating information empires and information robber barons as it can creating digital democracy and information equality.

We already know that this technology offers unprecedented opportunities for surveillance, for monitoring, for tracking. It can just as easily exploit us as empower us.

We need to put openness at the heart of this information age, and at the heart of the Net, if we are really to realize its possibilities for freedom, empowerment, and connection.

The fight then is on the soul of this information age and we have a choice.

A choice of open versus closed.

Of collaboration versus control.

Of empowerment versus exploitation.

Its a long road ahead – longer perhaps than our lifetimes. But we can walk it together.

In this 21st century knowledge revolution, William Tyndale isn’t one person. It’s all of us, making small and big choices: from getting governments and private companies to release their data, to building open databases and infrastructures together, from choosing apps on your phone that are built on open to using social networks that give you control of your data rather than taking it from you.

Let’s choose openness, let’s choose freedom, let’s choose the infinite possibilities of this digital age by putting openness at its heart.

Thank you.

Fantasy Frontbench – giving the public a way to compare politicians

Guest - April 17, 2015 in Open Data

This is a guest blog post by Matt Smith, who is a learning technologist at UCL. He is interested in how technology can be used to empower communities.

Introduction

Fantasy Frontbench is a not-for-profit and openly licensed project aimed at providing the public with an engaging and accessible platform for directly comparing politicians.

A twist on the popular fantasy football concept, the site uses open voting history data from Public Whip and They Work For You. This allows users to create their own fantasy ‘cabinet’ by selecting and sorting politicians on how they have voted in Parliament on key policy issues such as EU integration, Updating Trident, Same-sex marriage and NHS reform.

Once created, users can see how their fantasy frontbench statistically breaks down by gender, educational background, age, experience and voting history. They can then share and debate their selection on social media.

The site is open licensed and we hope to make datasets of user selections available via figshare for academic inquiry.

A wholly state educated frontbench, from our gallery.

A wholly state educated frontbench, from our gallery.

Aim of the project

Our aim is to present political data in a way that is engaging and accessible to those who may traditionally feel intimidated by political media. We wish to empower voters through information and provide them with the opportunity to compare politicians on the issues that most matter to them. We hope the tool will encourage political discourse and increase voter engagement.

Skærmbillede 2015-04-17 kl. 16.41.54

Uses in education

The site features explanations of the electoral system and will hopefully help learners to easily understand how the cabinet is formed, the roles and responsibilities of cabinet ministers and the primary processes of government. Moreover, we hope as learners use the site, it will raise questions surrounding the way in which MPs vote in Parliament and the way in which bills are debated and amended. Finally, we host a gallery page which features a number of frontbenches curated by our team. This allows learners to see how different groups and demographics of politicians would work together. Such frontbenches include an All Female Frontbench, Youngest Frontbench, Most Experienced Frontbench, State Educated Frontbench, and a Pro Same-sex Marriage Frontbench, to name but a few.

Users can see how their frontbench in Parliament has voted on 75 different policy issues.

Users can see how their frontbench in Parliament has voted on 75 different policy issues.

Development

Over the coming weeks, we will continue to develop the site, introducing descriptions of the main political parties, adding graphs which will allow users to track or ‘follow’ how politicians are voting, as well as adding historical frontbenches to the gallery e.g. Tony Blair’s 1997 Frontbench, Margaret Thatcher’s 1979 Frontbench and Winston Churchill’s Wartime Frontbench.

For further information or if you would like to work with us, please contact info@fantasyfrontbench.com or tweet us at [@FantasyFbench](http://twitter.com/FantasyFbench).

Acknowledgements

Fantasy Frontbench is a not-for-profit organisation and is endorsed and funded by the Joseph Rowntree Reform Trust Ltd.

Javiera Atenas provided advice on open licensing and open data for the project.

New research project to map the impact of open budget data

Jonathan Gray - March 4, 2015 in Featured, Open Data, Policy, Research

I’m pleased to announce a new research project to examine the impact of open budget data, undertaken as a collaboration between Open Knowledge and the Digital Methods Initiative at the University of Amsterdam, supported by the Global Initiative for Financial Transparency (GIFT).

The project will include an empirical mapping of who is active around open budget data around the world, and what the main issues, opportunities and challenges are according to different actors. On the basis of this mapping it will provide a review of the various definitions and conceptions of open budget data, arguments for why it matters, best practises for publication and engagement, as well as applications and outcomes in different countries around the world.

As well as drawing on Open Knowledge’s extensive experience and expertise around open budget data (through projects such as Open Spending), it will utilise innovative tools and methods developed at the University of Amsterdam to harness evidence from the web, social media and collections of documents to inform and enrich our analysis.

As part of this project we’re launching a collaborative bibliography of existing research and literature on open budget data and associated topics which we hope will become a useful resource for other organisations, advocates, policy-makers, and researchers working in this area. If you have suggestions for items to add, please do get in touch.

This project follows on from other research projects we’ve conducted around this area – including on data standards for fiscal transparency, on technology for transparent and accountable public finance, and on mapping the open spending community.

Financial transparency field network with the Issuecrawler tool based on hyperlink analysis starting from members of Financial Transparency Coalition, 12th January 2015. Open Knowledge and Digital Methods Initiative.

Financial transparency field network with the Issuecrawler tool based on hyperlink analysis starting from members of Financial Transparency Coalition, 12th January 2015. Open Knowledge and Digital Methods Initiative.

Building a Free & Open World-wide Address Dataset

tomlee - February 23, 2015 in Featured Project, Open Data

Skærmbillede 2015-02-20 kl. 09.50.25

Finding your way through the world is a basic need, so it makes sense that satellite navigation systems like GPS and Galileo are among open data’s most-cited success stories. But as wonderful as those systems are, they’re often more useful to robots than people. Humans usually navigate by addresses, not coordinates. That means that address data is an essential part of any complete mapping system.

Unfortunately, address data has historically been difficult to obtain. At best, it was sold for large amounts of money by a small set of ever-more consolidated vendors. These were often the product of public-private partnerships set up decades ago, under which governments granted exclusive franchises before the digital era unveiled the data’s full importance. In some cases, data exclusivity means that the data simply isn’t available at any price.

Fortunately, the situation is improving. Scores of governments are beginning to recognize that address data is an important part of their open data policy. This is thanks in no small part to the community of advocates working on the issue. Open Knowledge has done important work surveying the availability of parcel and postcode data, both of which are essential parts of address data. OpenAddresses UK has recently launched an ambitious plan to collect and release the country’s address data. And in France, the national OpenStreetMap community’s BANO project has been embraced by the government’s own open data portal.

This is why we’re building OpenAddresses.io, a global community collecting openly available address data. I and my fellow OpenAddresses.io contributors were pleased to recently celebrate our 100 millionth address point:

Getting involved in OpenAddresses is easy and can quickly pay dividends. Adding a new dataset is as easy as submitting a form, and you’ll benefit by improving a global open address dataset in one consistent format that anyone can use. Naturally, we also welcome developers: there are interesting puzzles and mountains of data that still need work.

Our most important tools to gather more data are email and search engines. Addresses are frequently buried in aging cadastral databases and GIS portals. Time spent hunting for them often reveals undiscovered resources. A friendly note to a person in government can unlock new data with surprising success. Many governments simply don’t know that citizens need this data or how to release it as an open resource.

If you work in government and care about open data, we’d like to hear from you. Around the world, countries are acknowledging that basic geographic data belongs in the commons. We need your help to get it there.

BudgetApps: The First All-Russia Contest on Open Finance Data

Ivan Begtin - January 16, 2015 in OKF Russia, Open Data

This is a guest post by Ivan Begtin, Ambassador for Open Knowledge in Russia and co-founder of the Russian Local Group.

budgetapps2

Dear friends, the end of 2014 and the beginning of 2015 have been marked by an event, which is terrific for all those who are interested in working with open data, participating in challenges for apps developers and generally for all people who are into the Open Data Movement. I’m also sure, by the way, that people who are fond of history will find it particularly fascinating to be involved in this event.

On 23 December 2014, the Russian Ministry of Finance together with NGO Infoculture launched an apps developers’ challenge BudgetApps based on the open data, which have been published by the Ministry of Finance over the past several years. There is a number of various datasets, including budget data, audit organisations registries, public debt, national reserve and many other kinds of data.

Now, it happened so that I have joined the jury. So I won’t be able to participate, but let me provide some details regarding this initiative.

All the published data can be found at the Ministry website. Lots of budget datasets are also available at The Single Web Portal of the Russian Federation Budget System. That includes the budget structure in CSV format, the data itself, reference books and many other instructive details. Data regarding all official institutions are placed here. This resource is particularly interesting, because it contains indicators, budgets, statutes and numerous other characteristics regarding each state organisation or municipal institution in Russia. Such data would be invaluable for anyone who considers creating a regional data-based project.

One of the challenge requirements is that the submitted projects should be based on the data published by the Ministry of Finance. However, it does not mean that participants cannot use data from other sources alongside with the Ministry data. It is actually expected that the apps developers will combine several data sources in their projects.

To my mind, one should not even restrict themselves to machine-readable data, because there are also available human-readable data that can be converted to open data formats by participants.

Many potential participants know how to write parsers on their own. For those who have never had such an experience there are great reference resources, e.g. ScraperWiki that can be helpful for scraping web pages. There are also various libraries for analysing Excel files or extracting spreadsheets from PDF documents (for instance, PDFtables, Abbyy Finereader software or other Abbyy services ).

Moreover, at other web resources of the Ministry of Finance there is a lot of interesting information that can be converted to data, including news items that recently have become especially relevant for the Russian audience.

Historical budgets

There is a huge and powerful direction in the general process of opening data, which has long been missing in Russia. What I mean here is publishing open historical data that are kept in archives as large paper volumes of reference books containing myriads of tables with data. These are virtually necessary when we turn to history referring to facts and creating projects devoted to a certain event.

The time has come at last. Any day now the first scanned budgets of the Russian Empire and the Soviet Union will be openly published. A bit later, but also in the near future, the rest of the existing budgets of the Russian Empire, the Soviet Union, and the Russian Soviet Federated Socialist Republic will be published as well.

These scanned copies are being gradually converted to machine-readable formats, such as Excel and CSV data reconstructed from these reference books – both as raw data and as initially processed and ordered data. We created these ordered normalised versions to make it easier for developers to use them in further visualisations and projects. A number of such datasets have already been openly published. It is also worth mentioning that a considerable number of scanned copies of budget reference books (from both the Russian Empire and USSR) have already been published online by Historical Materials, a Russian-language grass-root project launched by a group of statisticians, historians and other enthusiasts.

Here are the historical machine-readable datasets published so far:

I find this part of the challenge particularly inspiring. If I were not part of the jury, I would create my own project based on historical budgets data. Actually, I may well do something like that after the challenge is over (unless somebody does it earlier).

More data?

There is a greater stock of data sources that might be used alongside with the Ministry data. Here are some of them:

These are just a few examples of numerous available data sources. I know that many people also use data from Wikipedia and DBPedia.

What can be done?

First and foremost, there are great opportunities for creating projects aimed at enhancing the understandability of public finance. Among all, these could be visual demos of how the budget (or public debt, or some particular area of finance) is structured.

Second, lots of projects could be launched based on the data on official institutions at bus.gov.ru. For instance, it could be a comparative registry of all hospitals in Russia. Or a project comparing all state universities. Or a map of available public services. Or a visualisation of budgets of Moscow State University (or any other Russian state university for that matter).

As to the historical data, for starters it could be a simple visualisation comparing the current situation to the past. This might be a challenging and fascinating problem to solve.

Why is this important?

BudgetApps is a great way of promoting open data among apps developers, as well as data journalists. There are good reasons for participating. First off, there are many sources of data that provide a good opportunity for talented and creative developers to implement their ambitious ideas. Second, the winners will receive considerable cash prizes. And last, but not least, the most interesting and perspective projects will get a reference at the Ministry of Finance website, which is a good promotion for any worthy project. Considerable amounts of data have become available. It’s time now for a wider audience to become aware of what they are good for.

Pioneering Fellowships Will Help Rewire Africa’s Governments

Katelyn Rogers - November 25, 2014 in Open Data, Open Government Data

Open Knowledge and Code for Africa launch pilot Open Government Fellowship Programme. Apply to become a fellow today. This blog announcement is available in French here and Portuguese here.

C4A_logo (1) OpenKnowledge_LOGO_COLOUR_CMYK PforOD



Do you want to help us build African governments and societies that are more accountable and responsive to citizens?

We are looking for the best ideas for harnessing the power of digital technologies & open data, to improve the way that governments & citizens interact.

Code for Africa and Open Knowledge are offering three pilot Open Government Fellowships to give outstanding changemakers the skills, tools and resources necessary to kickstart open government initiatives in their countries.

The six-month fellowships are intended to empower pioneers who are already working in the open data or civic engagement communities, and are designed to augment their existing ‘day jobs’ rather than remove them from their organisations. Successful fellows will therefore only be expected to work part-time on their fellowship projects (which could include new initiatives at their ‘day jobs’), but will receive strategic and material support throughout their fellowship.

This support will include a modest $1,000 per month stipend, a $3,000 seed fund to kickstart projects, a travel budget to attend local and international events, access to workspace in Code for Africa affiliate civic technology labs across the continent, and technology support from Code for Africa developers and data analysts. Fellows will also be able to tap into Open Knowledge’s School of Data networks and resource kits, and its global network of specialist communities, as well as Code for Africa affiliate communities such as Hacks/Hackers.

The deadline for applications is 15 December 2014. The fellowships are scheduled to start in February 2015 and run until July 2015.

We are looking for candidates that fit the following profile:

  • Currently engaged in the open government and/or related communities . We are looking to support individuals already actively participating in the open government community
  • Understands the role of civil society and citizen based organisations in bringing about positive change through advocacy and campaigning
  • Understands the role and importance of monitoring government commitments on open data as well as on other open government policy related issues
  • Has facilitation skills and enjoys community-building (both online and offline).
  • Is eager to learn from and be connected with an international community of open government experts, advocates and campaigners
  • Currently living and working in Africa. Due to limited resources and our desire to develop a focused and impactful pilot programme, we are limiting applications to those currently living and working in Africa. We hope to expand the programme to the rest of the world starting in 2015.

The fellowship will initially be limited to African countries where either Code for Africa or Open Knowledge have extensive resources or deep partnerships. Applicants should therefore be based in one of the following countries: Angola, Burkina Faso, Cameroon, Ghana, Kenya, Morocco, Mozambique, Mauritius, Namibia, Nigeria, Rwanda, South Africa, Senegal, Tunisia, Tanzania, and Uganda. We hope to expand the initiative to include additional countries later in 2015.

The selection committee will pay particular attention to applicants’ current engagement in the open government movement at local, national and/or international level. The committee will also be interested in applicants’ ideas around proposed strategic partnerships and pilot projects for their fellowships. Neither Code for Africa nor Open Knowledge are being prescriptive about the proposed focus or scope for projects, but will prefer projects that demonstrate clear visions with tangible outputs. This could include fellows working with a specific government department or agency to make a key dataset available. It could also include helping communities use available data, or organising a series of events addressing a specific topic or challenge citizens are currently facing.

Successful candidates will commit to work on their fellowship activities a minimum of six days a month, including attending online and offline training, organising events, and being an active member both Open Knowledge and Code for Africa communities.

While the pilot fellowships are limited to 16 countries initially, we are exploring ways to expand it to other regions. Get in touch if you would like to work with us to do so.

Do you have questions? See more about the Fellowship Programme here and have a looks at this Frequently Asked Questions (FAQ) page. If this doesn’t answer your question, email us at Katelyn[dot]Rogers[at]okfn.org

Not sure if you fit the profile? Drop us a line!

Convinced? Apply now to become a Open Government fellow. If you would prefer to submit your application in French or Portuguese, translations of the application form are available in French here and in Portuguese here.

The application will be open until the 15th of December 2014 and the programme will start in February 2015. We are looking forward to hearing from you!

Joint Submission to UN Data Revolution Group

Rufus Pollock - October 16, 2014 in Featured, News, Open Data, Open Government Data, Policy

The following is the joint Submission to the UN Secretary General’s Independent Expert Advisory Group on a Data Revolution from the World Wide Web Foundation, Open Knowledge, Fundar and the Open Institute, October 15, 2014. It derives from and builds on the Global Open Data Initiative’s Declaration on Open Data.

To the UN Secretary General’s Independent Expert Advisory Group on a Data Revolution

Societies cannot develop in a fair, just and sustainable manner unless citizens are able to hold governments and other powerful actors to account, and participate in the decisions fundamentally affecting their well-being. Accountability and participation, in turn, are meaningless unless citizens know what their government is doing, and can freely access government data and information, share that information with other citizens, and act on it when necessary.

A true “revolution” through data will be one that enables all of us to hold our governments accountable for fulfilling their obligations, and to play an informed and active role in decisions fundamentally affecting their well-being.

We believe such a revolution requires ambitious commitments to make data open; invest in the ability of all stakeholders to use data effectively; and to commit to protecting the rights to information, free expression, free association and privacy, without which data-driven accountability will wither on the vine.

In addition, opening up government data creates new opportunities for SMEs and entrepreneurs, drives improved efficiency and service delivery innovation within government, and advances scientific progress. The initial costs (including any lost revenue from licenses and access charges) will be repaid many times over by the growth of knowledge and innovative data-driven businesses and services that create jobs, deliver social value and boost GDP.

The Sustainable Development Goals should include measurable, time-bound steps to:

1. Make data open by default

Government data should be open by default, and this principle should ultimately be entrenched in law. Open means that data should be freely available for use, reuse and redistribution by anyone for any purpose and should be provided in a machine-readable form (specifically it should be open data as defined by the Open Definition and in line with the 10 Open Data Principles).

  • Government information management (including procurement requirements and research funding, IT management, and the design of new laws, policies and procedures) should be reformed as necessary to ensure that such systems have built-in features ensuring that open data can be released without additional effort.
  • Non-compliance, or poor data quality, should not be used as an excuse for non-publication of existing data.
  • Governments should adopt flexible intellectual property and copyright policies that encourage unrestricted public reuse and analysis of government data.

2. Put accountability at the core of the data revolution

A data revolution requires more than selective release of the datasets that are easiest or most comfortable for governments to open. It should empower citizens to hold government accountable for the performance of its core functions and obligations. However, research by the Web Foundation and Open Knowledge shows that critical accountability data such as company registers, land record, and government contracts are least likely to be freely available to the public.

At a minimum, governments endorsing the SDGs should commit to the open release by 2018 of all datasets that are fundamental to citizen-state accountability. This should include:

  • data on public revenues, budgets and expenditure;
  • who owns and benefits from companies, charities and trusts;
  • who exercises what rights over key natural resources (land records, mineral licenses, forest concessions etc) and on what terms;
  • public procurement records and government contracts;
  • office holders, elected and un-elected and their declared financial interests and details of campaign contributions;
  • public services, especially health and education: who is in charge, responsible, how they are funded, and data that can be used to assess their performance;
  • constitution, laws, and records of debates by elected representatives;
  • crime data, especially those related to human rights violations such as forced disappearance and human trafficking;
  • census data;
  • the national map and other essential geodata.

    • Governments should create comprehensive indices of existing government data sets, whether published or not, as a foundation for new transparency policies, to empower public scrutiny of information management, and to enable policymakers to identify gaps in existing data creation and collection.

 3. Provide no-cost access to government data

One of the greatest barriers to access to ostensibly publicly-available information is the cost imposed on the public for access–even when the cost is minimal. Most government information is collected for governmental purposes, and the existence of user fees has little to no effect on whether the government gathers the data in the first place.

  • Governments should remove fees for access, which skew the pool of who is willing (or able) to access information and preclude transformative uses of the data that in turn generates business growth and tax revenues.

  • Governments should also minimise the indirect cost of using and re-using data by adopting commonly owned, non-proprietary (or “open”) formats that allow potential users to access the data without the need to pay for a proprietary software license.

  • Such open formats and standards should be commonly adopted across departments and agencies to harmonise the way information is published, reducing the transaction costs of accessing, using and combining data.

4. Put the users first

Experience shows that open data flounders without a strong user community, and the best way to build such a community is by involving users from the very start in designing and developing open data systems.

  • Within government: The different branches of government (including the legislature and judiciary, as well as different agencies and line ministries within the executive) stand to gain important benefits from sharing and combining their data. Successful open data initiatives create buy-in and cultural change within government by establishing cross-departmental working groups or other structures that allow officials the space they need to create reliable, permanent, ambitious open data policies.

  • Beyond government: Civil society groups and businesses should be considered equal stakeholders alongside internal government actors. Agencies leading on open data should involve and consult these stakeholders – including technologists, journalists, NGOs, legislators, other governments, academics and researchers, private industry, and independent members of the public – at every stage in the process.

  • Stakeholders both inside and outside government should be fully involved in identifying priority datasets and designing related initiatives that can help to address key social or economic problems, foster entrepreneurship and create jobs. Government should support and facilitate the critical role of both private sector and public service intermediaries in making data useful.

5. Invest in capacity

Governments should start with initiatives and requirements that are appropriate to their own current capacity to create and release credible data, and that complement the current capacity of key stakeholders to analyze and reuse it. At the same time, in order to unlock the full social, political and economic benefits of open data, all stakeholders should invest in rapidly broadening and deepening capacity.

  • Governments and their development partners need to invest in making data simple to navigate and understand, available in all national languages, and accessible through appropriate channels such as mobile phone platforms where appropriate.

  • Governments and their development partners should support training for officials, SMEs and CSOs to tackle lack of data and web skills, and should make complementary investments in improving the quality and timeliness of government statistics.

6. Improve the quality of official data

Poor quality, coverage and timeliness of government information – including administrative and sectoral data, geospatial data, and survey data – is a major barrier to unlocking the full value of open data.

  • Governments should develop plans to implement the Paris21 2011 Busan Action Plan, which calls for increased resources for statistical and information systems, tackling important gaps and weaknesses (including the lack of gender disaggregation in key datasets), and fully integrating statistics into decision-making.

  • Governments should bring their statistical efforts into line with international data standards and schemas, to facilitate reuse and analysis across various jurisdictions.

  • Private firms and NGOs that collect data which could be used alongside government statistics to solve public problems in areas such as disease control, disaster relief, urban planning, etc. should enter into partnerships to make this data available to government agencies and the public without charge, in fully anonymized form and subject to robust privacy protections.

7. Foster more accountable, transparent and participatory governance

A data revolution cannot succeed in an environment of secrecy, fear and repression of dissent.

  • The SDGs should include robust commitments to uphold fundamental rights to freedom of expression, information and association; foster independent and diverse media; and implement robust safeguards for personal privacy, as outlined in the UN Covenant on Civil and Political Rights.

  • In addition, in line with their commitments in the UN Millennium Declaration (2000) and the Declaration of the Open Government Partnership (2011), the SDGs should include concrete steps to tackle gaps in participation, inclusion, integrity and transparency in governance, creating momentum and legitimacy for reform through public dialogue and consensus.


Colophon

This submission derives and follows on from the Global Open Data Inititiave’s Global Open Data Declaration which was jointly created by Fundar, Open Institute, Open Knowledge and World Wide Web Foundation and the Sunlight Foundation with input from civil society organizations around the world.

The full text of the Declaration can be found here:

http://globalopendatainitiative.org/declaration/

This Index is yours!

Heather Leson - October 9, 2014 in Community, Open Data, Open Data Census, Open Data Census, Open Data Index

How is your country doing with open data? You can make a difference in 5 easy steps to track 10 different datasets. Or, you can help us spread the word on how to contribute to the Open Data Index. This includes the very important translation of some key items into your local language. We’ll keep providing you week-by-week updates on the status of the community-driven project.

We’ve got a demo and some shareable slides to help you on your Index path.

Priority country help wanted

The amazing community provided content for over 70 countries last year. This year we set the bar higher with a goal of 100 countries. If you added details for your country last year, please be sure to add any updates this year. Also, we need some help. Are you from one of these countries? Do you have someone in your network who could potentially help? Please do put them in touch with the index team – index at okfn dot org.

DATASETS WANTED: Armenia, Bolivia, Georgia, Guyana, Haiti, Kosovo, Moldova, Morocco, Nicaragua, Ukraine, and Yemen.

Video: Demo and Tips for contributing to the Open Data Index

This is a 40 minute video with some details all about the Open Data Index, including a demo to show you how to add datasets.

Text: Tutorial on How to help build the Open Data Index

We would encourage you to download this, make changes (add country specific details), translate and share back. Please simply share on the Open Data Census Mailing List or Tweet us @okfn.

Thanks again for sharing widely!

Open Definition v2.0 Released – Major Update of Essential Standard for Open Data and Open Content

Rufus Pollock - October 7, 2014 in Featured, News, Open Content, Open Data, Open Definition

Today Open Knowledge and the Open Definition Advisory Council are pleased to announce the release of version 2.0 of the Open Definition. The Definition “sets out principles that define openness in relation to data and content” and plays a key role in supporting the growing open data ecosystem.

Recent years have seen an explosion in the release of open data by dozens of governments including the G8. Recent estimates by McKinsey put the potential benefits of open data at over $1 trillion and others estimates put benefits at more than 1% of global GDP.

However, these benefits are at significant risk both from quality problems such as “open-washing” (non-open data being passed off as open) and from fragmentation of the open data ecosystem due to incompatibility between the growing number of “open” licenses.

The Open Definition eliminates these risks and ensures we realize the full benefits of open by guaranteeing quality and preventing incompatibility.See this recent post for more about why the Open Definition is so important.

The Open Definition was published in 2005 by Open Knowledge and is maintained today by an expert Advisory Council. This new version of the Open Definition is the most significant revision in the Definition’s nearly ten-year history.

It reflects more than a year of discussion and consultation with the community including input from experts involved in open data, open access, open culture, open education, open government, and open source. Whilst there are no changes to the core principles, the Definition has been completely reworked with a new structure and new text as well as a new process for reviewing licenses (which has been trialled with governments including the UK).

Herb Lainchbury, Chair of the Open Definition Advisory Council, said:

“The Open Definition describes the principles that define “openness” in relation to data and content, and is used to assess whether a particular licence meets that standard. A key goal of this new version is to make it easier to assess whether the growing number of open licenses actually make the grade. The more we can increase everyone’s confidence in their use of open works, the more they will be able to focus on creating value with open works.”

Rufus Pollock, President and Founder of Open Knowledge said:

“Since we created the Open Definition in 2005 it has played a key role in the growing open data and open content communities. It acts as the “gold standard” for open data and content guaranteeing quality and preventing incompatibility. As a standard, the Open Definition plays a key role in underpinning the “open knowledge economy” with a potential value that runs into the hundreds of billions – or even trillions – worldwide.”

What’s New

In process for more than a year, the new version was collaboratively and openly developed with input from experts involved in open access, open culture, open data, open education, open government, open source and wiki communities. The new version of the definition:

  • Has a complete rewrite of the core principles – preserving their meaning but using simpler language and clarifying key aspects.
  • Introduces a clear separation of the definition of an open license from an open work (with the latter depending on the former). This not only simplifies the conceptual structure but provides a proper definition of open license and makes it easier to “self-assess” licenses for conformance with the Open Definition.
  • The definition of an Open Work within the Open Definition is now a set of three key principles:
    • Open License: The work must be available under an open license (as defined in the following section but this includes freedom to use, build on, modify and share).
    • Access: The work shall be available as a whole and at no more than a reasonable one-time reproduction cost, preferably downloadable via the Internet without charge
    • Open Format: The work must be provided in a convenient and modifiable form such that there are no unnecessary technological obstacles to the performance of the licensed rights. Specifically, data should be machine-readable, available in bulk, and provided in an open format or, at the very least, can be processed with at least one free/libre/open-source software tool.
  • Includes improved license approval process to make it easier for license creators to check conformance of their license with the Open Definition and to encourage reuse of existing open licenses

More Information

  • For more information about the Open Definition including the updated version visit: http://opendefinition.org/
  • For background on why the Open Definition matters, read the recent article ‘Why the Open Definition Matters’

Authors

This post was written by Herb Lainchbury, Chair of the Open Definition Advisory Council and Rufus Pollock, President and Founder of Open Knowledge

Get Updates