Support Us
Open Knowledge Foundation Open Knowledge
Foundation Blog

You are browsing the archive for Open Government Data.

Avatar of Theodora Middleton

by Theodora Middleton

Hurricane Sandy and open data

November 1, 2012 in Exemplars, News, Open Data, Open Government Data

It is not an immediately obvious partnership, and yet open data and crisis response go together incredibly well. As storms have lashed the East coast of the US in recent days, causing tragic loss of life and enormous financial damage, many of the tools which have helped citizens to track its path and stay safe have been built on the back of open government data. Just as with the Open Street Map community’s response in the Haiti disaster, we find that with open data at their fingertips, civic hackers and developers are able to build useful tools in an emergency with a speed that far outstrips what centralised government agencies are able to produce.

Check out the Google Crisis Map of Hurricane Sandy, which predicts the future of the storm in real time, including power outages; or the New York Times’s evacuation map. Or if you’re a coder wanting to work with others in the tech community, check out HurricaneHackers who are working on projects and resources for Sandy. Alex Howard is tracking the datastorm here.

He writes:

When natural disasters loom, public open government data feeds become critical infrastructure … it’s key to understand that it’s government weather data, gathered and shared from satellites high above the Earth, that’s being used by a huge number of infomediaries to forecast, predict and instruct people about what to expect and what to do.

And New York City’s Chief Digital Officer, Rachel Haot, wrote to TechCrunch:

Open data is critical in crisis situations because it allows government to inform and serve more people than it ever could on its own through conventional channels. By making data freely available in a usable format for civic-minded developers and technology platforms, government can exponentially scale its communications and service delivery.

We’ve set up a CKAN group for data related to Sandy here: http://thedatahub.org/group/sandy-response-data

If you’re interested in contributing, there are some useful links to get started with here.

3 Comments »

Avatar of Guillermo Moncecchi

by Guillermo Moncecchi

Towards a public digital infrastructure: why do governments have a responsibility to go open?

November 1, 2012 in Featured, Ideas and musings, Open Government Data, WG Open Government Data

The most common argument in favor of open data is that it enhances transparency, and while the link may not always be causal, it is certainly true that both tend to go hand-in-hand. But there is another, more expansive perspective on open government data: that it is part of an effort to build public infrastructure.

Does making a shapefile available with all Montevideo’s traffic lights make Montevideo’s government more transparent? We don’t think so. But one of our duties as civil servants is building the city infrastructure. And we should understand that data is mainly infrastructure. People do things on it, as they do things on roads, bridges or parks. For money, for amusement, for philanthropy, there are myriads of uses for infrastructure: we should not try to determine or even guess which those uses are. We must just provide the infrastructure and ensure it will be available. Open data should be seen as a component of an effort to build a public digital infrastructure, where people could, within the law, do whatever they want. Exactly as they do with roads.

When you see open data in this light, several decisions become easier. Should we ask people for identification to give them our data? Answer: do you ask them for an identification to use the street? No, you don’t – then no, you shouldn’t. Why should we use open, non proprietary standards for publishing data? For the same reason you do not build a street where only certain car brands can pass. What happens if there are problems with my data, which causes problems for the users? Well, you will be liable, if the law decides that … but, would you avoid demands for accidents caused by pavement problems by not building streets? Of course you are responsible for your data: you are paid to create it, as you are paid for building bridges. Every question about open data we can imagine has already been answered for traditional infrastructure.

But of course the infrastructure required to enable people to create an information society goes beyond data. We will give you four examples.

The most direct infrastructure component is hardware and communications. The Uruguayan government recognises this, and is planning to have each home connected with fibre by then end of 2015, with 1 Gb traffic for free for everybody with a phone line. Meanwhile since 2007, every public school child gets an OLPC laptop and internet connection. This programme should be understood as being primarily about infrastructure: education encompasses much more than laptops, but infrastructure enables the development of new education paths.

Secondly, services. Sometimes it’s better to provide services than to provide data. Besides publishing cartography data, in Montevideo we provide WMS and WFS services to retrieve a map just using a URL. Services, as data, should be open: no registration, no access limit. Open services allow developers to use not only government data, but also government computation power, and, of course, government knowledge: the knowledge needed to, say, estimate the arrival time of a bus.

Thirdly, sometimes services are not enough, and we have to develop complete software components to enable public servants to do their work. Sometimes these software components should also be part of the public digital infrastructure. The people of Brazil are very clear on this: in 2007 they developed the Portal do Software Publico Brasileiro, where applications developed by or for the government are publicly available. Of course, this is not a new concept: its general version is called open source software. We believe that within this framework of public infrastructure, the discussion between open source and privative software makes no sense. Nobody would let a company be the owner of a street. If is public, it should be open.

Finally, there is knowledge. We, as the government, must tell the people what we are doing, and how we are doing it. Our knowledge should be open. We have the duty to publish our knowledge and to let others use it, so that we can participate actively in communities, propose changes, and act as an innovation factor in every task we face. Because we are paid for that: for building knowledge infrastructure.

We do not think government should let others do its work: on the contrary, we want a strong government, building the blocks of infrastructure to achieve its tasks, and making this infrastructure available to people to do whatever they want, within the law.

Exactly the same thing they do with streets.

5 Comments »

Avatar of Velichka Dimitrova

by Velichka Dimitrova

Hack4Health: London 2-4 November

October 22, 2012 in Events, External, Open Data, Open Government Data, Policy, Sprint / Hackday, Workshop

In the first November weekend – 2-4 November – the UK Open Data Institute in London will host Hack4Health, organised by Coadec, Healthbox Accelerator, the Cabinet Office, NHS Hackday and the Open Knowledge Foundation. The event brings together entrepreneurs, developers and technical startups working on health and fitness data to create innovative solutions and products.

After a weekend-long hacking all participating teams will present their ideas to a panel of experts with the chance to get one-on-one mentorship and other prizes. Some of the judges include Dr Ben Goldacre – author of Bad Science and Bad Pharma and Dr Carl Reynolds  - Co-Founder of Open Health Care UK. The winning team would also present their project at the launch of the Open Data Institute to Tim Berners-Lee.

 

Participants will be inspired by existing digital health startups, leading industry representatives and mentors throughout the weekend.

Are you a developer, designer, health professional, a mentor or a member of a start-up. Register to attend here: http://hack4health.co.uk/apply-to-attend/

This event will take place at the The Open Data Institute: 65 Clifton St, The City, London Borough of Hackney, EC2A UK: http://theodi.org/events/hack4health-2012

For any further questions please contact : info [at] hack4health.co.uk

Comments Off

Avatar of Theodora Middleton

by Theodora Middleton

US Congress data opened

October 9, 2012 in External, Featured Project, Open Government Data, WG Open Government Data

Exciting news on open legislative data from the US. Eric Mills (from the Sunlight Foundation), Josh Tauberer (of GovTrack.us) and Derek Willis have been beavering away on a public domain scraper and dataset from THOMAS.gov, the official source for legislative information for the US Congress. They’ve just hit a key milestone – the incorporation of everything that THOMAS has on Bills going back to 1973 when its records began!

Eric says:

We’ve published and documented all of this data in bulk, and I’ve worked it into Sunlight’s pipeline, so that searches for bills in Scout use data collected directly from this effort.

The data and code are all hosted on Github on a “unitedstates” organization, which is right now co-owned by me, Josh, and Derek – the intent is to have this all exist in a common space. To the extent that the code needs a license at all, I’m using a public domain “unlicense” that should at least be sufficient for the US (other suggestions welcome).

There’s other great stuff in this organization, too – Josh made an amazing donation of his legislator dataset, and converted it to YAML for easy reuse. I’ve worked that dataset into Sunlight’s products already as well. I’ve also moved my legal citation extractor into this organization — and my colleague Thom Neale has an in-progress parser for the US Code, to convert it from binary typesetting codes into JSON.

Github’s organization structure actually makes possible a very neat commons. I’m hoping this model proves useful, both for us and for the public.

2 Comments »

Avatar of admin

by admin

Amendments Liberated: new features for Parltrack

October 1, 2012 in Featured Project, Open Government Data, WG EU Open Data, WG Open Government Data

The following guest post is by Stef.

The European Parliament is one of the most notoriously impenetrable institutions that governs our lives. Shining a light into the murky corridors of Brussels and Strasbourg becomes increasingly vital as the reach of the Parliament grows. Opening up the EU to greater citizen scrutiny will help to improve understanding, participation, and democratic legitimary. Parltrack is one of a number of initiatives seeking to make different aspects of the European Union more digestible, in this case focussing on the legislative process.

Parltrack is a website that republishes detailed information of the European law-making process. It combines dossiers, MEPs, vote results and committee agendas into a unique database and allows the tracking of dossiers using email and RSS. Some of the data – like results of votes – comes from hard-to-process PDF documents. Recently two projects – the European Parliament’s own AT4AM and the German bundesgit – showed the need to have access to the amendments to legislative proposals in an easier to use format. Parltrack now offers this information. The newly added data allows Parltrack to display all the amendments a Member of the European Parliament (MEP) has made in the current parliamentiary term. Such a listing was unavailable to the public until now. Similarly new is the listing of all amendments for a certain law propsoal. Surprisingly, the new feature most warmly welcomed by Parltrack’s users is the ability to send direct links to amendments. This not only allows more direct discussion of the text, but also tweeting.

Parltrack also offers tracking of events concerning any legislative proposal. Users can sign up to get notifications if a proposal is scheduled on a committee, or if amendments are attached to it.

It’s important to note that this data contains errors. Current estimates are around 1%, which come from the fact that the PDFs sometimes themselves contain spelling and formatting errors – in one case the English version contains French text. So this is an informational source – anything serious should be cross-checked with the source PDF which is always linked.

Parltrack currently contains 171612 amendments starting from 14th of July, 2009. Included in this are 976 amended dossiers, and 775 amending MEPs. Some more statistics on the data:

Top 3 most amending MEPS:

  1. Olle SCHMIDT: 2038 amendments
  2. Philippe LAMBERTS: 1974 amendments
  3. Silvia-Adriana ŢICĂU: 1610 amendments

Top 3 dossiers with the most amendments:

  1. 3075: Structural instruments: common provisions for ERDF, ESF, Cohesion Fund, EAFRD and EMFF; general provisions applicable to ERDF, ESF and Cohesion Fund (2011/0276(COD))
  2. 2482: Common Fisheries Policy (2011/0195(COD))
  3. 2310: Public procurement (2011/0438(COD))

Come and check it out!

Comments Off

Avatar of Jonathan Gray

by Jonathan Gray

“Rest assured, the EU is behind you” says European Commissioner Neelie Kroes to OKFestival participants

September 20, 2012 in Events, Featured, OKFest, Open Access, Open Content, Open Data, Open Government Data, Policy

If you have more than a passing interest in EU policies related to the internet, digital content and digital technologies then you’ve probably heard of Neelie Kroes, Vice President of the European Commission and Commissioner for the Digital Agenda.

Today Neelie gave a virtual address for participants at OKFestival, one of the largest open knowledge events to date. The event has convened over 800 open data, open content and open access advocates in Helsinki for six days of talks, workshops and hackathons.

In her talk Neelie highlights the European Commission’s commitment to policies, projects and funding to support open data and open access, including for public sector information, cultural data, science and research.

Her address closes with the following remarks:

I know you at the OKFestival don’t need convincing about the benefits of openness, nor about the huge innovation that it can fuel. Rest assured, the EU is behind you.

You can watch the video of Neelie’s address and read the full transcript below.

Video address to participants at Open Knowledge Festival, Helsinki, 20th September 2012

Data is a 21st century commodity: it’s the new oil. There’s almost no limit to the economic and social wonders it can generate: new applications and new tools appear every day.

That’s why the Commission has an ambitious Open Data Strategy. To really make the most of this huge asset within our single market. Here are four things we’re doing in particular.

First, we’ve proposed new legislation to open up public sector information. So businesses and citizens can more easily access and use this great resource, across the EU: without complicated or costly conditions. For the first time, the scope also includes cultural institutions. And the Commission will be practising what it preaches: by putting our own data on a single online portal, with free and easy access.

All this takes a big culture change – but I’m confident that the countries of the EU can look ahead to the huge opportunity, and support our proposal. After all, opening up public sector data could generate economic gains around €40 billion a year, and that’s not something anyone can ignore right now.

Second, you may be aware of exciting recent developments for cultural open data. Over recent years, the EU has promoted Europeana as the access point for Europe’s libraries, museums, galleries and archives. It’s a treasure trove of cultural heritage; and a creative hub.

So I’m delighted that, just last week, Europeana announced it was putting the metadata for over 20 million exhibits into the public domain: using the creative commons CC-zero open licence.

This is a step change in open data access, and an international first. Moving away from a closed and controlled approach to this kind of data, to one based on large-scale, free re-use. Where you can link it up, for example with tourism or broadcasting data, and find ever more creative uses.

In the open and fertile environment that Europeana has now created, I hope we will see many more apps blooming. Just imagine the new applications for libraries, schools, or hotels: all powered by Europeana. So a big thank you to those cultural institutions that had the vision to take us there.

Third, openness also benefits science. Helping scientists collaborate and progress – and helping citizens, businesses and research funders benefit. We’ve proposed to make available, under open access, all publications that stem from EU-funded research. And to progressively open access to the data from scientific experiments and studies too. This is a great way to enable a new reality for science – and I’m glad the EU is playing its part.

Fourth, we will of course continue to support research related to open data. Like the LOD2 project, using open source software to help people publish linked data. Likewise, over the next two years we will invest €45 million on open access infrastructures for science, and on digital preservation. I hope that projects like this will continue under Horizon 2020, the next generation of EU research and innovation funding.

I know you at the OKFestival don’t need convincing about the benefits of openness, nor about the huge innovation that it can fuel. Rest assured, the EU is behind you.

Have a great Festival!

2 Comments »

Avatar of Jindřich Mynarz

by Jindřich Mynarz

Recycle public sector data with the Big Clean on November 3rd 2012 in Prague

September 17, 2012 in Events, OKF Czech Republic, Open Data, Open Government Data

Public sector data lives a short life. Its life spans the life of applications that are hidden deep inside of public bodies. Tied to application-specific data formats, the data dies with the application that hosts it. During its lifetime the data stays within the public sector, serving a few predetermined purposes, while the ability to access and make effective use of the data is often restricted to civil servants only.

The Big Clean is a conference for those who want to change that. Its goal is to empower its participants with the skills to recycle public sector data, so that it is open to anyone who wants to put it to use. Its goal is to share the knowledge how to make the life of public sector data longer and richer.

Recycling public sector data makes its life-cycle longer. The Big Clean will focus on the techniques that prevent the data used in the public sector to be wasted. The public may acquire the data by the means of screen-scraping, improve its quality by data refining and use it as a source for data-driven stories that journalists and other can write. These are the key topics of the Big Clean:

  • Screen-scraping — the skill of distilling data out of web pages and other poorly structured sources
  • Data refining — the techniques of transforming raw data into usable data
  • Data-driven journalism — the craft of telling stories with data

The idea of the Big Clean builds on the past. It dates back to the Open Government Data Camp in the fall of 2010, when its concept, originally conceived by Antti Poikola, was shaped during one of the camp’s workshops. This year’s Big Clean follows up on the topics laid out by the previous Big Clean in 2011. You can read about the experiences from the Big Clean on the OKFN blog.

Unlike the Big Clean in 2011, this year’s Big Clean is meant to be a truly international event. Based on the popularity growth of the Big Clean’s core topics, we want to make it bigger and better. We invited leading experts and important voices to talk about the key aspects of recycling public sector data from the viewpoints of screen-scraping, data refining, and data-driven journalism. To keep the event open to a wider audience, the event’s language will be English and no admission fee will be charged.

Practical Details

Do you want to know more? Read up on the Big Clean on its web site or follow @BigCleanCZ on Twitter.

Comments Off

Avatar of Naomi Lillie

by Naomi Lillie

#OpenDataEDB 3

September 14, 2012 in Bibliographic, Events, Join us, Linked Open Data, Meetups, OKF, OKScotland, Open Data, Open GLAM, Open Government Data

Amidst the kerfuffle and cacophony of the Fringe Festival packing up for another year, the Edinburgh contingent came together again to meet, greet, present and argue all aspects of Open Data and Knowledge.

OKFN Meet-ups are friendly and informal evenings for people to get together to share and debate all areas of openness. Depending on the number of people on a given evening, we have presentations and/or round-table discussions about Open Knowledge and Open Data – from politics and philosophy to the practicalities of theory and practice. We have had two previous events (see here for the ‘launch’ write-up and here for the invitation to the second instalment); this time we were kindly hosted by the Informatics Forum, and the weather stayed fine enough to explore the roof terrace (complete with vegetable garden, gizmos to record wind-speed and weather, a view across the city to Arthur’s Seat and even a blue moon).

Around 20 of us gathered together and presentations were given by the following people:

  • James Baster – Open Tech Calendar: an introduction to this early-stage project to bring tech meet-ups together, talk about the different ways we are trying to be open and ask for feedback and help;
  • Ewan Klein – a short overview of business models for Open Data, including for government bodies;
  • Gordon Dunsire – library standards and linked data;
  • Gill Hamilton – National Library of Scotland’s perspective of library standards and open data;
  • Bob Kerr – State of the Map Scotland (see here for Bob’s featured OKFN blog post);
  • Naomi Lillie – OKFN as part of the Scottish Open effort.

What struck me overall was that everybody already knows each-other… As well as cross-over in the talks, I kept trying to introduce people who would exclaim, “Ah yes! How was the holiday / conference / wedding?” or similar. This was quite useful, though, as it emphasised the point I made in my talk: OKFN doesn’t need to start anything in Scotland, as efforts towards Open are already ongoing and to great effect, we just want to provide support and possibly a brand under which these activities can be coordinated and promoted. With this in mind, we are going to look into a Scotland OKFN group as soon as things settle down again after OKFest – keep your eyes open for updates to follow!

To keep up-to-date with #OpenDataEDB and similar events, with the above and other interesting folks, and with the emerging Scotland OKFN group:

1 Comment »

Avatar of Mark Wainwright

by Mark Wainwright

Building a data portal with CKAN

September 5, 2012 in CKAN, Open Government Data

A while ago, Augusto Hermann wrote on this blog about a unique civic engagement project: the participatory process of building a government data portal in Brazil. The site, dados.gov.br, is still going strong, and Augusto has now written over on the CKAN blog about the process of building and deploying it using CKAN, the Open Knowledge Foundation’s free, open-source Data Management System for publishing data.

Augusto reports:

Our experience using CKAN has been positive throughout, from deploying an instance to work on extensions and translations. The beta version was created in a one-day open sprint of development on the portal (pictured below). The excellent installation documentation then meant that a person without much experience of Python or systems administration was able to put it online in a few hours.

Read the whole post here.

1 Comment »

Avatar of National Democratic Institute

by National Democratic Institute

Introducing the Declaration on Parliamentary Openness

August 31, 2012 in Campaigning, External, Featured, OKFest, Open Government Data

This guest post is by Scott Hubli and Andrew G. Mandelbaum from the National Democratic Institute (NDI). NDI is partering with the Sunlight Foundation and the Latin American Network for Legislative Transparency to enhance networking among parliamentary monitoring organizations on issues of parliamentary openness and democratic reform, with the support of the Omidyar Network, the Open Society Foundation and the National Endowment for Democracy.

In joining the Open Government Partnership (OGP), governments have been going on record in support of more open, participatory and accountable public institutions. To date, 55 governments have agreed to develop national action plans, with civil society input and monitoring, that commit them to this agenda. Although the quality of commitments made and intensity of civil society engagement may vary by country, OGP has stimulated a valuable discussion about the importance of citizen engagement in the political and governance processes—one that the international community of parliamentary monitoring organizations (PMOs) is advancing in the legislative realm through the Declaration on Parliamentary Openness.

As many commentators have noted, OGP has largely ignored parliaments and the issue of open legislative data, despite an increasing recognition that open data policies should be designed to support improved democracy, not just improved service delivery. Because the action plans are initiated by the executive branch, parliamentary buy-in may be something of an afterthought. Yet, as the institution responsible for approving laws, representing citizens and overseeing executive performance and policy implementation, parliaments are essential sources of public information. They are also critical venues for giving voice to citizens’ interests and ensuring that their needs are reflected in laws and their implementation. It’s no wonder that M. Steven Fish calls parliaments “a—or even the—institutional key to democracy.”

Recognizing the potential for improved collaboration with parliaments to deepen their commitments to openness and to citizen engagement in parliamentary work, more than 70 PMOs from over 50 countries — and counting — have expressed support for the Declaration on Parliamentary Openness. They have also established a network at OpeningParliament.org. In recent years, PMOs have commenced monitoring of legislative institutions around the world to improve citizen understaning of parliaments, and to advocate and support parliamentary efforts to become more representative, accessible, accountable and responsive. While some PMOs gather members of parliament (MPs) and citizens for face-to-face discussions on policy issues, others develop technologies to help citizens understand and use parliamentary information or contact their MPs. In all instances, PMOs’ abilities to promote parliamentary openness and engagement depend on access to parliamentary information.

Given the variety of challenges faced by PMOs in accessing parliamentary information, the Declaration on Parliamentary Openness highlights measures parliaments can take to promote a culture of parliamentary openness — recognizing the basic principle of public ownership of parliamentary information. The Declaration also specifies the categories of parliamentary information that should be made available to the public and the channels through which parliamentary information should be made accessible to ensure non-discriminatory public access. Recognizing the importance of providing parliamentary information in open and structured formats, the Declaration also contains specific “open data” provisions to enable electronic analysis, reuse and sharing of parliamentary information.

Forthcoming commentary, which will accompany the Declaration’s launch, highlights a number of instances of parliaments making concerted efforts to become more open. Italy’s Chamber of Deputies, which is co-hosting this year’s World e-Parliament Conference, is a leader in linked open data for parliaments, while Brazil’s lower house has had success generating citizen participation in the legislative process through it’s e-Democracia website. The U.S. and U.K. are among small group of parliaments making select information available in open and structured formats. Most excitingly, several parliaments in Africa and Latin America are playing with a promising open source software based on the Akoma Ntoso schema for legal documents that is currently undergoing the OASIS standardization process. The forthcoming World e-Parliament Report 2012 and the Global Parliamentary Report describe a number of online and offline measures parliaments have taken to improve openness and participation.

However, many parliaments have yet to benefit from these advances. In many countries, parliaments could do more to take advantage of opportunities presented by modern technology to engage citizens or provide information in open and structured formats that facilitate analysis and reuse. A number of parliaments still fail to provide access to basic information that would allow citizens to meaningfully participate in the legislative process, such as the text of draft legislation, hearing schedules and parliamentary votes. The increasing interconnectedness of the world’s citizens through social media, the Internet and mobile technology is rapidly changing citizens’ expectations for good governance and what parliaments must deliver. Parliaments that improve openness and participation can reinforce public confidence at a time when citizens around the world are becoming increasingly demanding of their representative institutions.

The Declaration will be formally launched at the upcoming World e-Parliament Conference, on the International Day of Democracy, and will be published in multiple languages. Members of the PMO community will also welcome dialogue and collaboration during the Transparency and Accountability track at OK Festival on Tuesday 18th September, and at a meeting to discuss future collaboration among PMOs on Wednesday the 19th. Interested individuals can also signal their support for the declaration here.

Comments Off

Please create an account to get started.

Sign up to the Open Knowledge Newsletter

Get Updates