Support Us

You are browsing the archive for Open Standards.

Newsflash! OKFestival Programme Launches

Beatrice Martini - June 4, 2014 in Events, Free Culture, Join us, Network, News, OKFest, OKFestival, Open Access, Open Data, Open Development, Open Economics, Open Education, Open GLAM, Open Government Data, Open Humanities, Open Knowledge Foundation, Open Knowledge Foundation Local Groups, Open Research, Open Science, Open Spending, Open Standards, Panton Fellows, Privacy, Public Domain, Training, Transparency, Working Groups

At last, it’s here!

Check out the details of the OKFestival 2014 programme – including session descriptions, times and facilitator bios here!

Screen Shot 2014-06-04 at 4.11.42 PM

We’re using a tool called Sched to display the programme this year and it has several great features. Firstly, it gives individual session organisers the ability to update the details on the session they’re organising; this includes the option to add slides or other useful material. If you’re one of the facilitators we’ll be emailing you to give you access this week.

Sched also enables every user to create their own personalised programme to include the sessions they’re planning to attend. We’ve also colour-coded the programme to help you when choosing which conversations you want to follow: the Knowledge stream is blue, the Tools stream is red and the Society stream is green. You’ll also notice that there are a bunch of sessions in purple which correspond to the opening evening of the festival when we’re hosting an Open Knowledge Fair. We’ll be providing more details on what to expect from that shortly!

Another way to search the programme is by the subject of the session – find these listed on the right hand side of the main schedule – just click on any of them to see a list of sessions relevant to that subject.

As you check out the individual session pages, you’ll see that we’ve created etherpads for each session where notes can be taken and shared, so don’t forget to keep an eye on those too. And finally; to make the conversations even easier to follow from afar using social media, we’re encouraging session organisers to create individual hashtags for their sessions. You’ll find these listed on each session page.

We received over 300 session suggestions this year – the most yet for any event we’ve organised – and we’ve done our best to fit in as many as we can. There are 66 sessions packed into 2.5 days, plus 4 keynotes and 2 fireside chats. We’ve also made space for an unconference over the 2 core days of the festival, so if you missed out on submitting a proposal, there’s still a chance to present your ideas at the event: come ready to pitch! Finally, the Open Knowledge Fair has added a further 20 demos – and counting – to the lineup and is a great opportunity to hear about more projects. The Programme is full to bursting, and while some time slots may still change a little, we hope you’ll dive right in and start getting excited about July!

We think you’ll agree that Open Knowledge Festival 2014 is shaping up to be an action-packed few days – so if you’ve not bought your ticket yet, do so now! Come join us for what will be a memorable 2014 Festival!

See you in Berlin! Your OKFestival 2014 Team

Draft Open Data Policy for Qatar

Rayna Stamboliyska - April 24, 2014 in Open Government Data, Open Knowledge Foundation Local Groups, Open MENA, Open Standards, Policy

The following post was originally published on the blog of our Open MENA community (Middle East and North Africa).

The Qatari Ministry of Information and Communication Technologies (generally referred to as ictQATAR) had launched a public consultation on its draft Open Data Policy. I thus decided to briefly present a (long overdue) outline of Qatar’s Open Data status prior to providing a few insights of the current Policy document.

Public sector Open Data in Qatar: current status

Due to time constraints, I did not get the chance to properly assess public sector openness for the 2013 edition of the Open Data Index (I served as the MENA editor). My general remarks are as follows (valid both end of October 2013 and today):

  • Transport timetables exist online and in digital form but are solely available through non-governmental channels and are in no way available as Open Data. The data is thus neither machine-readable nor freely accessible — as per the Open Definition, — nor regularly updated.
  • Government budget, government spending and elections results are nowhere to be found online. Although there are no elections in the country (hence no election results to be found; Qatar lacks elected Parliament), government budget and spending theoretically exist.
  • Company register is curated by the Qatar Financial Centre Authority, is available online for anyone to read and seems to be up-to-date. Yet, the data is not available for download in anything other than PDF (not a machine-readable format) and is not openly licensed which severely restricts any use one could decide to make out of it.
  • National statistics seem to be partly available online through the Qatar Information Exchange office. The data does not, however, seem to be up-to-date, is mostly enclosed in PDFs and is not openly licensed.
  • Legislation content is provided online by Al-Meezan, the Qatari Legal Portal. Although data seems available in digital form, it does not seem to be up-to-date (no results for 2014 regardless of the query). The licensing of the website is not very clear as the mentions include both “copyright State of Qatar” and “CC-by 3.0 Unported”.
  • Postcodes/Zipcodes seem to be provided through the Qatar Postal Services yet the service does not seem to provide a list of all postcodes or a bulk download. The data, if we assume it’s available, is not openly licensed.
  • National map at a scale of 1:250,000 or better (1cm = 2.5km) is nowhere to be found online, at least I did not manage to (correct me if I am wrong).
  • Emissions of pollutants data is not available through the Ministry of Environment. (Such data is defined as “aggregate data about the emission of air pollutants, especially those potentially harmful to human health. “Aggregate” means national-level or more detailed, and on an annual basis or more often. Standard examples of relevant pollutants would be carbon monoxides, nitrogen oxides, or particulate matter.”)

This assessment would produce an overall score of 160 (as per the Open Data Index criteria) which would rank Qatar at the same place as Bahrain, that is much lower than other MENA states (e.g., Egypt and Tunisia). A national portal exists but it does not seem to comprehend what open format and licensing mean as data is solely provided as PDFs and Excel sheets, and is the property of the Government. (The portal basically redirects the user to the aforementioned country’s national statistics website.) Lastly, information requests can be made through the portal.

The 2013 edition of the Open Data Barometer provides a complementary insight and addresses the crucial questions of readiness and outreach:

[There is] strong government technology capacity, but much more limited civil society and private sector readiness to secure benefits from open data. Without strong foundations of civil society freedoms, the Right to Information and Data Protection, it is likely to be far harder for transparency and accountability benefits of open data to be secured. The region has also seen very little support for innovation with open data, suggesting the economic potential of open data will also be hard to realise. This raises questions about the motivation and drivers for the launch of open data portals and platforms.

Screenshot from the Open Data Barometer 2013.

2014 Open Data Policy draft

Given the above assessment, I was pleasantly surprised to discover that a draft Open Data Policy is being composed by ictQATAR. The document sets the record straight from the beginning:

Information collected by or for the government is a national resource which should be managed for public purposes. Such information should be freely available for anyone to use unless there are compelling privacy, confidentiality or security considerations by the government. [...] Opening up government data and information is a key foundation to creating a knowledge based economy and society. Releasing up government-held datasets and providing raw data to their citizens, will allow them to transform data and information into tools and applications that help individuals and communities; and to promote partnerships with government to create innovative solutions.

The draft Policy paper then outlines that “all Government Agencies will put in place measures to release information and data”. The ictQATAR will be in charge of coordinating those efforts and each agency will need to nominate a senior manager internally to handle the implementation of the Open Data policy through the identification and release of datasets as well as the follow-up on requests to be addressed by citizens. The Policy emphasizes that “each agency will have to announce its “Terms of Use” for the public to re-use the data, requirement is at no fees”.

The Policy paper also indicates how the national Open Data portal will operate. It will be “an index to serve as gateway to public for dataset discovery and search, and shall redirect to respective Government Agencies’ data source or webpage for download”. Which clearly indicates that each individual Agency will need to create own website where the data will be released and maintained.

The proposed national Open Data portal is also suggested to operate as an aggregator of “all public feedback and requests, and the government agencies’ responses to the same”. Alongside, the portal will continue to allow the public to submit information requests (as per the freedom of information framework in the country). This is an interesting de facto implementation of the Freedom of Information Act Qatar still lacks.

The draft Policy further states:

Where an Agency decides to make information available to the public on a routine basis, it should do so in a manner that makes the information available to a wide range of users with no requirement for registration, and in a non-proprietary, non-exclusive format.

This is an interesting remark and constitutes one of my main points of criticism to the proposed paper. The latter neither contains a mention about what the recommended formats should be nor about licensing. Thus, one is left wondering whether the Agencies should just continue to stick to Microsoft Excel and PDF formats. If these were adopted as the default formats, then the released data would not be truly open as none of these two formats is considered open and the files are not machine-readable (a pre-requisite for data to be defined as open). Indeed, instead of going for a lengthy description of various formats, it would have been much more useful to elaborate on preferred format, e.g. CSV.

An additional concern is the lack of mention of a license. Even though the Policy paper does a great job emphasizing that the forthcoming data needs to be open for anyone to access, use, reuse and adapt, it makes no mention whatsoever about the envisioned licensing. Would the latter rely on existing Creative Commons licenses? Or would the ictQATAR craft its own license as have done other governments across the world?

An additional reason for concern is the unclear status of payment to access data. Indeed, the Policy paper mentions at least three times (sections 4.2 (i); 4.4 (ii); Appendix 6, ‘Pricing Framework’ indicator) that the data has to be provided at no cost. Yet, the Consultation formulates the question:

Open Data should be provided free of charge where appropriate, to encourage its widespread use. However, where is it not possible, should such data be chargeable and if so, what are such datasets and how should they be charged to ensure they are reasonable?

This question indicates that financial participation from potential users is considered probable. If such a situation materialized, this would be damaging for the promising Open Data Policy as paying to access data is one of the greatest barriers to access to information (regardless of how low the fee might be). Thus, if the data is provided at a cost, it is not Open Data anymore as by definition, Open Data is data accessible at no cost for everyone.

My personal impression is that the Policy draft is a step in the right direction. Yet the success of such a policy, if implemented, remains very much dependent on the willingness of the legislator to enable a shift towards increased transparency and accountability. My concerns stem from the fact that the national legislation has precedence over ictQATAR’s policy frameworks which may make it very difficult to achieve a satisfactory Open Data shift. The Policy draft states:

Agencies may also develop criteria at their discretion for prioritizing the opening of data assets, accounting for a range of factors, such as the volume and quality of datasets, user demand, internal management priorities, and Agency mission relevance, usefulness to the public, etc.

The possibility that an Agency might decide to not open up data because it would be deemed potentially harmful to the country’s image or suchlike is real. Given that no Freedom of Information Act exists, there is no possible appeal mechanism allowing to challenge a negative decision citing public interest as outweighing deemed security concerns. The real test for how committed to openness and transparency the government and its Agencies are will come at that time.

The Appendix 6 is thus very imprecise regarding the legal and security constraints that might prevent opening up public sector data. Furthermore, the precedence of the national legislation should not be neglected: it for ex. prohibits any auditing or data release related to contracting and procurement; no tenders are published for public scrutiny. Although the country has recently established national general anti-corruption institutions, there is a lack of oversight of the Emir’s decisions. According to Transparency International Government Defence Anti-Corruption Index 2013, “the legislature is not informed of spending on secret items, nor does it view audit reports of defence spending and off-budget expenditure is difficult to measure”.

Note: I have responded to the consultation in my personal capacity (not as OpenMENA). Additional insights are to be read which I have chosen not to feature here.

The Open Definition in context: putting open into practice

Laura James - October 16, 2013 in Featured, Linked Open Data, Open Data, Open Definition, Open Knowledge Definition, Open Standards

We’ve seen how the Open Definition can apply to data and content of many types published by many different kinds of organisation. Here we set out how the Definition relates to specific principles of openness, and to definitions and guidelines for different kinds of open data.

Why we need more than a Definition

The Open Definition does only one thing: as clearly and concisely as possible it defines the conditions for a piece of information to be considered ‘open’.

The Definition is broad and universal: it is a key unifying concept which provides a common understanding across the diverse groups and projects in the open knowledge movement.

At the same time, the Open Definition doesn’t provide in-depth guidance for those publishing information in specific areas, so detailed advice and principles for opening specific types of information – from government data, to scientific research, to the digital holdings of cultural heritage institutions – is needed alongside it.

For example, the Open Definition doesn’t specify whether data should be timely; and yet this is a great idea for many data types. It doesn’t make sense to ask whether census data from a century ago is “timely” or not though!

Guidelines for how to open up information in one domain can’t always be straightforwardly reapplied in another, so principles and guidelines for openness targeted at particular kinds of data, written specifically for the types of organisation that might be publishing them, are important. These sit alongside the Open Definition and help people in all kinds of data fields to appreciate and share open information, and we explain some examples here.

Principles for Open Government Data

In 2007 a group of open government advocates met to develop a set of principles for open government data, which became the “8 Principles of Open Government Data”.

In 2010, the Sunlight Foundation revised and built upon this initial set with their Ten Principles for Opening up Government Information, which have set the standard for open government information around the world. These principles may apply to other kinds of data publisher too, but they are specifically designed for open government, and implementation guidance and support is focused on this domain. The principles share many of the key aspects of the Open Definition, but include additional requirements and guidance specific to government information and the ways it is published and used. The Sunlight principles cover the following areas: completeness, primacy, timeliness, ease of physical and electronic access, machine readability, non-discrimination, use of commonly owned standards, licensing, permanence, and usage costs.

Tim Berners-Lee’s 5 Stars for Linked Data

In 2010, Web Inventor Tim Berners-Lee created his 5 Stars for Linked Data, which aims to encourage more people to publish as Linked Data – that is using a particular set of technical standards and technologies for making information interoperable and interlinked.

The first three stars (legal openness, machine readability, and non-proprietary format) are covered by the Open Definition, and the two additional stars add the Linked Data components (in the form of RDF, a technical specification).

The 5 stars have been influential in various parts of the open data community, especially those interested in the semantic web and the vision of a web of data, although there are many other ways to connect data together.

Principles for specific kinds of information

At the Open Knowledge Foundation many of our Working Groups have been involved with others in creating principles for various types of open data and fields of work with an open element. Such principles frame the work of their communities, set out best practice as well as legal, regulatory and technical standards for openness and data, and have been endorsed by many leading people and organisations in each field.

These include:

The Open Definition: the key principle powering the Global Open Knowledge Movement

All kinds of individuals and organisations can open up information: government, public sector bodies, researchers, corporations, universities, NGOs, startups, charities, community groups, individuals and more. That information can be in many formats – it may be spreadsheets, databases, images, texts, linked data, and more; and it can be information from any field imaginable – such as transport, science, products, education, sustainability, maps, legislation, libraries, economics, culture, development, business, design, finance and more.

Each of these organisations, kinds of information, and the people who are involved in preparing and publishing the information, has its own unique requirements, challenges, and questions. Principles and guidelines (plus training materials, technical standards and so on!) to support open data activities in each area are essential, so those involved can understand and respond to the specific obstacles, challenges and opportunities for opening up information. Creating and maintaining these is a major activity for many of the Open Knowledge Foundation’s Working Groups as well as other groups and communities.

At the same time, those working on openness in many different areas – whether open government, open access, open science, open design, or open culture – have shared interests and goals, and the principles and guidelines for some different data types can and do share many common elements, whilst being tailored to the specific requirements of their communities. The Open Definition provides the key principle which connects all these groups in the global open knowledge movement.

More about openness coming soon

Don’t miss our other posts about Defining Open Data, and exploring the Open Definition, why having a shared and agreed definition of open data is so important, and how one can go about “doing open data”.

Principles for Open Contracting

Guest - June 24, 2013 in Featured Project, Open Standards, Uncategorized

The following guest post is by the Open Contracting Partnership, announcing the release of their Principles for Open Contracting. It is cross-posted from their website.

Contracts

Over the past year, the Open Contracting Partnership has facilitated a global consultation process to create a set of global principles that can serve as a guide for all of those seeking to advance open contracting around the world.

The principles reflect norms and best practices from around the world related to disclosure and participation in public contracting.

They have been created with the inputs and feedback of nearly 200 members the open contracting community from government, private sector, civil society, donor organizations, and international financial institutions. These collaborators contributed inputs from various sector-specific perspectives (such as service delivery, infrastructure, extractive industries, and land).

The Open Contracting Partnership welcomes all your questions, comments or feedback. Please contact us at partnership@open-contracting.com

OPEN CONTRACTING GLOBAL PRINCIPLES

Preamble: These Principles reflect the belief that increased disclosure and participation in public contracting will have the effects of making contracting more competitive and fair, improving contract performance, and securing development outcomes. While recognizing that legitimate needs for confidentiality may justify exemptions in exceptional circumstances, these Principles are intended to guide governments and other stakeholders to affirmatively disclose documents and information related to public contracting in a manner that enables meaningful understanding, effective monitoring, efficient performance, and accountability for outcomes. These Principles are to be adapted to sector-specific and local contexts and are complementary to sector-based transparency initiatives and global open government movements.

Affirmative Disclosure

  1. Governments shall recognize the right of the public to access information related to the formation, award, execution, performance, and completion of public contracts.
  2. Public contracting shall be conducted in a transparent and equitable manner, in accordance with publicly disclosed rules that explain the functioning of the process, including policies regarding disclosure.
  3. Governments shall require the timely, current, and routine publication of enough information about the formation, award, execution, performance, and completion of public contracts to enable the public, including media and civil society, to understand and monitor as a safeguard against inefficient, ineffective, or corrupt use of public resources. This would require affirmative disclosure of:
    1. Contracts, including licenses, concessions, permits, grants or any other document exchanging public goods, assets, or resources (including all annexes, schedules and documents incorporated by reference) and any amendments thereto;
    2. Related pre-studies, bid documents, performance evaluations, guarantees, and auditing reports.
    3. Information concerning contract formation, including:
      1. The planning process of the procurement;
      2. The method of procurement or award and the justification thereof;
      3. The scope and specifications for each contract;
      4. The criteria for evaluation and selection;
      5. The bidders or participants in the process, their validation documents, and any procedural exemptions for which they qualify;
      6. Any conflicts of interest uncovered or debarments issued;
      7. The results of the evaluation, including the justification for the award; and
      8. The identity of the contract recipient and any statements of beneficial ownership provided;
    4. Information related to performance and completion of public contracts, including information regarding subcontracting arrangements, such as:
      1. General schedules, including major milestones in execution, and any changes thereto;
      2. Status of implementation against milestones;
      3. Dates and amounts of stage payments made or received (against total amount) and the source of those payments;
      4. Service delivery and pricing;
      5. Arrangements for ending contracts;
      6. Final settlements and responsibilities;
      7. Risk assessments, including environmental and social impact assessments;
      8. Assessments of assets and liabilities of government related to the contract;
      9. Provisions in place to ensure appropriate management of ongoing risks and liabilities; and
      10. Appropriate financial information regarding revenues and expenditures, such as time and cost overruns, if any.
  4. Governments shall develop systems to collect, manage, simplify and publish contracting data regarding the formation, award, execution, performance and completion of public contracts in an open and structured format, in accordance with the Open Contracting Data Standards as they are developed, in a user-friendly and searchable manner.
  5. Contracting information made available to the public shall be as complete as possible, with any exceptions or limitations narrowly defined by law, ensuring that citizens have effective access to recourse in instances where access to this information is in dispute.
  6. Contracting parties, including international financial institutions, shall support disclosure in future contracting by precluding confidentiality clauses, drafting confidentiality narrowly to cover only permissible limited exemptions, or including provisions within the contractual terms and conditions to allow for the contract and related information to be published.
  7. Participation, Monitoring, and Oversight

  8. Governments shall recognize the right of the public to participate in the oversight of the formation, award, execution, performance, and completion of public contracts.
  9. Governments shall foster an enabling environment, which may include legislation, that recognizes, promotes, protects, and creates opportunities for public consultation and monitoring of public contracting, from the planning stage to the completion of contractual obligations.
  10. Governments shall work together with the private sector, donors, and civil society to build the capacities of all relevant stakeholders to understand, monitor and improve public contracting and to create sustainable funding mechanisms to support participatory public contracting.
  11. Governments have a duty to ensure oversight authorities, including parliaments, audit institutions, and implementing agencies, to access and utilize disclosed information, acknowledge and act upon citizen feedback, and encourage dialogue and consultations between contracting parties and civil society organizations in order to improve the quality of contracting outcomes.
  12. With regard to individual contracts of significant impact, contracting parties should craft strategies for citizen consultation and engagement in the management of the contract.

Frictionless Data: making it radically easier to get stuff done with data

Rufus Pollock - April 24, 2013 in Featured, Ideas and musings, Labs, Open Data, Open Standards, Small Data, Technical

Frictionless Data is now in alpha at http://data.okfn.org/ – and we’d like you to get involved.

Our mission is to make it radically easier to make data used and useful – our immediate goal is make it as simple as possible to get the data you want into the tool of your choice.

This isn’t about building a big datastore or a data management system – it’s simply saving people from repeating all the same tasks of discovering a dataset, getting it into a format they can use, cleaning it up – all before they can do anything useful with it! If you’ve ever spent the first half of a hackday just tidying up tabular data and getting it ready to use, Frictionless Data is for you.

Our work is based on a few key principles:

  • Narrow focus — improve one small part of the data chain, standards and tools are limited in scope and size
  • Build for the web – use formats that are web “native” (JSON) and work naturally with HTTP (plain-text, CSV is streamable etc)
  • Distributed not centralised — designed for a distributed ecosystem (no centralized, single point of failure or dependence)
  • Work with existing tools — don’t expect people to come to you, make this work with their tools and their workflows (almost everyone in the world can open a CSV file, every language can handle CSV and JSON)
  • Simplicity (but sufficiency) — use the simplest formats possible and do the minimum in terms of metadata but be sufficient in terms of schemas and structure for tools to be effective

We believe that making it easy to get and use data and especially open data is central to creating a more connected digital data ecosystem and accelerating the creation of social and commercial value. This project is about reducing friction in getting, using and connecting data, making it radically easier to get data you need into the tool of your choice. Frictionless Data distills much of our learning over the last 7 years into some specific standards and infrastructure.

What’s the Problem?

Today, when you decide to cook, the ingredients are readily available at local supermarkets or even already in your kitchen. You don’t need to travel to a farm, collect eggs, mill the corn, cure the bacon etc – as you once would have done! Instead, thanks to standard systems of measurement, packaging, shipping (e.g. containerization) and payment, ingredients can get from the farm direct to my local shop or even my door.

But with data we’re still largely stuck at this early stage: every time you want to do an analysis or build an app you have to set off around the internet to dig up data, extract it, clean it and prepare it before you can even get it into your tool and begin your work proper.

What do we need to do for the working with data to be like cooking today – where you get to spend your time making the cake (creating insights) not preparing and collecting the ingredients (digging up and cleaning data)?

The answer: radical improvements in the “logistics” of data associated with specialisation and standardisation. In analogy with food we need standard systems of “measurement”, packaging, and transport so that its easy to get data from its original source into the application where you can start working with it.

Frictionless DAta idea

What’s Frictionless Data going to do?

We start with an advantage: unlike for physical goods transporting digital information from one computer to another is very cheap! This means the focus can be on standardizing and simplifying the process of getting data from one application to another (or one form to another). We propose work in 3 related areas:

  • Key simple standards. For example, a standardized “packaging” of data that makes it easy to transport and use (think of the “containerization” revolution in shipping)
  • Simple tooling and integration – you should be able to get data in these standard formats into or out of Excel, R, Hadoop or whatever tool you use
  • Bootstrapping the system with essential data – we need to get the ball rolling

frictionless data components diagram

What’s Frictionless Data today?

1. Data

We have some exemplar datasets which are useful for a lot of people – these are:

  • High Quality & Reliable

    • We have sourced, normalized and quality checked a set of key reference datasets such as country codes, currencies, GDP and population.
  • Standard Form & Bulk Access

    • All the datasets are provided in a standardized form and can be accessed in bulk as CSV together with a simple JSON schema.
  • Versioned & Packaged

    • All data is in data packages and is versioned using git so all changes are visible and data can becollaboratively maintained.

2. Standards

We have two simple data package formats, described as ultra-lightweight, RFC-style specifications. They build heavily on prior work. Simplicity and practicality were guiding design criteria.

Frictionless Data: package standard diagram

Data package: minimal wrapping, agnostic about the data its “packaging”, designed for extension. This flexibility is good as it can be used as a transport for pretty much any kind of data but it also limits integration and tooling. Read the full Data Package specification.

Simple data format (SDF): focuses on tabular data only and extends data package (data in simple data format is a data package) by requiring data to be “good” CSVs and the provision of a simple JSON-based schema to describe them (“JSON Table Schema”). Read the full Simple Data Format specification.

3. Tools

It’s early days for Frictionless Data, so we’re still working on this bit! But there’s a need for validators, schema generators, and all kinds of integration. You can help out – see below for details or check out the issues on github.

Doesn’t this already exist?

People have been working on data for a while – doesn’t something like this already exist? The crude answer is yes and no. People, including folks here at the Open Knowledge Foundation, have been working on this for quite some time, and there are already some parts of the solution out there. Furthermore, much of these ideas are directly borrowed from similar work in software. For example, the Data Packages spec (first version in 2007!) builds heavily on packaging projects and specifications like Debian and CommonJS.

Key distinguishing features of Frictionless Data:

  • Ultra-simplicity – we want to keep things as simple as they possibly can be. This includes formats (JSON and CSV) and a focus on end-user tool integration, so people can just get the data they want into the tool they want and move on to the real task
  • Web orientation – we want an approach that fits naturally with the web
  • Focus on integration with existing tools
  • Distributed and not tied to a given tool or project – this is not about creating a central data marketplace or similar setup. It’s about creating a basic framework that would enable anyone to publish and use datasets more easily and without going through a central broker.

Many of these are shared with (and derive from) other approaches but as a whole we believe this provides an especially powerful setup.

Get Involved

This is a community-run project coordinated by the Open Knowledge Foundation as part of Open Knowledge Foundation Labs. Please get involved:


  • Spread the word! Frictionless Data is a key part of the real data revolution – follow the debate on #SmallData and share our posts so more people can get involved

Document Freedom Day 2013

Erik Albers - March 12, 2013 in Events, Featured Project, Open Standards


What is document freedom?

Have you ever been stuck with some data that you
have not been able to open because it was in a format that needs some
specific kind of software to open it? The same thing
happens tens of thousands of times each day. Can you imagine how much
knowledge exchange doesn’t happen just because sender and receiver
(intentionally or not) are using different data formats? Can you imagine how
much knowledge future generations will lose if we keep on using proprietary,
closed data formats that one day no one will ever be able to open because
the company behind it had business secrets and patents on it but then went
bankrupt?


Open Standards, on the other hand, are data formats that have an open
documentation and everyone is free to use or implement in their own
software. The first characteristic (open documentation) guarantees that now
and even in a hundred of years everybody interested can understand the data
format and read it. The second characteristic (free to use) guarantees that
now and in even in a hundred years everybody is free to write some piece of
software to give everyone else the ability to read a specific piece of data.
That is why everyone and every public institution should be using Open Standards.


This is exactly the point where our document freedom campaign comes in.
Every year on the last Wednesday of March, the Free Software Foundation
Europe
runs a global campaign that is called
“Document Freedom Day”. The aim of the campaign
is to raise awareness of the usefulness of Open Standards. Therefore we
encourage local groups to organise an event that highlights the
importance of the use of Open Standards. Last year there were more than
50 events in more than 20 countries
. This year, Document Freedom Day
(DFD) will be on the 27th of March 2013.

The most important part of the whole campaign is done by guys like you and
me! In order to celebrate information accessibility and Open Standards, we
heavily depend on local activity on public places, in universities, in
hackerspaces or everywhere you can imagine. I am pretty sure that you have
very good ideas what you can do to raise some attention.


If you are interested, please have a look at some ideas of what you can
do
and feel free to support your event with our promotion material
that you can order for no cost. You can order the material on the webpage. Finally, if you are planning some activity, don’t
forget to register your event on our events page.

Thank you very much for your attention.
Your participation in Document Freedom Day can make the difference!

Images: Last year’s audience in Jakarta; DFD around the world; Document Freedom Day in Rio de Janeiro. All CC-BY-SA

From Open Data to GovData: why the OGP matters in Germany

Theodora Middleton - February 19, 2013 in OKF Germany, Open Government Data, Open Standards

The following post is by Maria Schröder and Christian Heise from the Open Knowledge Foundation Germany. It is cross-posted (and slightly shortened) from the Open Government Partnership blog.

Germany’s official policy on transparency and accountability is lacking commitment and leadership. Disappointed by the political elites, the community is continuously trying to make the case for true open government as a means to achieving digital democracy. The Open Government Partnership (OGP) could be one part of the solution.

The road so far: German government screws up open data

Last year, the German government commissioned a study on Open Government Data. In the summer, the Interior Ministry announced it would take up one of the study’s core recommendations, and started preparing an open government data portal.

But weeks before Germany’s Interior Minister Friedrich plans to officially unveil the portal, it has become clear that the portal lacks two of the most important things for an open data portal: A coherent and compatible licensing policy, and interesting, politically relevant data sets. In other words: govdata.de is not going to be an open portal, according to the 10 principles for opening government information. Consequently, the government removed the word “open” from its title. In the words of politicians:

In order to make the portal’s priorities regarding its content even more explicit, the “Open Government Portal Gemrnay” will from now on be operated under the name “GovData – the Data Portal for Germany”

(Source: govdata.de)

The reaction of the open data community: Dropping the word ‘open’ from the national portal’s name and concept cannot be the solution

The community’s representatives issued a joint declaration on not-your-govdata.de, outlining why the German GovData platform is not acceptable in its currently planned form (even if the government is at least not “open-washing”, a lovely apt term derived from greenwashing).

The reasons for community criticism are as follows:

a. the chosen license models do not conform to internationally recognized standards

b. the portal, as seen so far, is lacking provisions for usability and security

c. there are no visible efforts to motivate users to reuse the data provided

d. the data provided is boring

One week later, more than 300 people from various backgrounds have signed the statement (http://not-your-govdata.de/en/ – come co-sign!). Considering that the open data community in Germany is still small, the signature rate is quite impressive.

Germany needs the Open Government Partnership, the Open Government Partnership needs Germany

After the first countries joined to the OGP in September 2011, an open working group on OGP Germany consisting of German NGOs and enthusiastic individuals started working to get the German government to join.

However, the German government put a unambiguous dampener on the working group’s endeavours, denying the necessity of the OGP for Germany. They claim that co-commitments on a European and national level are more important than on the global level. An open government data initiative between Germany, Switzerland, Austria and Lichtenstein (DACHLi) is meant to distract from these countries’ reluctance to join an international, US-driven knowledge exchange.

We believe that the OGP needs Germany, too. Looking at Germany’s role in the EU, a decision by Germany to join would probably have a great effect on neighbouring countries, and could add a lot of value to the initiative, increasing the opportunities for information sharing, public participation, and collaboration.

As members of the community, we are tired of being patient. Open (government) data is an essential part of government transparency, efficiency, accountability and citizen participation – topics which hundreds of thousands of Germans find critically important. Given the lack of political will from the government, it is in the community’s hands to connect these dots better.

From our point of view, the latest events have re-emphasised that Germany needs to sign the OGP declaration in order to promote true open government from the highest political level.

Image: Dodoïste on Wikimedia

Open Research Data Handbook Sprint

Velichka Dimitrova - February 15, 2013 in Open Access, Open Content, Open Data, Open Economics, Open Science, Open Standards, Our Work, WG Economics

On February 15-16 we are updating the Open Research Data Handbook to include more detail on sharing research data from scientific work, and to remix the book for different disciplines and settings. We’re doing this through an open book sprint. The sprint will happen at the Open Data Institute, 65 Clifton Street, London EC2A 4JE.

The Friday lunch seminar will be streamed through the Open Economics Bambuser channel. If you would like to participate, please see the Online Participation Hub for links to documents and programme updates. You can follow this event at the IRC channel #okfn-rbook and follow on twitter with hashtags #openresearch and #okfnrbook.

The Open Research Data Handbook aims to provide an introduction to the processes, tools and other areas that researchers need to consider to make their research data openly available.

Join us for a book sprint to develop the current draft, and explore ways to remix it for different disciplines and contexts.

Who it is for:

  • Researchers interested in carrying out their work in more open ways
  • Experts on sharing research and research data
  • Writers and copy editors
  • Web developers and designers to help present the handbook online
  • Anyone else interested in taking part in an intense and collaborative weekend of action

What will happen:

The main sprint will take place on Friday and Saturday. After initial discussions we’ll divide into open space groups to focus on research, writing and editing for different chapters of the handbook, developing a range of content including How To guidance, stories of impact, collections of links and decision tools.

A group will also look at digital tools for presenting the handbook online, including ways to easily tag content for different audiences and remix the guide for different contexts.

Agenda:

Where: 65 Clifton Street, EC2A 4JE (3rd floor – the Open Data Institute)

Friday, February 15th

  • 13:00 – 13:30: Arrival and sushi lunch
  • 13:30 – 14:30: Open research data seminar with Steven Hill, Head of Open Data Dialogue at RCUK.
  • 14:30 – 17:30: Working in teams

Friday, February 16th

  • 10:00 – 10:30: Arrival and coffee
  • 10:30 – 11:30: Introducing open research lightning talks (your space to present your project on research data)
  • 11:30 – 13:30: Working in teams
  • 13:30 – 14:30: Lunch
  • 14:30 – 17:30: Working in teams
  • 17:30 – 18:30: Reporting back

As many already registered for online participation we will broadcast the lunch seminar through the Open Economics Bambuser channel. Please drop by in the IRC channel #okfn-rbook

Partners:

OKF Open Science Working Group – creators of the current Open Research Data Handbook
OKF Open Economic Working Group – exploring economics aspects of open research
Open Data Research Network - exploring a remix of the handbook to support open social science
research in a new global research network, focussed on research in the Global South.
Open Data Institute – hosting the event

Protecting the foundations of Open Knowledge

Mike Linksvayer - February 13, 2013 in Open Definition, Open Knowledge Definition, Open Knowledge Foundation, Open Standards

The foundations of the Foundation

The Open Knowledge Definition (OKD) was one of the Open Knowledge Foundation’s very first projects: drafted in 2005, 1.0 in 2006. By stipulating what Open means, the OKD has been foundational to the OKF’s work, as illustrated by this several-years-old diagram of the Open Knowledge “stack”.

Knowing your foundations seems a must in any field, but even more so in an explosively growing and cross-disciplinary one. The OKD has kept the OKF itself on-track, as it has started and facilitated dozens of projects over the last years.

Burgeoning movements for open access, culture, data, education, government, and more have also benefited from a shared understanding of Open in face of “openwashing” on one hand, and lack of understanding on another. In either case, when works and projects claimed or intended as Open are actually closed, society loses: closed doesn’t create an interoperable commons.

A selection of OKF blog posts from the past few years illustrates how the OKD plays a low-profile but essential role in setting the standard for Open in a variety of fields:

Recent developments

In 2008
an Advisory Council was inaugurated to steward the OKD and related defintions. I joined the council later in 2008, and recently agreed to serve as its chair for a year.

Since then we’ve discussed and provided feedback on intended-open licenses, in particular an Open Government License Canada proposal, iterated on an ongoing discussion about refinements needed in the next version of the OKD, and made our processes for approving licenses – as well as new council members – slightly more rigorous.

We’ve also taken the crucial step of adding new council members with deep expertise in Public Sector Information/Open Government Data, where we expect much of the “action” in Open and intended-open licenses in the next years to be. I’m very happy to welcome:

  • Baden Appleyard, National Programme Director at AusGOAL
  • Tariq Khokhar, Open Data Evangelist at the World Bank
  • Herb Lainchbury, Citizen, Developer and Founder of OpenDataBC.ca
  • Federico Morando, Managing Director at the Nexa Center
  • Andrew Stott, Former Director for Transparency and Digital Engagement and Co-Chair of the Open Government Data Working Group at the Open Knowledge Foundation.

While many of them will be well known to many of our readers, you may find their brief bios and websites on the Advisory Council page.

It is also time to thank three former council members for their service in years past:

  • Paul Jacobson
  • Rob Styles
  • John Wilbanks

Open movements will continue to grow rapidly (unless we fail miserably). You can help ensure we succeed splendidly! We could always use more help reviewing and providing feedback on licenses, but there are also roles for designers, programmers, translators, writers, and people committed to sound open strategy. See a recent get involved update for more.

Most of all, make sure your open access / culture / education / government / science project is truly open — OpenDefinition.org is a good place for you and your colleagues to start!

Dutch PhD-workshop on research design, open access and open data

Velichka Dimitrova - February 1, 2013 in Open Access, Open Economics, Open Standards

This blog post is written by Esther Hoorn, Copyright Librarian, University of Groningen, the Netherlands. It is cross-posted from the Open Economics Blog.

If Roald Dahl were still alive, he would certainly be tempted to write a book about the Dutch social psychologist Diederik Stapel. For not only did he make up the research data to support his conclusions, but also he ate all the M&M’s, which he bought with public money for interviews with fictitious pupils in fictitious high schools. In the Netherlands the research fraud by Stapel was a catalyst to bring attention to the issue of research integrity and availability of research data. A new generation of researchers needs to be aware of the policy on sharing research data by the Dutch research funder NWO, the EU policy and the services of DANS, the Dutch Data archiving and networked services. In the near future, a data management plan will be required in every research proposal.

Verifiability

For some time now the library at the University of Groningen is organizing workshops for PhDs to raise awareness on the shift towards Open Access. Open Access and copyright are the main themes. The question also to address verifiability of research data came from SOM, the Research Institute of the Faculty of Economics and Business. The workshop is given as part of the course Research Design of the PhD program. The blogpost Research data management in economic journals proved to be very useful to get an overview of the related issues in this field.

Open Access

As we often see, Open Access was a new issue to most of the students. Because the library buys licenses the students don’t perceive a problem with access to research journals. Moreover, they are not aware of the big sums that the universities at present pay to finance access exclusively for their own staff and students. Once they understand the issue there is a strong interest. Some see a parallel with innovative distribution models for music. The PhDs come from all over the world. And more and more Open Access is addressed in every country of the world. One PhD from Indonesia mentioned that the Indonesian government requires his dissertation to be available through the national Open Access repository. Chinese students were surprised by availability of information on Open Access in China.

Assignment

The students prepared an assignment with some questions on Open Access and sharing research data. The first question still is on the impact factor of the journals in which they intend to publish. The questions brought the discussion to article level metrics and alternative ways to organize the peer review of Open Access journals.

Will availability of research data stimulate open access?

Example of the Open Access journal Economics

The blogpost Research data management in economic journals presents the results of the German project EdaWax, European Data Watch Extended. An important result of the survey points at the role of association and university presses. Especially it appears that many journals followed the data availability policy of the American Economic Association.

[quote] We found out that mainly university or association presses have high to very high percentages of journals owning data availability policies while the major scientific publishers stayed below 20%.

Out of the 29 journals with data availability policies, 10 used initially the data availability policy implemented by the American Economic Review (AER). These journals either used exactly the same policy or a slightly modified version.

For students it is assuring to see how associations take up their role to address this issue. An example of an Open Access journal that adopted the AER policy is Economics. And yes, this journal does have an impact factor in the Social Science Citation Index and also the possibility to archive the datasets in the Dataverse Network.

Re-use of research data for peer review

One of the students suggested that the public availability of research data (instead or merely research findings) may lead to innovative forms of review. This may facilitate a further shift towards Open Access. With access to underlying research data and methodologies used, scientists may be in a better position to evaluate the quality of the research conducted by peers. The typical quality label given by top and very good journals may then become less relevant, over time.
It was also discussed that journals may not publish a certain numbers of papers in a volume released e.g. four times a year, but rather as qualifying papers are available for publication throughout the year. Another point raised was that a substantial change in the existing publication mechanics will likely require either top journals or top business schools to lead the way, whereas associations of leading scientists in a certain field may also play an important role in such conversion.

Get Updates