External | Open Knowledge Foundation Blog

You are browsing the archive for External.

Mapping the Republic of Letters

March 22, 2012 in External, Open GLAM, Visualization, WG Cultural Heritage, WG Humanities

The following post is crossposted from the OpenGLAM blog, and is about Stanford’s Mapping the Republic of Letters Project – one of the finest examples of what can be done with cultural heritage data and open source tools. Mapping the Republic of Letters is a collaborative, interdisciplinary humanities research project looking at 17th and 18th century correspondence, travel, and publication to trace the exchange of ideas in the early modern period and the Age of Enlightenment.

What unites the researchers involved in Mapping the Republic of Letters is the opportunity to explore historical material in a spatial context and ask big-data questions across archives: Did the Republic of Letters have boundaries? Where was the Enlightenment?

The Republic of Letters is an early modern network of intellectuals whose connections transcended generations and state boundaries. It has been described as a lost continent and debate continues about whether or not it really existed. Though the ‘letters’ of the title refers to scholarly knowledge, epistolary exchange was, in fact, the net that held this community together. Letters could be shipped around the world and shared across generations. Among our case studies, Athanasius Kircher’s correspondence network was the most widely distributed, exchanging letters with Jesuit outposts from Macau to Mexico.

Since the early stages of our project, we used open-source graphics libraries to visualize our collected data. The first step is to understand the ‘shape’ of the archive. A timeline + histogram, for example, reveals at a glance the distribution of letters in the collection over hundreds of years. And the map connecting cities as source and destination of sent letters reveals geographic “cold-spots” as well as hot-spots in the archive.

As we begin to dive in and pursue specific research questions, visualization tools in the form of maps, network graphs and charts, help us to make sense of piles of data all at once. Voltaire’s correspondence alone includes about 15,000 letters. Putting those letters on a map instantly gives us a picture of where Voltaire traveled and reveals temporal and spatial patterns in his letter-writing. And while there is no record of epistolary exchange between Voltaire and American inventor and statesman, Benjamin Franklin, a network graph of their combined correspondence quickly reveals three second degree connections.

One outcome of this project is a visualization layer to complement the well-established text-based search model for archives. To begin to really piece together a map of the Republic of Letters, we need to find a way to thread a path through the many dispersed and otherwise silo-ed correspondence archives. Another great challenge is to visually reflect the gaps, uncertainty and ambiguity in the historical record. It is often those gray areas that provide new research opportunities for humanists. In this effort we are very pleased to be working in partnership with DensityDesign Research Lab in Milan.

We have also been working closely with the Cultures of Knowledge project at Oxford. Cultures of Knowledge recently released a beta version available of their open access union catalog of early modern letters, aptly named Early Modern Letters Online. Their model is not to be the repository, but to provide a rich search layer across existing correspondence collections pointing back out to the source repository. Our friends at the Dutch 17th Circulation of Knowledge project are addressing the challenges of mining early modern correspondence for topics across many languages.

The code-base for our visualizations is open source and available for download at athanasius-project.github.com. Our code-base is available, but that is not to say that our visualizations are pret-a-porter. Since our research is devoted to knowledge production in the humanities and not software development, the code is rather idiosyncratic and constrained by our changing data model. Please contact us if you would like to learn more or would like to join the effort.

Comments Off

by Karolis Granickas

Will technology boost the fight against corruption in the Post-Soviet region?

March 22, 2012 in Events, External, Open Government Data, WG EU Open Data, WG Open Government Data

Having come across the recent UNDP study on the role of social media for enhancing public transparency and accountability in Eastern Europe (download here), one cannot help feeling optimistic about the potential to raise the level of civic empowerment and to fight corruption in the post-Soviet countries.

Looks like the Transparency Works event jointly organized by the Sunlight Foundation and Transparency International comes at the right time in the right place. The first ever in Lithuania transparency camp-like event will take place on 29 – 30 March in Vilnius.

Why Lithuania, the country which is considered to be among anti-corruption champions in the post-Soviet region but at the same time is only an average performer on the global scale (ranked 50th with the score of 4.8 at the Corruption Perception Index by Transparency International)?

The abovementioned UNDP study came up with a number of interesting and somewhat rationalizing figures.

Figure 1. The more people have access to Internet in a particular country, the less corrupt that country is perceived to be.

Lithuania is among the leading countries in the world in terms of the Internet usage. It is leading in deploying fiber-optic Internet access (FTTH) technologies with the fastest rate in Europe and sixth on the global ranking. Around 70% of the households have Internet connection. Evidently, the high levels of Internet penetration did not massively help with Corruption Perception. There is clear potential to better use the information and communication technologies to increase the public transparency and encourage e-participation in the country.

Figure 2. The higher the country’s e-government index, the less corrupt that country is perceived to be.

Lithuania is ranked 29th in the UN e-government survey 2012 – one position lower than in 2010. The main aspect pulling the score down is the fact that only a small number of citizens use e-services. While one can argue about whether demand or supply should come first, an open government that is willing to increase public transparency ideally should do more to pro-actively promote e-services.

Figure 3. The better the advocacy capacity of NGOs are in the country, the less corrupt that country is perceived to be.

The ability to distinguish between objective data and speculative information on mainstream commercial TV, radio and newspapers can significantly improve the corruption perception index. Therefore, the information collected by the civil society should be available to the wider population.

So, what do these figures suggest? There is not a lot of room to improve the Internet penetration in Lithuania and the region. However, a lot can be done with educating the government and the society about the benefits of e-participation. While the ideas of open government data (OGD) and e-democracy have gained momentum in most developed countries, the expansion of the movement to the post-Soviet space has been slow and fragmented.

Why is it important that e-democracy becomes topical in the region? Freely accessible data is becoming a must for transparent and accountable states and their institutions. Data presentation online is inherent to its publicity. It is important to identify the local communities that can constantly come up with ideas on how to achieve greater government transparency and increase civic participation in public decision-making. Such multi-stakeholder communities could more than ever contribute to a further democratization of the post-Soviet region.

The Transparency Works event will for the first time in Lithuania bring together international IT developers, think tanks and civil society actors, online marketing specialists, journalists, government officials, students, academics and others to share their experience and knowledge about how to use new technology to make the government really work for the people and contribute to greater transparency, accountability and citizen engagement in the region and beyond.

While a lot of events of this kind tend to be the place to hack and meet your fellow OGD geeks, the Transparency Works event will attempt to have the full palette of stakeholders present. Hopefully, this will serve as a qualitative attempt to spread the word about e-democracy and lay a fertile ground for open government data ideas to grow in Lithuania and the post-Soviet region.

The deadline for the registration for the Transparency Works event is 26th March at www.transparencyworks.lt/register/

1 Comment »

by Deirdre Molloy

Open Plaques: Community Powered Heritage

March 9, 2012 in External, Featured Project, Open GLAM, Public Domain, WG Cultural Heritage, WG Public Domain

This is a shortened version of a post from the OpenGLAM blog, where you can keep up-to-date with goings-on around open data in heritage and arts.

Historical plaques by their very nature are objects in the public domain, so creating a platform to collect them with the public – and for the collected data to be available for the broadest possible public use – seemed an obvious starting point. That’s why Open Plaques data has been open data from birth.

Those little historical markers dotted around buildings and other places we see everyday are physical portholes through time, connecting past and present. The caveat being, our experience of them is largely fleeting, easily forgotten. Even in the UK a myriad of bodies large and small put up plaques, and the digital data provided is mixed, often non-existent. But what if each encapsulated story was instantly accessible, its backstory and context linked? How would our experience of places change if we could knit plaques in the material world together with the fabric of the web?

‘We’ is the operative word with Open Plaques, a project born out of two basic thoughts: how could you feasibly tackle collecting all these plaques together and what could be done with the tapestry of stories then woven if the data was open?

What the journey so far has made clear to us is this: open data is great, but just being open isn’t enough. You need to be either (a) vitally useful enough to attract resources to pour into your service, or (b) interesting enough to attract sufficient people who care about what you’re doing and enjoy helping. We fall into the latter category; a plaque map doesn’t save lives or make the trains run on time after all. But without one or both of these drivers it’s just inert data, unlikely to grow and going nowhere. And even if you have (a) or (b), it’s still (c) a lot of hard work. But being a Type-B open data project, we’re also highly motivated

Our community of contributors, collaborators and supporters is an enthusiastic mish mash of people who like collecting plaques, playing with cultural data, finding out about history, supporting heritage and tourism, curating archives, exploring their area and further afield, and learning by having fun whilst discovering and mapping these noteworthy objects that dot the landscape. Having said that, the data we’re gathering is far from trivial and tells us huge amounts about our surroundings and past and present-day world. It’s a goldmine of location-based history.

Apps built from our data so far include two iPhone apps from Radical Robot and London Smartphone and another forthcoming from PlaceWhisper, a Kindle ebook ‘London’s Blue Plaques In a Nutshell’, and an optical character recognition (OCR) challenge. Our data has also been used at History HackDay 2011, and loaded into a TomTom satnav. A further new app is in the pipeline. Read more about the apps here.

These are a mixture of free and paid for services but as the core database grows, especially beyond the UK, so does the potential for other interesting re-uses. Recently Ireland has seen an increase in listings – with two sizeable datasets contributed by Limerick Civic Trust and Waterford City Council, and purely community-driven growth in Dublin. Other big clusters already exist in our New York and Toronto listings. These are historic cities that could benefit from savvy re-use of the data, whether by tourism bodies, cultural organisations or local entrepreneurs. We are here to facilitate them.

At the end of February, the histonauts2 pervasive gaming event demonstrated the scope for re-using and contributing data creatively. Held as part of Manchester Histories Festival, the organisers simply referred to our online list of unphotographed Manchester plaques, and set daily missions for people taking part in their digital treasure hunt around the city to find and photograph the plaques. They posted the resulting pictures on Flickr with CC licenses, and added the relevant machine tags. The upshot being the players augmented our data whilst tracking history in the real world, and 20% of the unphotographed plaques locally got an image!

The data is there if you want to look at it, published under ‘Public Domain Dedication and License 1.0′. Or you can help us build it – add new plaque listings and photos if you find any, or contact us about contributing to the web development. We’re a museum of the street, and we’d love to get more organisations and individuals contributing to the collection.

So feel free to unlock your inner plaquetivist!

Comments Off

by Esa Mäkinen

Finnish data journalism app contest

February 21, 2012 in Data Journalism, External, OKF Finland, Visualization

Helsingin Sanomat, Finland’s leading national paper, is organizing an article app contest to find data visualizations.

For many journalists today, it’s not a lack of open data that’s the problem, but a lack of the skills and off-the-shelf visualizations needed to make that open data useful to them.

A year ago, the Finnish government decided that in principle all data generated with taxpayer money should be free.

This has been leading to tonnes of great releases. At the beginning of May, the National Land Survey of Finland will release all its maps as open data. The National Audit Office of Finland has already released campaign funding data as a kind of API. The City of Helsinki has the Helsinki Region Infoshare project, that collects city-level data into one place. Statistics Finland are also publishing all their data openly.

Furthermore, Open data activists such as Antti Poikola and Petri Kola have been doing great work in lobbying the Government and creating a data ecosystem. An Open Knowledge Foundation chapter is about to be formed and Open Data activists are crowdsourcing Freedom of Information Act-related data requests on Tietopyynto.fi.

So we have plenty of data, but using and publishing it is still lagging behind. This is especially true with the major media outlets. Journalists are still publishing static charts with their articles online or using Google Fusion Tables to make very basic visualizations. Not very innovative.

To tackle this problem, Helsingin Sanomat is organizing a contest to find article apps.

By article apps we mean applications that can be embedded into any web site in 560×400 pixel Iframe. An article app should visualize some interesting data, with the possibility of user interaction or of displaying data inputted by the users.

There are 3000 euros worth of prizes. Developers will not lose any rights to the works they submit to the contest. The contest is open to everyone, and the deadline for submissions is the 8th April 2012. More info can be found here.

There are few limitations for the article apps, but we hope that the apps use open data. If the article app crowdsources data from the users, it would be great if the data could be exported openly.

One part of this process has been to think about the business models of open data journalism. The idea behind the article app format is to standardize at least one format in data journalism. When we have some kind of standard, it will be easier to buy and to sell data journalism.

Our suggestion is that outlets buy the license to publish an article app once with each article – regardless whether it’s published at the HS.fi site, in our iPad application or some other channel. The next time we use the same graph with different data, we would pay the license fee again. For one article the compensation would be quite low, but if the app is used hundred times, it would be higher.

This business model is still theoretical, as we have not published anything using this model. Also, the amount we would pay for one article is still unclear, as we have not had any discussions with developers. We’d love to hear your thoughts on

2 Comments »

by Liliana Bounegru

Data Journalism Awards – Call for Entries!

February 20, 2012 in Data Journalism, External

Showcase your work and win a chance to €45,000 in prizes by applying for the first ever Data Journalism Awards

In an age of overwhelming abundance of data, journalists and media organisations are learning to separate signal from noise in order to provide valuable insights to society. From the Guardian to the New York Times, La Stampa to Die Zeit, journalists and media organisations are experimenting with new ways of using data to improve reportage of complex issues and to give readers direct access to the sources behind the headlines. As Tim Berners-Lee says, “data-driven journalism is the future.”

To recognize and showcase outstanding work, as well as highlight best practices in this fast-growing field, the first international Data Journalism Awards (DJA) has been established this year. The DJA is organised by the Global Editors and is sponsored by Google. The competition is run by the European Journalism Centre.

A jury of data journalism experts and editors from all over the world, including from prestigious organisations like New York Times, Reuters, and Les Echos will award a total of €45,000 (over $55,000) to six winners. The jury is headed by Paul Steiger, founder of ProPublica.

There are three award categories awarded at both (i) national and international and (ii) local and regional levels to give a total of six prizes. The three categories are:

Data-driven investigative journalism
Data visualisation & storytelling
Data-driven applications

How to apply

The competition is open to media companies, non-profit organisations, freelancers and individuals. Applicants are welcome to submit their best data journalism projects before 10 April 2012 at http://datajournalismawards.org/ submit-your-work/.

Find out more about the competition and how to apply at datajournalismawards.org. If you have any questions about the competition get in touch with Liliana Bounegru, DJA Coordinator (bounegru [at] ejc [dot] net).

Relevant links:
Website: www.datajournalismawards.org
Facebook page: http://www.facebook.com/pages/Data-Journalism-Awards/305250662849826
Twitter: @ddjournalism, @EditorsNet
Twitter hashtag #dja

Comments Off

by Velichka Dimitrova

Effectopedia – An Open Data Project for Collaborative Scientific Research, with the aim of reducing Animal Testing

February 14, 2012 in External, Ideas and musings, Interviews, Metadata, Open Access, Open Data, Open Science, Open Standards

The following post is by Velichka Dimitrova, Coordinator of the Open Economics Working Group and Hristo Alajdov, Associate Professor at Institute of Biomedical Engineering at the Bulgarian Academy of Sciences.

One of the key problems in natural science research is the lack of effective collaboration. A lot of research is conducted by scientists from different disciplines, yet cross-discipline collaboration is rare. Even within a discipline, research is often duplicated, which wastes resources and valuable scientific potential. Furthermore, without a common framework and context, research that involves animal testing often becomes phenomenological and little or no general knowledge can be gained from it. The peer reviewed publishing process is also not very effective in stimulating scientific collaboration, mainly due to the loss of an underlying machine readable structure for the data and the duration of the process itself.

If research results were more effectively shared and re-used by a wider scientific community – including scientists with different disciplinary backgrounds – many of these problems could be addressed. We could hope to see a more efficient use of resources, an accelerated rate of academic publications, and, ultimately, a reduction in animal testing.

Effectopedia

Effectopedia is a project of the International QSAR Foundation. Effectopedia itself is an open knowledge aggregation and collaboration tool that provides a means of describing adverse outcome pathways (AOPs)¹ in an encyclopedic manner. Effectopedia defines internal organizational space which helps scientist with different backgrounds to know exactly where their knowledge belongs and aids them in identifying both the larger context of their research and the individual experts who might be actively interested in it. Using automated notifications when researchers create causal linkage between parts of the pathways, they can simultaneously create a valuable contact with a fellow researcher interested in the same topic who might have a different background or perspective towards the subject. Effectopedia allows creation of live scientific documents which are instantly open for focused discussions and feedback whilst giving credit to the original authors and reviewers involved. The review process is never closed and if new evidence arises it can be presented immediately, allowing the information in Effectopedia to remain current, while keeping track of its complete evolution.

How can this coordinate better scientific research?

The type of knowledge needed in Effectopedia requires a paradigm shift in the way research is conducted: from phenomenological to more hypothesis driven. Instead of testing an individual chemical with results often applicable only in the context of the specific experimental design, Effectopedia is targeted at defining the conditions under which certain knowledge can be transferred to other species, levels of biological organization, exposure routes, exposure durations, chemicals and so on. A key element to make this approach work is to provide a common framework where scientists with different backgrounds can contribute and encode their knowledge. Since adoption of a new multidisciplinary data representation format could be quite challenging, an ontology-enhanced natural language interface for encoding this knowledge is envisioned.

Entering the results of one’s research in a machine-readable format could enable a sharing platform with a holistic concept, which could unite scientists from different academic backgrounds in a multidisciplinary research framework. The data format of chemical scientists, biologists and toxicologists may be different, but if there is a unified data format, there can be “a common language” in science. Additionally, this could serve as a platform for students in the natural sciences to download the datasets of previous research in order to learn about working with and analysing data.

How can this enable wider access to scientific research?

Many universities in the developing world and in transitional economies are not able to pay the full subscription fees to access published work, which keeps them out of the mainstream up-to-date scientific discussions. Other experienced scientists with a lot of relevant expertise who are either retired or no longer affiliated with an academic institutions may be able to contribute their knowledge in an open online encyclopaedia. Making this encyclopedia semantic would also allow the development of multi-lingual support, enabling academic exchange between scientists who don’t even speak the same language.

How could this reduce animal testing?

The simplest way Effectopedia is envisioned to help in the reduction of unnecessary animal testing is by providing a centralized repository for open, easily searchable access to the existing knowledge. Effectopedia could also aid the design of alternative test methods by providing statistical information of what are the most used and needed tests. Once established, the adverse outcome pathways themselves could become an invaluable tool for discriminating when simple chemical or assay-based test can be used to predict whole animal effects for one or more species, eliminating the need for actual testing. Much of the research which has already been done on cells and animals could be extrapolated to other species, as many have evolved in similar ways and therefore have similar functions. Therefore, cross-species extrapolation could reduce the need for further animal testing, as scientists could use information about research that has already been done.

I am interested. How can I get involved?

If you come from the chemical, pharmaceutical, flavoring, cosmetics or many other industries, the products your company creates are often subject to regulations. A system like Effectopedia could dramatically reduce the cost for development and registration of new products without compromising your competitive edge. The ability to store your data on your own in-house server in the same format as the public data allows you to mix them during the product research phase and then send them for registration using the same format. Without spending any additional resources you can also create a better public image by publishing information that no longer provides any competitive advantage or that you are required to publish by governmental policies. You can also reduce the cost of building your in-house knowledge management system by concentrating your efforts on the custom and company specific modules, while sharing the cost and resources of development of the common modules with all involved stake holders.

Large organizations like OECD, WHO and agencies like EPA had already expressed interest in the development of Effectopedia as a tool for development and documentation of adverse outcome pathways that meet the standards for regulatory acceptance. If you or your organization see the benefit of such system please contact us and see how you can help establish Effectopedia as a an open standard for representation of adverse outcome pathways.

If you are scientist you can help us to form the initial expert panels creating the guidelines for all future adverse outcome pathways contributions. Your feedback is also valuable to us in these beta stages of development so please fill free to contact us with any comments ideas and suggestions.

Contact: aladjov [at] qsari.org

Sourceforge Link

The adverse outcome pathways (AOPs) describe the molecular interactions of a chemical with biological systems and the biological response models that document how molecular effects lead to adverse effects many levels of biological organization. ↩

Tags: Effectopedia, Open Data, science 4 Comments »

by Theodora Middleton

Linked Open Data and Low Carbon Development

January 27, 2012 in External, Open Data

The following guest post is by Denise Recheis from reegle, the clean energy info portal.

Offering multiple explanations for a concept increases understanding and using LOD allows both humans and machines to semantically connect related content. This is a huge advantage in our increasingly complex world!

Especially in the field of clean energy, the increasing availability of LOD is really beneficial. To make sense of the often complex factors contributing to climate change and the highly technical solutions thereof, as well as rapid development in national and international policy regarding these factors, access to high quality and timely information is crucial.

The clean energy info portal www.reegle.info and the energy info wiki www.openEI.org see themselves as gateways to a wealth of information regarding renewable energy, energy efficiency and climate change issues. They are hosted by REEEP (Renewable Energy and Energy Efficiency Partnership – where I work) and NREL (National Renewable Energy Laboratory) respectively. Both organizations have a strong commitment to the idea of Linked Open Data (LOD) and have been integrating the core principles of LOD into their online portals.

In an effort to increase awareness about the possibilities associated with publishing and consuming LOD, we organized a well-attended workshop in Abu Dhabi in January 2012. Alongside the event, we brought out a publication explaining the basics of LOD, as well as the first steps for any organization considering joining the LOD cloud. “Linked Open Data: The Essentials” (published by Semantic Web Company and REEEP) is available as a downloadable PDF, as well as a booklet which can be ordered.

“Linked Open Data: The Essentials” also highlights some best practice examples, two of them being reegle and OpenEI. Reegle’s country energy profiles are a prime example of mashed up open data. These dossiers present the reader with statistics, maps, general facts and policy and regulatory details in a pleasant design. The information is provided by LOD providers such as DBpedia (Wikipedia), the UN and the World Bank, OpenEI and other highly trusted sources. Reegle has also developed an extensive thesaurus covering clean energy and climate compatible development with full liked data capabilities, which is available for free to re-use as a widget or word press plugin, and which is currently used as the basis for a brand-new API. Of course reegle provides all its datasets as Linked Open Data free for re-use and provides datasets in RDF (Resource Description Framework) format and via a SPARQL endpoint on our data portal.

OpenEI (Open Energy Information) has always seen sharing as one of its key missions. The data is available in RESTful API, RDF and SPARQL, for integration into external websites. But even when browsing the site, users benefit from a variety of LOD sources which enhance and increase the information presented. For example, several definitions offered in the glossary are collected from different LOD sources and OpenEI’s country pages feature information from a variety of sources, including reegle’s country energy profiles. This is easily possible when organizations rely on LOD, because when several websites describe the same things they can all be connected and give users a more rounded picture of sometimes difficult subjects.

Our expected end-users include the educational sector, helping students across the world study laws and regulation, efficient engineering, and the latest ideas in clean energy from many different authoritative sources in a single gateway. Specialists and project developers can quickly gather valuable information about specific regions and areas focusing on energy-relevant issues.

Integrating the principles of LOD has had a pleasant side-effect which has been highlighted in the recent workshop in Abu Dhabi: sharing data is often a starting point for fruitful collaborations between organizations with a similar agenda. Sharing data very often also means sharing the work burden. Each organization can then focus on their specific areas of expertise, while freeing up resources from areas that can be taken over by other organizations. Sharing the results of such targeted efforts generates high-quality content, and makes it available to all stakeholders in renewable energy, energy efficiency and climate adaptation/mitigation.

We are committed to increasing the share of information available as LOD, and will continue to actively support other organizations thinking of joining the LOD cloud.

1 Comment »

by Theodora Middleton

LODLAM-NZ Round Up

December 20, 2011 in External, WG Cultural Heritage, WG Humanities, WG Open Bibliographic Data

The following guest post is by Jon Voss, whose projects include History Pin and Civil War Data 150.

I recently traveled to Wellington, New Zealand to take part in the National Digital Forum of New Zealand (#ndf2011), which was held at the national museum of New Zealand, Te Papa. Following the conference, the amazing team at Digital NZ hosted and organized a Linked Open Data in Libraries, Archives & Museums unconference (#lodlam). The two events were well attended by Kiwis as well as a large number of international attendees from Australia, and a few from as far as the US, UK and Germany.

When it comes to innovative digital initiatives in cultural heritage, the rest of the world has been looking to New Zealand and Australia for some time. Federated metadata exchanges and search has been happening across institutions in projects like Digital NZ and Trove. I was able to learn more about the Digital NZ APIs as well as those from Museum Victoria, Powerhouse Museum, and State Records New South Wales. In fact, the remarkable proliferation of APIs in Australasia has allowed us to consider the possibilities of Linked Open Data to harvest and build upon data held in databases in multiple institutions.

Given the extent to which tools for opening access to data have been developed here, I was surprised by the level of frustration that exists around copyright issues. There’s a clear sense that government is moving too slowly in making materials available to the public with open licensing. We talked a lot about the idea of separately licensing metadata and assets (i.e. information about a photo vs the digital copy of the photo), as has been happening across Europe and increasingly the United States. There are strong advocates within the GLAM sector (galleries, libraries, archives & museums) here, and demonstrating use cases utilizing openly licensed metadata will go far in helping to move those conversations forward with policy makers.

To that end, a session was convened to explore the possibilities of an international LODLAM project focused on World War I, the centennial commemoration of which is fast approaching. The Civil War Data 150 project we’ve been slowly moving forward in the US may provide a rough framework to build from. At least a half dozen or more libraries, archives and museums have expressed interest in participating in a WWI project already. First steps may be identifying openly licensed datasets to be contributed, key vocabularies and ontologies to apply, and ideas for visualizations that would leverage the use of Linked Open Data. For anything to happen here, someone will need to take the lead in organizing (not me, we’re still trying to build some tools around the Civil War Data 150 concept!). Good notes were posted on the LODLAM blog about the conversation and how to convene future conversations. Anyone who gets involved with this, please spread the word and keep the LODLAM community apprised of your progress and ways to contribute.

We also had a workshop on using Google Refine by Carlos Arroyo from the Powerhouse Museum, with props to the FreeYourMetadata crew. Some lively sessions dug into just what and how Linked Data is and some of the pitfalls and potentials. Another session explored the importance and potential of local vocabularies, and how they can contribute to Linked Data implementations. One great example was the vocabularies surrounding Maori artifacts (Taonga) at Te Papa, and how publishing those datasets can aid other museums around the world to better describe and provide digital access to Maori collections.

As I’ve attended various LODLAM meetups since June, I’ve noticed clear momentum from one to another as these conversations progress rapidly, with those further along helping those of us just learning. After LODLAM-DC I realized the importance of including library, archive, and museum vendors in all of these gatherings. At LODLAM-NZ I could see the potential of bringing together developers in the GLAM sector and those utilizing Linked Data in commercial settings. In places like San Francisco, where commercial interests are already leading the charge on Linked Data (which is not a bad thing) and there’s an active Semantic Web developer community, the GLAM sector may be playing catchup. But the sheer number of datasets potentially available as open data coming from the GLAM sector, together with the expertise of managing massive amounts of structured data, creates a space ripe for collaboration and experimentation, and these lines will continue to blur.

Comments Off

by Theodora Middleton

Opening Government Data in Bulgaria

December 16, 2011 in External, Open Government Data, WG EU Open Data, WG Open Government Data

The following guest post is by Boyan Yurukov, blogger and open government data activist.

In the beginning of 2011 some open data was released by the Bulgarian government on www.parliament.bg. Visitors could export information of bills and members of parliament as XML or CSV. They could also download the votes of individual MPs or parliamentary groups as Excel files. While what data was useful and an important step forward, I found problems in the format and the exported files. Also, one could find a lot more information on the website, that could not be exported as open structured data.

So I started a project to scrape the website, fix the available data, refine, enrich and link it. After several versions of the schema, the final dataset was released in the beginning of December. It contains over 11,000 data points and over 1.12 Gb of data. The items are as follows:

Profiles for each MP – general biography, previous parliamentary terms, participation in parliamentary groups, committees, “friendship groups” and delegation, supported bills, absences, external consultants, questions during plenary meetings.
Data on bills – laws, legislative proposals, decisions and official declarations.
Parliamentary groups and committees – current members and member history, proposed bills, external consultants, meeting schedule, agenda, transcripts and reports.
Parliamentary delegations and “friendship groups” – current members and member history.
Parliamentary sittings – program for the sitting with questions and legislative proposals; transcripts; voting history for each MP on each discussion point.
Parliamentary procurements – description, topic, procurement registry code.

The dataset can be downloaded as two ZIP files together with the XSD schema. The scraping scripts are also open sourced in GitHub. You can find all this open Bulgarian Parliament data on the DataHub.

Although refined, this data is not without its flaws. Some historical data on MPs’ biography and questions is missing. Also, transcripts are not structured, but in free text, making it almost impossible to parse. There is some hope that the parliamentary administration will release the transcripts in XML, but I’m not holding my breath. Currently the transcripts go back 20 years, and those back to the ’70s are being parsed and will be released soon. All other data is since 2001, except individual votes, which are since 2009.

This data can be quite useful for parliamentary journalism, but in itself consists only of raw XML files. This is why another project is being set up that aims at building a platform for analyzing and visualizing the refined dataset. It will be targeted at data journalists and visualization experts. It is sponsored by the Institute for Public Environment Development and all results will be released as open data. I hope that in the first quarter of 2012 the first beta will come out.

3 Comments »

by Theodora Middleton

SNCF launches a debate on open transport data in France

December 15, 2011 in External, Open Data, WG Open Transport

The following guest post is by Pieter Colpaert from iRail npo and Pierre Chrzanowski, and was reviewed by Regards Citoyens. Pieter and Pierre are both members of our brand new Working Group on Open Transport – watch this space for a full announcement of the working group’s activities and details on how to get involved!”

At first sight, you may think that data.sncf.com is the new open data website of the SNCF, the National Corporation of French Railways. Not yet. The company preferred to launch a consultation website before opening up its data. Anyone can add their thoughts on open transport data on data.sncf.com.

In a country struggling to involve the transport industry in the open data movement, this initiative is most welcome. After the release of data.gouv.fr, we hope transport data will soon be part of the available datasets. The lack till today of open transport data in France led independent initiatives to extract the data without authorisation, placing them in legal insecurity. A change by SNCF is therefore really welcome.

Although SNCF seems to be ready for open data, other public transport operators in France are still reluctant. RATP, the state-owned subway operator for Paris area, recently refused to let other app developers use its map for free. This inspired CheckMyMetro, a startup which was forced to remove the RATP map from its smartphone application, to organize a subway map design contest.

As a lot of organizations are launching similar debates on open data, it is important that they rightfully apply the word “open” and that while doing this they know how to gain an added value for themselves and their customers. Data.scnf.com is a great opportunity to remind the SNCF and other transport actors in Europe of the actual meaning of the word “open” and to help introduce a productive open data policy.

Open data for multimodal transport

Today, commuters use different types of transport to go to work or to travel across Europe. For them, access to timetables, networks maps and real-time transport data is the key to organize their journey or to get informed of disruptions. Multimodal transport is part of the last European Commission transport policy which has announced the launch of a contest for the best European multimodal journey planner. The software behind these intermodal journey planners can be as intelligent as can be, but when there is no data, the software is useless.

Some countries are already doing their part. The UK Government recently committed itself to the release of high-value transport data. Which also seems to provide a good input to answer the data.sncf.com consultation. Here is the comprehensive list of transport data soon to be released: - Rail timetable information on a weekly basis - Real-time running data from Network Rail - Location about Great Britain Rail Network and GB rail network stations - Traveline National Dataset on a weekly basis (Great Britain buses) - Next Buses API of planned and real-time information at 350 000 GB bus stops

There are already many journey planner apps offered either by transport companies or developed by independent developer teams, but only a few can help you to organize your journey across the whole EU – deutschebahn offers the closest. Furthermore, with open data, there are new services to come that transport companies did not think about.

Transport innovation through real open data

By starting a debate on open data, data.sncf.com wants to take the first steps towards clearing the path for innovative services. The definition of open data is clear and not debatable. As defined by the Open Knowledge Foundation: “A piece of content or data is open if anyone is free to use, reuse, and redistribute it – subject only, at most, to the requirement to attribute and share-alike”. This means data need to be released for free in an open license and available in open formats. The French statements on open data also give a clear definition of what “open” means. SNCF could then choose to open its datasets either under the new French Open License or among other open licenses available like the ODbL, already in use in different French cities. On open formats, the 5 star-ranking of the W3C is a good reference. But open transport data is part of an industry and a new market. If we want to help developers to develop multimodal apps, the respect of standards is required.

Let’s hope this initiative from the SNCF is the beginning of a real shift towards open transport data in France and beyond.

You can participate to the SNCF debate here

The ePSIplatform is also working on a report on the re-use of transport data in Europe. You can reply to their questionnaire here.

1 Comment »