Support Us

You are browsing the archive for Open Spending.

Just Released: “Where Does Europe’s Money Go? A Guide to EU Budget Data Sources”

Jonathan Gray - July 2, 2015 in Data Journalism, Featured, open knowledge, Open Spending, Policy, Research, Where Does My Money Go

The EU has committed to spending €959,988 billion between 2014 and 2020. This money is disbursed through over 80 funds and programmes that are managed by over 100 different authorities. Where does this money come from? How is it allocated? And how is it spent?

Today we are delighted to announce the release of “Where Does Europe’s Money Go? A Guide to EU Budget Data Sources”, which aims to help civil society groups, journalists and others to navigate the vast landscape of documents and datasets in order to “follow the money” in the EU. The guide also suggests steps that institutions should take in order to enable greater democratic oversight of EU public finances. It was undertaken by Open Knowledge with support from the Adessium Foundation.

Where Does Europe's Money Go?

As we have seen from projects like Farm Subsidy and journalistic collaborations around the EU Structural Funds it can be very difficult and time-consuming to put together all of the different pieces needed to understand flows of EU money.

Groups of journalists on these projects have spent many months requesting, scraping, cleaning and assembling data to get an overview of just a handful of the many different funds and programmes through which EU money is spent. The analysis of this data has led to many dozens of news stories, and in some cases even criminal investigations.

Better data, documentation, advocacy and journalism around EU public money is vital to addressing the “democratic deficit” in EU fiscal policy. To this end, we make the following recommendations to EU institutions and civil society organisations:

  1. Establish a single central point of reference for data and documents about EU revenue, budgeting and expenditure and ensure all the information is up to date at this domain (e.g. at a website such as ec.europa.eu/budget). At the same time, ensure all EU budget data are available from the EU open data portal as open data.
  2. Create an open dataset with key details about each EU fund, including name of the fund, heading, policy, type of management, implementing authorities, link to information on beneficiaries, link to legal basis in Eur-Lex and link to regulation in Eur-Lex.
  3. Extend the Financial Transparency System to all EU funds by integrating or federating detailed data expenditures from Members States, non-EU Members and international organisations. Data on beneficiaries should include, when relevant, a unique European identifier of company, and when the project is co-financed, the exact amount of EU funding received and the total amount of the project.
  4. Clarify and harmonise the legal framework regarding transparency rules for the beneficiaries of EU funds.
  5. Support and strengthen funding for civil society groups and journalists working on EU public finances.
  6. Conduct a more detailed assessment of beneficiary data availability for all EU funds and for all implementing authorities – e.g., through a dedicated “open data audit”.
  7. Build a stronger central base of evidence about the uses and users of EU fiscal data – including data projects, investigative journalism projects and data users in the media and civil society.

Our intention is that the material in this report will become a living resource that we can continue to expand and update. If you have any comments or suggestions, we’d love to hear from you.

If you are interested in learning more about Open Knowledge’s other initiatives around open data and financial transparency you can explore the Where Does My Money Go? project, the OpenSpending project, read our other previous guides and reports or join the Follow the Money network.

Where Does Europe’s Money Go - A Guide to EU Budget Data Sources

Presenting public finance just got easier

Tryggvi Björgvinsson - March 20, 2015 in CKAN, Open Spending

This blog post is cross-posted from the CKAN blog.

mexico_ckan_openspending

CKAN 2.3 is out! The world-famous data handling software suite which powers data.gov, data.gov.uk and numerous other open data portals across the world has been significantly upgraded. How can this version open up new opportunities for existing and coming deployments? Read on.

One of the new features of this release is the ability to create extensions that get called before and after a new file is uploaded, updated, or deleted on a CKAN instance.

This may not sound like a major improvement but it creates a lot of new opportunities. Now it’s possible to analyse the files (which are called resources in CKAN) and take them to new uses based on that analysis. To showcase how this works, Open Knowledge in collaboration with the Mexican government, the World Bank (via Partnership for Open Data), and the OpenSpending project have created a new CKAN extension which uses this new feature.

It’s actually two extensions. One, called ckanext-budgets listens for creation and updates of resources (i.e. files) in CKAN and when that happens the extension analyses the resource to see if it conforms to the data file part of the Budget Data Package specification. The budget data package specification is a relatively new specification for budget publications, designed for comparability, flexibility, and simplicity. It’s similar to data packages in that it provides metadata around simple tabular files, like a csv file. If the csv file (a resource in CKAN) conforms to the specification (i.e. the columns have the correct titles), then the extension automatically creates the Budget Data Package metadata based on the CKAN resource data and makes the complete Budget Data Package available.

It might sound very technical, but it really is very simple. You add or update a csv file resource in CKAN and it automatically checks if it contains budget data in order to publish it on a standardised form. In other words, CKAN can now automatically produce standardised budget resources which make integration with other systems a lot easier.

The second extension, called ckanext-openspending, shows how easy such an integration around standardised data is. The extension takes the published Budget Data Packages and automatically sends it to OpenSpending. From there OpenSpending does its own thing, analyses the data, aggregates it and makes it very easy to use for those who use OpenSpending’s visualisation library.

So thanks to a perhaps seemingly insignificant extension feature in CKAN 2.3, getting beautiful and understandable visualisations of budget spreadsheets is now only an upload to a CKAN instance away (and can only get easier as the two extensions improve).

To learn even more, see this report about the CKAN and OpenSpending integration efforts.

Newsflash! OKFestival Programme Launches

Beatrice Martini - June 4, 2014 in Events, Free Culture, Join us, Network, News, OKFest, OKFestival, Open Access, Open Data, Open Development, Open Economics, Open Education, Open GLAM, Open Government Data, Open Humanities, Open Knowledge Foundation, Open Knowledge Foundation Local Groups, Open Research, Open Science, Open Spending, Open Standards, Panton Fellows, Privacy, Public Domain, Training, Transparency, Working Groups

At last, it’s here!

Check out the details of the OKFestival 2014 programme – including session descriptions, times and facilitator bios here!

Screen Shot 2014-06-04 at 4.11.42 PM

We’re using a tool called Sched to display the programme this year and it has several great features. Firstly, it gives individual session organisers the ability to update the details on the session they’re organising; this includes the option to add slides or other useful material. If you’re one of the facilitators we’ll be emailing you to give you access this week.

Sched also enables every user to create their own personalised programme to include the sessions they’re planning to attend. We’ve also colour-coded the programme to help you when choosing which conversations you want to follow: the Knowledge stream is blue, the Tools stream is red and the Society stream is green. You’ll also notice that there are a bunch of sessions in purple which correspond to the opening evening of the festival when we’re hosting an Open Knowledge Fair. We’ll be providing more details on what to expect from that shortly!

Another way to search the programme is by the subject of the session – find these listed on the right hand side of the main schedule – just click on any of them to see a list of sessions relevant to that subject.

As you check out the individual session pages, you’ll see that we’ve created etherpads for each session where notes can be taken and shared, so don’t forget to keep an eye on those too. And finally; to make the conversations even easier to follow from afar using social media, we’re encouraging session organisers to create individual hashtags for their sessions. You’ll find these listed on each session page.

We received over 300 session suggestions this year – the most yet for any event we’ve organised – and we’ve done our best to fit in as many as we can. There are 66 sessions packed into 2.5 days, plus 4 keynotes and 2 fireside chats. We’ve also made space for an unconference over the 2 core days of the festival, so if you missed out on submitting a proposal, there’s still a chance to present your ideas at the event: come ready to pitch! Finally, the Open Knowledge Fair has added a further 20 demos – and counting – to the lineup and is a great opportunity to hear about more projects. The Programme is full to bursting, and while some time slots may still change a little, we hope you’ll dive right in and start getting excited about July!

We think you’ll agree that Open Knowledge Festival 2014 is shaping up to be an action-packed few days – so if you’ve not bought your ticket yet, do so now! Come join us for what will be a memorable 2014 Festival!

See you in Berlin! Your OKFestival 2014 Team

Bonding with Hong Kong and upcoming Open Spending

Heather Leson - May 16, 2014 in Events, Featured, OKF Hong Kong, Open Spending

Learning and sharing across the global Open Knowledge community are the two core purposes of our regular Community Sessions.

odhk - logo

This week Mart van de Ven and Bastien Douglas joined us to share all about the Open Data Hong Kong community.

Some of the key lessons they advised are: ask your community for help more, have regular events, translation is key and be ready for longer term engagement. Mart, Bastien and the ODHK folks: Have a great Longitudinal Hack!

See more about Open Data Hong Kong.

Next Community Session: All about OpenSpending

Around the world, citizens are getting involved in OpenSpending. So, far there are OpenSpending activities in 66 countries resulting in 735 datasets and 25207863 entries.

Join Anders Pedersen, Community Manager for OpenSpending to learn more about this project and how you can get involved.

  • Date: Wednesday, May 28. 2014
  • Time: 10:00 – 11:00 EDT/14:00-15:00 UTC (See worldtimebuddy.com for your timezone)
  • How to Register (G+)

Join the OpenSpending community See some Spending Stories.

We will record this.

NOTE: We are booking June 2014 Community Sessions. Contact heatherDOTleson AT OKFN DOT org if you have an idea, discussion or skillshare.

Talk soon!

Mapping the Open Spending Data Community

Neil Ashton - January 6, 2014 in Featured, Open Spending

Mapping the Open Spending Data Community

We’re pleased to announce the official release of “Mapping the Open Spending Data Community” by Anders Pedersen and Lucy Chambers, an in-depth look at how citizens, journalists, and civil society organisations around the world are using data on government finances to further their civic missions.

The investigation began in 2012 with three goals:

  • To identify Civil Society organisations (CSOs) around the world who are interested in working with government financial data
  • To connect these CSOs with each other, with open data communities, and with other key stakeholders to exchange knowledge, experiences, and best practices in relation to spending data
  • To discover how CSOs currently work with spending data, how they would like to use it, and what they would like to achieve

This report is the result. It brings together key case studies from organisations who have done pioneering work in using technology to work with public finance data in each of budgets, spending, and procurements, and it presents a curated selection of tools and other advice in an appendix.

As part of this research, we’ve also produced a four-part video series “Athens to Berlin“, which you can watch to meet some of the fascinating characters in the world of CSOs working with government spending data and to learn firsthand about their successes and their challenges.

Originally Published on the Open Spending Blog Jan 3rd, 2014

Launching Spending Stories: How much is it really?

Anders Pedersen - November 21, 2013 in Data Journalism, Featured, Open Spending, Releases

spendingstories

Spending Stories is a new way to put spending figures in their proper perspective. Developed by the Open Knowledge Foundation and Journalism++ with funding from the Knight Foundation, Spending Stories is an app that helps citizens and journalists understand and compare amounts in stories from the news.

When we hear that the UK’s school meals programme costs £6 million, what does that really mean? It means, for one thing, that it costs about a fifth of the annual spending on the monarchy.

Spending Stories draws out comparisons between amounts of money, giving users a context in which to understand how money is being spent across society while referencing the original news stories.

Users can enter a figure into Spending Stories and get a scale visualisation showing how it compares with spending stories from the app’s database.

£700,000: scale visualisation

The app displays the big picture, and users can then click through to a card visualisation that shows how the amount relates to specific stories.

£700,000: card visualisation

Users can filter stories to only show amounts that relate to the user’s interests, for example aid or energy.

Filtering stories

If users find news stories of interest, they can contribute these to the database in three easy steps and share them.

Contribute new data

Due to the good availability of UK spending data in OpenSpending, this first release of Spending Stories focuses on the UK. Spending Stories is, however, an open source project and can easily be forked and translated into other languages.

We hope to help Spending Stories sites launch on their own and expand with new features and local news stories. At launch, we are already in touch with Open Knowledge Foundation Japan about the potential deployment of Spending Stories in Japanese.

If you would like to know more about the options for setting up a local Spending Stories site, get in touch.

budzeti.ba: Following the money in Bosnia and Herzegovina

Neil Ashton - October 29, 2013 in Open Spending, Tools in Use

We’re pleased to announce the launch of budzeti.ba, a new way to understand public spending in Bosnia and Herzegovina (BiH). budzeti.ba is a joint project of Bosnian civil society advocates CPI and the Open Knowledge Foundation. It allows Bosnian citizens to navigate the complexities of the BiH tax system and to explore the distribution of public funds across administrative regions and categories of social spending. budzetiba budzeti.ba is designed to make public spending accessible to every Bosnian citizen. Users can inspect the details of national spending on categories like health and education through a simple point-and-click interface, and they can dig down into regional spending priorities by clicking through an interactive map. budzeti.ba’s tax calculator feature allows Bosnians to learn how much they contribute to public services and where it goes by specifying their home region and their monthly income. No specialized knowledge is required to use budzeti.ba or to understand its intuitive presentation of spending.

The budzeti.ba site brings together data from several sources, such as government websites and the Official Gazette of Bosnia and Herzegovina. The site’s initial launch is based on data from 2012 and will be updated as new data becomes available. CPI intends to deepen budzeti.ba’s regional coverage by including municipal data and to enrich its analysis of government spending with data from the Open Budget Survey, infographics on debt and deficit, and more.

This blog post is cross-posted from the Open Spending blog – to find out more about the Open Spending community, join their mailing list.

Visualizing How the Brazilian Government Underspends on the Public Good

Christian Villum - August 22, 2013 in OKF Brazil, Open Knowledge Foundation Local Groups, Open Spending

This post is authored by Vitor Batista, who works as developer for the Open Knowledge Foundation, and Neil Ashton, Data Roundup Editor for the School of Data blog. It is cross-posted from the PBS Ideas and OpenSpending blogs.

Brazilian NGO INESC (Institute of Socio-Economic Studies) and Open Knowledge Foundation Brasil want Brazilians to participate in the allocation of their public spending and ensure that it is used to construct a free, fair, and sustainable society.

That’s why we partnered to create Orçamento ao seu Alcance, a site which presents the execution of the Brazilian federal budget in an interactive and intuitive form.

We used OpenSpending as our database. This made it easier to focus and develop our visualizations without the need for setting up additional infrastructure for data hosting, and it made the data readily available in an accessible way.

What’s the project about?

Millions of Brazilians pay the taxes that fund the federal budget, but few actually understand it. Most are unaware of Brazil’s unjust regressive tax regime and of the scale of the losses to the public through misallocation. The information they need to understand these realities is simply not available in a comprehensible form. By building Orçamento ao seu Alcance, we hope to change that.

Orçamento ao seu Alcance’s development focused particularly on the issue of underspending. All Brazilian public bodies spend less money than is allocated to them, to varying degrees. The Ministry of Education, for example, left 16.3% of its budget (about US$ 6.1 billion) unspent in 2012, and the Ministry of Culture only spent 47.5% of its budget in 2012. If Brazilians’ needs were really being met – if every Brazilian who wanted to study had access to good public schools, for example – this underspending would not be a problem. But that is far from the case; in fact less than 1% of schools have an ideal infrastructure (a problem we have explored previously). To explore and highlight the problem, and we created a special-purpose data visualization.

How we used OpenSpending

Orçamento ao seu Alcance took data collected by SIGA Brasil, an aggregator for the many systems used by the Brazilian government to organize budget data, and added it to the OpenSpending database. Using OpenSpending freed us from creating our own database and allowed us to use the OpenSpending API to construct visualizations and a full-text search system.

Visualizing underspending

We designed our own graph to tackle the problem of underspending. The result is a time series graph that combines bars, lines, and an area. The site constructs such a graph for each budgetary unit, showing how its budget and spending compare for a given year.

Orçamento ao seu Alcance: underspending

The blue area in the graph represents the total budget – which, as you can see, changes over the year. Each red bar shows how much was spent in a particular month, and the red line tracks total spending. The distance from the red line to the tip of the blue area gives the share of the budget remaining to be spent. The amount remaining in December is money that is underspent.

This graph was built using NVD3, a JavaScript library with a collection of reusable charts made on top of D3.js. The data comes from OpenSpending via its Aggregate API.

Budget treemap

For the index page, we wanted to show a broad view of the budget across all public bodies. More than that, we wanted to show the amount of money used in each function and subfunction (e.g. Education and Basic Education). To do this, we used the OpenSpending treemap visualization.

Orçamento ao seu Alcance: treemap

OpenSpending allows you to create a treemap as a “widget” which can be simply dropped into a site. We used a modified version of the widget code with customized colours and a “back” button for improved navigation.

Searching

To help the user find public bodies, we implemented a search box with auto-complete using Twitter Bootstrap‘s typeahead library.

Orçamento ao seu Alcance: search

To make the search instantaneous for the user, we load all data entries as soon as the user enters the page. The OpenSpending Aggregate API once again helped with this, allowing us to get a list of all public bodies with a simple query.

Problems we had

We did run into a few problems using OpenSpending to build the site, though all of them could be overcome.

The Aggregate API only allows you to request one financial quantity (one measure) at a time. You can’t request both a budget quantity and a payment at the same time, for example. Our underspending graph ended up using three measures, requiring three requests. This is a performance problem. Because the API caches results, however, it ends up being OK – and there are already plans to support multiple measures in future versions, so this problem will soon be solved.

With the treemap visualization, our problem was that widgets are not customizable. They’re made to be dragged and dropped into a blog post or a newspaper article, not integrated into a site with its own design. To change the treemap’s colours and fonts, we had to use a modified version of the widget’s code.

Conclusions

We’re happy with how Orçamento ao seu Alcance turned out, and OpenSpending contributed a lot to its success.

For developers, OpenSpending made it possible to run the site without its own database and to publish its content in a sleek, cacheable form. For the project’s NGO supporters, using OpenSpending makes it possible to update the data without needing to deal with the site’s developers. Everyone is happy.

We hope that Orçamento ao seu Alcance will inspire other OpenSpending satellite sites that will help spread budgetary awareness around the globe.

Predicting city bankruptcies with open data: The case of Detroit

Guest - July 31, 2013 in Featured, Open Spending

This is a guest post by Marc Joffe of Public Sector Credit Solutions.

Used to Make Money Selling Baked Goods

Many have noticed that the United States last week was struck by its biggest municipal bankruptcy ever, when the City of Detroit declared bankruptcy. Less well known is the fact that Moody’s, the major credit rating agency, downgraded the City of Chicago by three notches at about the same time.

Earlier this year, I used audited financial disclosures to estimate the risk of city bond defaults, which often accompany bankruptcies, in the state of California. The research was funded by a grant from the state, but its conclusions are mine and not those of any official agency. The goal was to see whether open data collected from so called Comprehensive Annual Financial Reports (CAFR), that US governments typically file as PDFs, and open analytics (or open economic modeling), can serve as an alternative to standard credit rating agency analysis.

The model created during the research is openly available and designed to calculate default probabilities, where higher scores are worse than lower scores. Since 1940, the annual default rate for American cities has been 0.10%. In 2012 however, 2 out of 265 or 0.75% of California cities defaulted on their debt, and so that is the average score in the model. Scores substantially higher than 0.75% therefore represent heightened credit risk.

A number of people have asked me how the scoring model would have treated Detroit (Michigan) and Chicago (Illinois), which are in other US states. Here is a my response.

A Google spreadsheet containing the model is available here and embedded below. It is a modified version of our original model. I entered data from Detroit’s 2012 CAFR, which was published on December 28, 2012 and the Chicago’s 2012 CAFR which appeared more recently. Based on our open model Detroit’s probability score is 3.34%, which is worse than almost every California city in our survey. Chicago’s score is also pretty bad: at 1.77% it is worse than the score for Stockton, which was one of the two California cities to default in 2012.

The main driver of Detroit’s high default probability score is its negative general fund balance. The ratio of Detroit’s general fund balance to general fund expenditure is -27%. As reported in our April working paper general fund exhaustion, which means very low or negative general fund balances, were associated with the Vallejo, Stockton and San Bernardino bankruptcies. The situation in Detroit provides further evidence that municipal bond investors and other stakeholders would benefit by monitoring this indicator.

Although Chicago does not have a negative general fund balance, it has an annual general fund deficit and declining revenue, two of the four indicators that drive the default probability score. Chicago also has a relatively high ratio of interest and pension costs to total governmental fund revenues. When these uncontrollable costs become relatively high, bankruptcy is harder to avoid.

Assessing government default probability rates based on open data is today a challenging task, as most cities publish this data in PDF-format. Getting cities to publish such data in machine readable format, would make such research a lot easier. For the OpenSpending community the the bankruptcy of Detroit also underlines the need for addressing not only spending, but also revenue flows and liabilities.

Collecting, extracting and analyzing data from public financial disclosures can help us evaluate the credit risk of our local governments openly and transparent. This could be an important way of using the OpenSpending concept and platform.

What’s the deal with the UK government’s new spending tool?

Jonathan Gray - July 30, 2013 in Featured, Open Data, Open Spending, Policy, Public Money, Where Does My Money Go

We were pleasantly surprised to learn that this morning the UK government launched a new tool to explore UK public spending.

The ‘Government Interrogating Spending Tool’ (fear not – you the user are supposed to be the giver, not the receiver, of interrogation) or ‘GIST’ is, according to the Cabinet Office, “one of the first of its kind in the world”, giving an “unprecedented view” of public spending, which was previously “only published in clunky spreadsheet form”.

The site gives you a high level overview of quarterly departmental spending, as well as enabling you to see how the big numbers break down.

A little bit of history repeating?

“But wait!”, you might say. Doesn’t this all sound a bit familiar? Haven’t the Cabinet Office and others already released things like this in the past?

Is it really true that before today’s release citizens could only explore UK government spending in “clunky spreadsheet form”?

Around five years ago I wrote a concept note for a project called Where Does My Money Go?, which would enable citizens to explore public spending through interactive visualisations. The idea was a winner of the Cabinet Office’s 2008 Show Us A Better Way competition, and an early prototype of the project was picked up by the BBC. The Open Knowledge Foundation worked with the information designer David McCandless to create new visualisations that let citizens explore how much tax they pay towards different things every day, as well as giving an overview of regional and departmental spending.

Building on this work, our OpenSpending project now has over 14 million spending transactions from over 70 countries, 130 cities – including some of the most detailed data available on UK government spending ever published, such as COINS and all departmental spending above £25k. Some of this data is also available through the government’s Data.gov.uk Spend Browser.

Many others have also been working hard to present government spending to citizens through intuitive visualisations – such as the Guardian’s annual spending overview diagram and their budget visualisations with the Miso Project.

What’s new?

Today’s new spending tool includes two kinds of spending information which, as far as we know, have not previously been released: ‘Project Oscar’ and the ‘Quarterly Data Summary’ (which together would make a pretty great band name).

Project Oscar is the much anticipated replacement for the ‘Combined Online Information System’ (or COINS). It took years of campaigning and carefully crafted freedom of information requests before COINS was released in all (or rather most of) its glorious 120 gigabytes in July 2010. However some of the coverage claimed that it was too big and too difficult for most journalists and citizens to download and make sense of.

It seems likely that the focus of today’s release on usability and presentation will have been at least partly inspired by feedback from the 2010 release. And we’re pleased to see that Project Oscar has been able to see the light of day with such greater ease than its predecessor.

What is it good for? Not just hunting for waste

We think that releasing open data about public money is an essential step towards increasing government accountability and democratising our public institutions – and has many different benefits regardless of where you might be on the political spectrum.

However, we are disappointed to see such one-sided framing around the release, which shoehorned the manifold ways that citizens, journalists and civil society organisations might be interested in using data about public spending into a narrative that strongly focuses on efficiency, waste and cost savings.

With the headline that urges taxpayers to “join the hunt for government savings”, and with quotes that focus on “wasteful spending” and “saving money for taxpayers” the press release leaves little room for the positive characterisation of public spending, the tax system and all of the essential public services (roads, schools, hospitals), that taxpayers contribute to, and little room for many of the ways that citizens and civil society groups might use and interact with this data – beyond hunting for waste.

This very one-sided characterisation highlights the importance of enabling citizens, the media and civil society organisations to be able to use, share, republish, and make sense of spending data for themselves, rather than just taking the way official information is presented at face value. This is why our OpenSpending project strives to enable groups with lots of different views to use spending data in lots of different ways.

In any case, we’re glad to see the UK continuing to lead the world in financial transparency, proactively releasing some of the most detailed information on public money ever seen. We look forward to further developments in this area over the coming months, as the government moves beyond expenditure to focus on things like tax and company registries.

If you’re interested in exploring data on public spending, then you can join our openspending mailing list.

Get Updates