Publishing Budget and Spending Open Data

This blog was written by Lorena Rivero del Paso (GIFT) and Oscar Montiel (Open Knowledge Foundation) and was originally posted on the GIFT blog.

Increasingly, we see examples where lack of transparency and accountability from governments affects trust. Being able to follow public money flows is an important step to recover trust and aim towards more effective governance of public funds. Despite this, according to the most recent edition of the Open Data Barometer, the number of national governments that publish their budget and spending reports and figures as data isn’t growing consistently.

Considering the lack of progress in such publication and the relevance of fiscal data, from Open Knowledge Foundation (OKF) and Global Initiative for Fiscal Transparency (GIFT) we have partnered to support governments in the publication of budget and spending open data, through the Open Fiscal Data Package (OFDP). This specification allows publishers to structure their data in a way that makes its description and use as easy as possible and provides visualizations and developer tools for publishers and users with the OpenSpending platform.

As a follow up to the -Towards a schema for spending Open Data, Helpdesk included-blog post where you can read more in detail about the characteristics of the (OFDP), in this post we will guide you through, for a successful publication.

First: what information is expected in a budget/spending file?

Any budget and spending open dataset should have four basic components 1) fiscal year presented 2) budget classifications 3) source of funding and 4) amounts for each stage of the transaction. Additionally, the dataset can be complemented with other relevant data and classifications included in the Financial Management Information Systems (FMIS).

1) Period

Fiscal Year- The fiscal year is the framework in which the approved budget is executed. While some countries have already developed the annual budget within a multiyear perspective, through the preparation of medium-term fiscal and budget frameworks, these frameworks are usually established at a higher level of disaggregation than the annual budget and expenditures.

The dataset can include several past and future years according to data availability. For this, each fiscal year can be a separate dataset.

2) Budget classifications

Regarding the second component, budget classifications, we refer to those indicated in the Fiscal Transparency Code of the International Monetary Fund. Furthermore, for those countries who have progressed in incorporating program classification, it should be included as well. These are:

Administrative unit (government ministries/agencies and departments/divisions within agencies).
Economic type (“inputs” such as salaries, transfers, other non-salary current expenditures, capital spending) (following the international classification if available)
Functional and subfunctions (following the international classification if available)
Program/subprogram/activity/project; alternatively outcomes and/or outputs.

Some of these are cross-classifications, mainly in respect of the Program classification.

3) Source of funding

It serves to distinguish, in the financial statements, the origin of the domestically- financed expenditures (on-budget, extra-budgetary, counterpart fund), as well as from project aid financed expenditures.

4) Stages of the transaction

For the stage of the transaction, there is not a unique form of registration, but it is important to register clearly the type of data presented and which phase of the budget the user is looking at. The Expenditure Control: Key Features, Stages, and Actors^[1] identify the next seven typical stages of the expenditure cycle, which should be considered for the dataset, according to the availability in the country:

Authorization of expenditure- A fundamental principle of public finance is that expenditure and revenue proposals must be legally authorized to ensure accountability.
Apportionment of authorization for specific periods and spending units- The purpose of apportionment is to prevent spending agencies from incurring obligations at a rate which would require the authorization of additional funds for the fiscal year in progress.
Reservation- Once the apportionment of expenditure authorization is made and the spending authority has been released, some countries’ Public Financial Management (PFM) systems include a stage at which funds are reserved for a specifically known expense but for which no contract has yet been issued. At this stage, there is no legal commitment, but it is known that the expense will be incurred during the budget year and, therefore, the reserved funds should not be used for other activities.
Commitment- The commitment stage is the point at which a potential future obligation to pay is established. A commitment occurs when a formal action, such as placing an order or awarding a contract, is taken that renders the government liable to pay at some time in the future when the order or contract is honoured by its counterpart.
Verification (or certification)- after goods have been delivered and/or services have been rendered by a supplier, an authorized officer within the spending unit concerned verifies their conformity with the contract or order, and that liability and due date of payment are recognized.
Payment order- Once checks are made to ensure that all previously stipulated controls have been performed and documented, a payment order is issued.
Payment- Once a payment order has been issued, payments are made through various instruments including checks, electronic funds transfer (EFT), and sometimes cash, in of a supplier or other recipient to discharge the liability. In line with internationally accepted good practice, the payment should be made through a Treasury Single Account (TSA) system-

5) Additional data and classifications

Geographic classification-

A representation of which part of the country benefits from each of the government financial operations. This classification is difficult in most cases, so adaptations have taken the form of classifying by location of administrative units, taxpayers, recipients of government transfers, among others.

Investment projects-

Authorized public investment projects, including Public-Private Partnerships, if available. Data over these projects can be paired with geolocation by including its latitude and longitude. Furthermore, if more data is available in the Ministry of Finance systems, such as description or links cost-benefit analysis among others, we can analyze on a case to case basis.

Contracts-

Between 15 and 25 per cent of public expenditures are exercised through contracts. These data is useful data for complete traceability of the related part of the transactions. The number and detailed data of the procurement process and/or the awarded contract can be included as part of the file. If the Country has already implemented the Open Contracting Data Standard (OCDS), an additional column can be included with the Open Contacting ID (OCID), linking both data structures without the need for duplicating data (Learn more about the OCDS here). The information on contracts in a standardised form will also allow us to link beneficiaries of these contracts and where the money goes.

Second: Structure of the Dataset

Disaggregation

The classifications should be disaggregated to the lowest level available so users can do a more specific use of the data.

Codes and descriptions

All of the classifications mentioned in the section above should always include one column for the code and one for the description for each level of disaggregation as displayed in the example below. These fields will allow the users to know what bits of the budget the dataset refers to. A clear data structure will allow the users to understand the different levels of the budget, how the programs, projects, etc are built and how money is allocated to them in the different stages of the budget.

Having these IDs and descriptions clear, will also allow mapping to the specification in a way that will later make it easy to visualise and navigate the data set once it is uploaded to the Open Fiscal Data Package.

The following image exemplifies the structure of ID + Description of the different levels of economic classification.

Horizontal structure

After analyzing use cases of the dataset and overseeing users interact with different structures, for the stages of the transaction it has been defined that each stage should be structured in one column (the other option is one column for all stages and only one column with amounts).

Third: Extension of the file

It’s common for budget or spending reports to be PDF files directly from the data. This might be easy to read for a human, but it’s not very easy to process by a computer. This is why data uploaded to OpenSpending should be produced or saved in a comma separated value (CSV) file. A CSV is the simplest machine-readable file that requires no special software to be used, like XLS that requires Excel or similar programs. It’s very light and allows you to use software that your computer may already have to start working with it. A CSV can be managed with scripts but it’s also very friendly for beginners to navigate, filter and modify without specific knowledge about databases.

You can see an example of the data from Mexico’s Federal Government below.

We have also prepared a data template where you can see the classifications and other data that will ensure quality data being published.

Fourth: Uploading the dataset using OFDP?

There are two ways of getting your budget data into OpenSpending. Both are available to everyone and can be tried today. We will discuss the two options and then compare under which circumstances you can best use which approach.

1. Upload directly with OS Packager

If you have a budget file already available as a tabular file in CSV form, you have everything you need to start using the Packager. You just need to create an account hereand you will be able to start uploading the data. If you have already published data using CKAN or any other open data platform, you can link directly to the CSV and start working with that data.

The packager tool divides the publication task into 4 steps:

Data upload
Data mapping/description
Metadata input
Data use

Each of these steps will guide you and in the end, you will have a data package. That is, a CSV file with the fields you originally had, as well as a JSON file with the description of these fields, mapped to the specification. This can be directly used in the OpenSpending Viewer.

2. Set up an Open Spending pipeline

While moving towards a more timely and disaggregated publication that would in turn be more useful for the users, there are different kinds of needs than for one time publishers. For example, to publish time series of spending we need all years spending data merged into one dataset, with this the size of the complete dataset will also be bigger. In these cases of governments that have progressed to a more advanced publication, we can use a pipeline.

A pipeline is basically a set of instructions that we provide to map the data to the specification while keeping some of the nuances of our data. This process implies writing pieces of code to perform data processing and loading to selected endpoints. This option will give a more flexible publication but would require to define the best approach along with the Helpdesk.

Which option is best for you?

There is no unique answer to this, but there are a few questions that might help guide the initial decision to begin publishing using OpenSpending.

For example,

Has your government implemented a Financial Management Integrated System in which budget and spending are registered?
Are budget and spending data stored in different systems?
Is there any human intervention to consolidate the budget and/or spending data?
How often does the government present spending reports to the legislature? Does the system allow this periodic data extraction?
Does the country have historical data on budget and spending? How far in time is it available either in systems or in stored files?
Do you follow any standardised publication patterns at the moment?

If you’re interested in pushing for better publication, please contact us, we’re happy to help. Our Helpdesk can be reached at openspending-support@okfn.org.

^[1] Pattanayal, Sailendra. Expenditure Control: Key Features, Stages, and Actors Prepared by Sailendra Pattanayak Fiscal Affairs Department. International Monetary Fund, 2016.