JJ Abrams, the creator of Lost gave a great talk at TED 2007 themed around “Mystery Boxes”. A Mystery Box is a box of unspecified goods. When you buy a Mystery Box – from a magic shop, as Abrams describes doing with his grandfather as a child, or from a pet store, sweet shop, or wherever – you don’t know what’s going to be in it, you can only guess.


Abrams says Lost and other TV shows are littered with mystery boxes, not literally, but in the sense that the plot and characters within them are like mystery boxes: you don’t know what they are about until the end. The strength of the show is the big tease about what might be in the boxes:

So there’s this thing with mystery boxes that I started feeling, like, compelled. Then there’s the thing of, like, mystery in terms of imagination — the withholding of information. You know, doing that intentionally is much more engaging.

If you like the idea of magical mystery boxes then you’re going to like this.

I’ve been looking at where our tax money is spent. That might sound completely unrelated to popular TV shows, but this research has been all about the tease of what might be in the mystery boxes of the British Government’s spending records. In this post, I’m going to share the questions I’ve asked about the Government’s mystery boxes, and the replies I’ve got (as well as the ones I haven’t).

I’d appreciate help and advice on every line of enquiry I mention here.

This is the rough picture of public spending databases I’ve gathered so far, as part of my work as a researcher at ‘Where Does My Money Go?’ (click on the image to see the full picture).


On the left, you’ll see that our taxes go in to HMRC, then travel through the Treasury (HMT), on their way to Central Government departments, from where they fan out into various spending bodies, including communities and local government. What I’m interested in is tracing records of spending as they pass from one department to another.

Now I’ll list my investigations and findings for each department I’ve looked into.

HM Revenue and Customs (HMRC)

I want to know how our tax records are stored, but so far I’ve only got this very very sketchy description (in response to one of many requests I’ve made under FOI). That reply does give some good clues about how National Insurance tax is stored in the statement:

For example the collection of National Insurance is supported by the National Insurance and PAYE system (NPS). This is a substantial build on what was previously known as the National Insurance Recording system (NIRS) to add on the PAYE business processes given the close interaction between the collection of PAYE tax and national Insurance. It has its own underlying database with customer information, but links to other databases within HMRC and in DWP to ensure that customer data is treated consistently throughout both departments.

Another thing we know about how our tax records are stored is that most of HMRC is outsourced and the contract for this outsourcing is not public yet. We know the contract is called ‘the Aspire contract’, and we have some background reading to do on it.

HM Treasury (HMT)

There is a project within the Treasury called ‘The Whole of Government Accounts‘, which aims to:

“consolidate the accounts of about 1300 bodies from within the central government, health service, local government and public corporation sectors.”

A database called the Combined Online INformation System (COINS) was developed to make this consolidation of accounts possible or easier or both. The Whole of Government Accounts has yet to report its work.

When I first started looking at the COINS system, very little had been made public about how detailed the spending records are. We’ve now got a pretty good understanding of the data in COINS as a result of our ‘where does my money go?‘ research.


We are now working to get:

A sample of complete COINS entries.
The costs associated with the spending codes.

The Office of National Statistics (ONS)

I know of two databases that store spending data in the ONS:

  • The CSDB (central shared database) does not use an ORACLE database. I’m in the process of finding out more details about the CSDB.
  • The CORD system (Centralised ONS Repository of Data) used for some parts of the annual production process in ELS does use an ORACLE database. I have the schema for this one.


As well as these two databases, I’ve been told the ONS gets a copy of the COINS data for it’s reports and I’ve had a request for the COINS data sent to the ONS rejected.

Other Departmental Accounts

Contacts at HMT have let me know that each government department has its own record of accounts. I’ve been investigating how the Department for Work and Pensions store their accounts, and my next step is to request the data.


Local Spending

I’ve also got the schema for the Oracle database Cambridge County Council use to store their accounts. The next step is to ask for the data or a sample of it.


These are my mystery boxes. Hopefully, by now, you’re as curious to know what’s inside them as I am.

+ posts

2 thoughts on “In The Beginning There Were Mystery Boxes”

  1. I’ve been searching in yahoo for some ideas and fortuitously found this blog.okfn.org web site. I’m so amazed by your way of thinking and writing. Have you thought about writing a book?

Comments are closed.