We’re pleased to announce Data Protocols, a community-driven effort to develop simple, light-weight protocols and formats for distributed and collaborative work with data.
If you’re interested in the project got check out the Knight News Challenge Data Protocols application and give our proposal the thumbs up!
What’s the Idea
The civic and open data community is hampered by a lack of simple standards for interaction between services and tools.
We believe there is a clear need for the kinds of lightweight but useful protocols and formats for doing things like:
- Sharing and syncing data changes over HTTP
- Querying data and databases over the web
- “Packaging” data for publication and installation
- Creating web services for data transformation and reconciliation
DataProtocols isn’t a formal standardization process like W3C, ISO or OASIS, but rather is aiming to be a more informal community where we can hammer out RFCs for data – rough consensus, running code and integrated data (and building wherever possible on what already exists).
We believe consensus needs to come from a community not just a single vendor or organization and the user and developer community is now developed enough to take this on and make it happen.
Moreover, technology and especially web technology has reached the point where this is both feasible and needed – be that in terms of standards, use of HTTP and REST, browser maturity, or, in terms of ad-hoc development in tools that is ripe for “standardization”.
What’s Happening Already
Data Protocols initially started out of some informal chats on IRC last Autumn with various folks including Francis Irving and Aidan McGuire of ScraperWiki, Max Ogden (then a Code for America fellow) and Chris Taggart of OpenCorporates to name a few.
We’ve been operating in stealth mode since then and we feel it is now high time to announce the project and invite wider participation.
The current DataProtocols site combines existing proposals and material on the following areas:
- Syncing protocol
- Refining protocol
- Data Packages
- Simple Data Format
- Web-Orientated Data Formats
- Manifesto– the original version of which was drafted at last year’s Open Government Data Camp
Next Steps
The next steps for Data Protocols are to get more people directly invovled and engage the wider community.
We also believe that this process needs some assistance and curating. We recently applied to the Knight News Challenge to help us develop the Data Protocols project. For example, we think assistance in the following areas would be really valuable:
- Fund a dedicated staff member to act as community steward
- Engage lead experts on specific topic areas
- Sprint workshops on particular areas and implementations
- Support prototype implementations and validators – specs without implementations are of little value
Get Involved
DataProtocols is a community project, so we actively encourage contributions. If you want to get involved, comments and corrections can be submitted via one of the following routes:
- Write a patch to the git repo (fork and pull recommended)
- Sign up the mailing list and contribute to the community conversations
- User the issue tracker to highlight general issues and suggestions
- Tweet ideas at us using the Data Protcols Twitter account
Rufus Pollock is Founder and President of Open Knowledge.
