This week saw the release of version 1.1.0 of the Open Data Editor (ODE), the new Open Knowledge Foundation’s app that makes it easier for people with little to no technical skills to work with data. The app is now ready to enter a crucial phase of user testing. In October, we are starting a pilot programme with early adopters that will provide much-needed feedback and report bugs before the first official stable release, planned for December 2024.

(See below how you can get involved)

The Open Data Editor helps you find errors in your datasets and correct them in no time – a process called “data validation” in industry jargon. It also checks that your spreadsheet or dataset has all the necessary information for other people to use. ODE increases the quality of the data that is produced and consumed, and guarantees, again in technical jargon, “data interoperability”.

Thanks to funding from the Patrick J. McGovern Foundation, our team has been working since the beginning of the year to create this no-code tool for making data manipulation easier for non-technical people, such as journalists, activists, and public administration. This work seeks to put into practice our new vision of open technologies that OKFN will present and discuss at the upcoming The Tech We Want Summit.

It’s been an intense journey, which we briefly recap in this post.

What is in the latest version

1. A large number of functionalities were removed from the app to transform the Open Data Editor into a table validation tool.
2. Key UX changes made the application simpler to use. Examples: new button layout, new logic for uploading data and new dialogue boxes explaining some complex things about the tool.
3. Code improved: it’s now simplified, more accessible and documented to facilitate contributions from the community.
4. Different data communities engaged in discussions about how the Open Data Editor can help them in their everyday work with data.

ODE in figures

What we have done so far

A February of Sharing
The project plan was shared with the Open Knowledge Network in the monthly call, to gather input and feedback.

A March of Listening
After testing the app and reviewing all the documentation, interviews were conducted with data practitioners to understand the challenges they face when working with data. 

An April of New Directions
The patterns and insights emerged from the interviews were organised to review the application’s concept note and define a new vision for the product. Initially, ODE provided a wide range of options for people: working with maps, images, articles, scripts and charts. From the interviews, we learned that people working with data spend a lot of time understanding the tables and trying to identify problems in them so that they can analyse the data at a later stage. Therefore, we decided to redirect the ODE to a tool for checking errors in tables.

A May of Cleaning
Through a survey, we started asking questions about certain terms used in the application, such as the word ‘Validate’. We realised that a translation for non-technical users was required instead of simply using the vocabulary from Frictionless, the framework used behind the scenes to detect errors in tables. 

During that month we also started to remove many features from the application that did not align with the new product vision. The road was not particularly easy. As is always the case in coding, several things were interconnected and we had to make many decisions at every step. The whole process led us to deeper reflections about how to build civic technology. 

As part of that reflection, we decided to openly share the mistakes, pitfalls, and key learnings from our development journey. The title of our talk at csv,conf,v8 in Puebla, Mexico, was ‘The tormented journey of an app’.

A June of Interfacing
At this time our UX specialist joined the team to focus on making adjustments to clearly communicate the functionalities of the Open Data Editor.

Intending to create a truly intuitive application that addresses existing UX issues, key workflows were redefined, such as processes like Launch and Onboarding, Validation, File Import, File Management, and Datagrid Operations. Leveraging prior user research and agile software methodology, we went through multiple iterations and refinements. This process involved brainstorming, validating ideas, rapid prototyping, updating UX copies, A/B testing, and technical feasibility reviews with the development team. 

Built on Google’s Material UI framework, a new design system was also developed – the single source of truth comprising vibrant colours and patterns aligned with the OKFN’s branding – delivering a fresh, modern, and cohesive user experience, seamlessly extending from our website to the application.

July-September for Rebuilding
The cleanup process of the application continued. But this time the changes in the user interface led to new complexities: changes in workflows, new bugs with the implemented changes, etc. It was a time strongly focussed on development. 

In August, we opened this process in the panel ‘Frictionless data for more collaboration’ at Wikimania 2024, in Katowice, Poland. The community of Wikimedians and open activists discussed data friction and learned how ODE can help enhancing data quality and FAIRness.

At the end of August, we started working with Madelon Hulsebos, professor at CWI Amsterdam and an expert in Table Representation Learning (TRL). She is currently helping us think about the integration of artificial intelligence (AI) in the Open Data Editor by raising great questions and providing key ideas.

What is next

👉🏼 Address two key and complex components of the app: the metadata and the error panels. Adapting both elements to non-technical users requires more in-depth conversations and decisions since the Frictionless Framework creates some constraints for customisation options.

👉🏼 Pilots: To further improve the ODE, we need to receive feedback and recommendations from real users. Therefore, from October until December, two external organisations will be incorporating ODE into their data workflow to test the application, documenting their experience and reporting challenges to improve it.

👉🏼 User testing sessions: In October, we will hold a series of sessions to receive feedback from our community and from other potential users of Open Data Editor. 

👉🏼Codebase testing: As an effort to bring more contributors in the project, in October and November we will have 4 external developers testing the codebase and solving some code issues selected by the core team.

👉🏼 Documentation review: In November, we will hold two sessions to review all the documentation with a selected group of people. This way we will make sure the documentation is as easy to understand as possible for a broad audience.

👉🏼 Translations: In December, the user interface and the documentation will be translated into three languages other than English. 

👉🏼 AI integration: We are now discussing ideas and having conversations on how to make the integration transparent to users. In addition, our AI consultant will provide guidance on how new integrations should look in the future.

👉🏼 Online Course: By December, we will also release a free online course on how to use the Open Data Editor to enhance data FAIRness.

Now we are counting on you! You can apply to take part in the Open Data Editor testing sessions. Please register using this form or by clicking the button below.

Are you a developer? We are also looking for developers interested in testing the codebase and contributing to the project pushing a couple of PRs to solve 3 issues selected by our core team. If you’re interested in open data tools, this is your chance to get involved and make a difference. You can read about the programme here.

You can also email us at info@okfn.org, follow the GitHub repository or join the Frictionless Data community. We meet once a month.

Read more

• Are you a developer? Help us test the Open Data Editor!
• Open Data Editor: The tormented journey of an app
• Open Data Editor: 5 tips for building data products that work for people
• Open Data Editor: Meet the team behind the app
• Open Data Editor: What we learned from user research
• Announcement of strategic funding for the Open Data Editor
• Introducing Open Data Editor (beta): Towards a No-Code Data App for Everyone