September 2014

M&E-data tidying can help increasing your team’s productivity, saving resources and making more time available for the data-analysis stage and, of course, for your beneficiaries. If your project M&E data is “messy”, you will probably need to spend quite some time until you can begin any comprehensive analysis. Understanding well what tidy is data and ensuring that the output of your project’s M&E system is tidy can save you a great deal of trouble.

In the video below, you will find some useful tips and functions for tidying data with R, presented by Hadley Wickham, creator of many popular R packages.

Wickham’s paper on Tidy Data is another very useful resource and its abstract is presented below. You might also want to read his complete article on Tidy Data.

A huge amount of effort is spent cleaning data to get it ready for analysis, but there has been little research on how to make data cleaning as easy and effective as possible. This paper tackles a small, but important, component of data cleaning: data tidying. Tidy datasets are easy to manipulate, model and visualise, and have a specific structure: each variable is a column, each observation is a row, and each type of observational unit is a table. This framework makes it easy to tidy messy datasets because only a small set of tools are needed to deal with a wide range of un-tidy datasets. This structure also makes it easier to develop tidy tools for data analysis, tools that both input and output tidy datasets. The advantages of a consistent data structure and matching tools are demonstrated with a case study free from mundane data manipulation chores.