Reproducible processing

The best efforts are worth nothing in science if we cannot trace back gapless which sample has been measured with what parameters, and without a full trace of processing and analysis steps. Furthermore, we want to be able to reproduce a figure in a publication from the raw data and vice versa, meaning reaching out to the raw data (and the log of processing and analysis steps) given a (published) figure or table.

Few scientists can fully reproduce a figure or table in a publication (or thesis) from the raw data, and many publications lack the details of how the data have been processed and analysed to reproduce the results. Due to the inherent complexity of EPR spectroscopy, requiring usually quite a number of different processing and analysis steps that are carried out mostly manually, this is not surprising.

The answer to this problem is twofold: We need to automate all processes that can be automated, and we need to apply a modular framework for data processing and analysis that relieves the operator of the duty to manually log each processing and analysis step (including not only the full set of parameters, but as well the version of each individual program, library, or script used).

The author has developed and is further developing a series of tools for reproducible data processing and analysis as well as an overall infrastructure to keep track of samples, data, and analyses.