Reproducibility¶
The original name of "Annals of Improbable Research", the organizers of the Ignobel Prizes, was "Journal of Irreproducible Results". This was meant as a joke, but how reproducible are computational results published in "real" journals?
Open Source¶
- Reproducibility is not only about the developer reproducing the results, everyone else should be able to.
Version Control System¶
- Subversion
- Git
- Bazaar
- Mercurial, etc.
The output files must be coupled to a revision. Of course, this implies always commiting before a production run.
Simplest approach¶
int main(int argc, char * argv[]) {
system("svn info > svn.info");
...
Improvements¶
- We could also check status, commit automatically before a production, etc.
- For that, it would be better to use an API instead of system(), e.g. http://rapidsvn.tigris.org/svncpp.html
- Always branch on production run?
Parameters and Configuration Files¶
- Output files must also be coupled with runtime parameters.
- Use parsers for parameters and command line arguments (e.g. getopt, boost.program_options)
Automation¶
- Not only to save time - automated tasks are inherently reproducible