One of the hallmarks of a valid scientific study is reproducible results. At the beginning of this month, a team of three of us attempted to recreate the results of this paper. While we were able to get pretty close, the task was made a little more difficult by not knowing what the parameters of their simulation were.
When a paper is published, there must always be some talk of methodology, and understandably, the details are left out so that the writers can get to the heart of their findings. This is appropriate, but I believe that a more-or-less obligatory appendix should be included, specifying exactly how the the simulations or experiments being reported on were run with an eye toward reproducing results. This might consist of a table of parameters and their values, or code, or perhaps simply a repository link where researchers could obtain and inspect code and documentation relating to the paper.
Perhaps it's just my own impatience, but I feel like it shouldn't be the work of an experiment itself to reproduce the results of a previously-performed experiment.
No comments:
Post a Comment