Reproducibility is a fundamental principle of research, and crucial in scientific disciplines such as climate research which impact decision and policy making and, eventually, our daily lives. This does not only apply to experimental and empirical research, but also to research relying on software and simulations. In the past decade or so, the so-called replication crisis has led to a number of initiatives and networks which address issues with computational reproducibility. One such initiative is the ReproHack community and hub.
The Institute of Computing for Climate Science (ICCS) Research Software Engineers (RSE) team is organising a ReproHack for the climate research community. A ReproHack is an event where the participants come together to reproduce computational key results of research papers which have been submitted for this purpose before the event. This provides a number of benefits for both the participants and the authors of the papers:
The participants learn hands-on about research reproducibility and software best practices. Going back to their own codes and research after the event they will be aware how they can improve their practices.
The participants will also have the experience of working in a team with an unfamiliar code. Less experienced programmers can see how experienced coders approach the task, and gain confidence talking about their thought processes and problems.
The paper authors receive feedback on the reproducibility of their papers as well as the code, which helps them improve. This is particularly useful if they submit papers which have not yet been published, as they can incorporate the feedback.
By joining our event, you agree to abide by our Code of Conduct
We invite nominations for papers that have both associated code and data publicly available. We also encourage analyses based on open source tools as we cannot guarantee participants will have access to specialised licenced software.
See list below for paper associated with this event or browse the full list of submitted papers. You can also consult our author guidelines for more information.
Join us at the hack to dig into exciting science, learn more about reproducibility, working with other people’s code and data and more! Have a look at the participant guidelines for more information.
Time (UTC) | Event |
---|---|
13:00 | Welcome and Orientation |
13:10 | Ice breaker session |
13:40 | Talk: Alejandro Coca-Castro: The Turing Way |
14:00 | Tips and Tricks for Reproducing and Reviewing. |
14:10 | Select papers, chat and coffee |
14:30 | Round I of ReproHacking (break-out rooms) |
15:30 | Re-group and sharing of experiences |
15:50 | Coffee break |
16:15 | Round II of ReproHacking (break-out rooms) - Complete Feedback Form |
17:15 | Re-group and sharing of experiences |
17:45 | Feedback and Closing |
18:00 | End |
I tried hard to make it reproducible, so hopefully this paper can serve as an example on how reproducibility can be achieved. I think that being reproducible with only few commands to type in a terminal is quite an achievment. At least in my field, where I usually see code published along with paper, but with almost no documentation on how to rerun it.
I used a lot of different tools and strategies to make this paper easily reproducible at different levels. There's Docker container for the highest level of reproducibility, and package versions are managed with renv. The data used in the paper is hosted on Zenodo to avoid long queue times when downloading from the Climate Data Store and future-proof for when it goes away and checksumed before using it.