We are all excited by the progress made by authors in making their papers reproducible by publishing associated code and data.
We know how challenging it can be so we want to showcase the value of the practice, both for original authors and as a learning experience for those who attempt to reproduce the work.
Prior to the session, we will invite authors to submit paper with associated code and data for review. During the session, participants will attempt to reproduce submitted papers of their choice and feed back their experiences to authors by completing a review form. Authors get friendly feedback on the reproducibility of their work while participants get practical experience in reproducibility through working with other people’s materials.
We will also spend time sharing and learning from the experiences of participants in the session."
It’s important to highlight that our events are by no means an attempt to criticise or discredit work. We see reproduction as beneficial scientific activity in itself, with useful outcomes for authors and valuable learning experiences for the participants and the research community as a whole.
By joining our event, you agree to abide by our Code of Conduct
We invite nominations for papers that have both associated code and data publicly available. We also encourage analyses based on open source tools as we cannot guarantee participants will have access to specialised licenced software.
Join us at the hack to dig into exciting science, learn more about reproducibility, working with other people’s code and data and more!
|18:00||Welcome and Introductions|
|18:10||Tips and Tricks for Reproducing and Reviewing.|
|18:20||Team formation & paper selection|
|18:30||Round I of ReproHacking (break-out rooms)|
|19:35||Round II of ReproHacking (break-out rooms) - Complete Feedback form|
|20:45||Re-group and sharing of experiences|
We’ll track of discussions and collaborative notes on the event hackpad
As all ReproHack events, we strive to make this event open and inclusive to all. As such the event is governed by the ReproHack Code of Conduct.Please read it before participating. By participating, you are expected to uphold this code.
We encourage authors that are keen and submitting to a specific event, to engage with the event, even just to say hello! You can do so on twitter (check for event specific hashtags or use
@ReproHack) or by
review-chat channel in our
slack group. We’ve
also set up an
authors channel in our
slack group as a
place where you can reach the organisers prior to the hack.
We’ll start with a brief review of the available papers and then form into teams. Feel free to work on your own if you prefer but we highly recommend you discuss your experiences with fellow participants as you work.
Add your details in the participants section of the hackpad so we have a complete list of participants (e.g name, affiliation, social links, pronouns etc).
In your teams, (or individually if you prefer) decide which paper you wish to work on.
Register your team and paper by logging the title of the selected paper and the name(s) of the reviewer(s) in the hackpad. You can use the following template:
### **Paper:** <Title of the paper reproduced> **Reviewers:** Reviewer 1, Reviewer 2 etc.
Follow any instructions/documentation associated with the papers and try and reproduce the work. As you work through your paper, keep in mind the main points on which feedback to the authors will provided, Access, Reproducibility, Documentation and Reusability (see our participant guidelines for more information). You will need to create an account on our Hub to submit your review. It might help to have a look at the Review form before you begin and keep notes during your review.
The most important part of the activity is recording our experiences as feedback to the authors. Please make sure to complete a Review feedback form for the paper you've selected, ideally, by the end of the day. Please also remember to be kind and constructive. Reproducibilty is hard and all authors submitting their papers have been incredibly brave. Feel free to browse any public feedback submissions to get inspiration.
We’ll come together at the end of the session to discuss how we got on. You can use the hackpad to take notes and summarise your experiences.
Feel free to contribute any additional thoughts on our collaborative hackpad. These can help form the basis for a blogpost on the event.