SIB Days - Reproducibility Hackathon

SIB Days - Reproducibility Hackathon

      June 24, 2024, 9 a.m. - June 24, 2024, 4:30 p.m. (Europe/Zurich)

   Congress Centre Biel/Bienne, Zentralstrasse 60, Biel, 2502, Switzerland

Hosted by: SIB Swiss Institute of Bioinformatics


Event submitted by: sigmaSleuth on May 21, 2024, 7:09 a.m.

Register

Contact Host

Event Description

The event

Ever heard of the reproducibility crisis, the four horsemen of irreproducibility, or do you know what a pre-registered report is? The answers are irrelevant! Like climate change, nothing is more important than this…

We will venture together to a place many have spoken of, but few have gone to – trying to reproduce the results of scientific papers . After a quick introduction into reproducibility and the setup of the hackathon, the participants will be split into teams and assigned publications to reproduce. Teams will collaboratively find ways to reproduce, to gain access, and to discover new sources of tolerance, navigating through the (potential) hurdles of replicating papers. Each team will present their findings, detailing their journey of attempting to replicate the results in a final discussion round. This will include insights gained, obstacles encountered, and suggestions for improving reproducibility.

Please note: This is not an introduction to programming or data analysis. Participants should have a favorite programming language and skills in analyzing data on their own. Participants should bring their own laptops (incl. power cords) and can suggest publications for reproduction (but do not have to).

By joining the SIB Days, you agree to abide by the SIB's Code of Conduct

On the day

We’ll track of discussions and collaborative notes on the event hackpad

As all ReproHack events, we strive to make this event open and inclusive to all. As such the event is governed by the ReproHack Code of Conduct.Please read it before participating. By participating, you are expected to uphold this code.

Participants

Introduction & Welcome

1. Project review and team formation

We’ll start with a brief review of the available papers and then form into teams. Feel free to work on your own if you prefer but we highly recommend you discuss your experiences with fellow participants as you work.

2. Select and register a paper

  • Add your details in the participants section of the hackpad so we have a complete list of participants (e.g name, affiliation, etc).

  • In your teams, (or individually if you prefer) decide which paper you wish to work on.

  • Create user accounts on the Hub for all reviewers that want to be associated with the review.

  • Register your team and paper by logging the title of the selected paper and the name(s) of the reviewer(s) in the hackpad. You can use the following template:

    ### **Paper:** <Title of the paper reproduced>
**Reviewers:** Reviewer 1, Reviewer 2 etc.

3. Work on your paper!

Follow any instructions/documentation associated with the papers and try and reproduce the work. As you work through your paper, keep in mind the main points on which feedback to the authors will provided, Access, Reproducibility, Documentation and Reusability (see our participant guidelines for more information). It might help to have a look at the Review form before you begin and keep notes during your review. Feel free to use the event hackpad to record general findings you wish to share with the group.

We’ll come together during the day to discuss progress and troubleshoot any sticking points.

Should you finish reproducing your paper quickly, feel free to explore the work more deeply. For example, you could try and run additional analyses, create new plots or even combine materials with your own or other open materials available on the web!

Should you produce any additional materials relating to your reproduction during the session (e.g. a markdown report, jupyter notebook, issue or pull requets in authors repository), feel free to share it publicly and add any links to such materials to the hackpad.

4. Complete your Review with feedback for the authors

The most important part of the day is recording our experiences as feedback to the authors. Please make sure to complete a Review feedback form for the paper you've selected, ideally, by the end of the day. Please also remember to be kind and constructive. Reproducibilty is hard and all authors submitting their papers have been incredibly brave. Feel free to browse any public feedback submissions to get inspiration.

5. Feedback to the group on your experiences

You can use the hackpad to take notes and summarise your experiences.

6. Closing Remarks

Collaborative note taking:

Feel free to contribute any additional thoughts on our collaborative hackpad. These can help form the basis for a blogpost on the event.

Associated papers