Papers



Submit a Paper!

Browse ReproHack papers

  • Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA

    Authors: Sahil Loomba, Alexandre de Figueiredo, Simon J. Piatek, Kristen de Graaf, Heidi J. Larson
    DOI: 10.1038/s41562-021-01056-1
    Submitted by samuelpawel      
      Mean reproducibility score:   8.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    In the middle of the COVID-19 pandemic, this paper provided important evidence regarding the effect of misinformation on vaccination intent. Its analyses and conclusions were extremely important for decision makers. Therefore, it is also important that the analyses are reproducible.

  • Machine learning a model for RNA structure prediction

    Authors: Nicola Calonaci, Alisha Jones, Francesca Cuturello, Michael Sattler, Giovanni Bussi
    DOI: 10.1093/nargab/lqaa090
    Submitted by giovannibussi      

    Why should we attempt to reproduce this paper?

    The method is trained on the data that were available, but it is meant to be re-trainable as soon as new data are published. It would be great to be really sure that even someone else will be able to do it. In case we receive any feedback, we would be really happy to improve our Github repository so as to make the reproduction easier!

  • Accelerating the prediction of large carbon clusters via structure search: Evaluation of machine-learning and classical potentials

    Authors: Bora Karasulu, Jean-Marc Leyssale, Patrick Rowe, Cedric Weber, Carla de Tomas
    DOI: 10.1016/j.carbon.2022.01.031
    Submitted by bkarasulu    
    Number of reviews:   1
    Why should we attempt to reproduce this paper?

    This paper presents a fine example of high-throughput computational materials screening studies, mainly focusing on the carbon nanoclusters of different sizes. In the paper, a set of diverse empirical and machine-learned interatomic potentials, which are commonly used to simulate carbonaceous materials, is benchmarked against the higher-level density functional theory (DFT) data, using a range of diverse structural features as the comparison criteria. Trying to reproduce the data presented here (even if you only consider a subset of the interaction potentials) will help you devise an understanding as to how you could approach a high-throughput structure prediction problem. Even though we concentrate here on isolated/finite nanoclusters, AIRSS (and other similar approaches like USPEX, CALYPSO, GMIN, etc.,) can also be used to predict crystal structures of different class of materials with applications in energy storage, catalysis, hydrogen storage, and so on.

  • Automatic learning of hydrogen-bond fixes in an AMBER RNA force field

    Authors: Thorben Fröhlking, Vojtěch Mlýnský, Michal Janeček, Petra Kührová, Miroslav Krepl, Pavel Banáš, Jiří Šponer, Giovanni Bussi
    Submitted by giovannibussi      

    Why should we attempt to reproduce this paper?

    We do care about reproducibility. In case we receive any feedback, we would be really happy to improve our Github repository and/or submitted manuscript so as to make the reproduction easier!

  • REMoDNaV: robust eye-movement classification for dynamic stimulation

    Authors: Asim H. Dar, Adina S. Wagner, Michael Hanke
    DOI: https://doi.org/10.3758/s13428-020-01428-x
    Submitted by adswa    
      Mean reproducibility score:   7.0/10   |   Number of reviews:   2
    Why should we attempt to reproduce this paper?

    In theory, reproducing this paper should only require a clone of a public Git repository, and the execution of a Makefile (detailed in the README of the paper repository at https://github.com/psychoinformatics-de/paper-remodnav). We've set up our paper to be dynamically generated, retrieving and installing the relevant data and software automatically, and we've even created a tutorial about it, so that others can reuse the same setup for their work. Nevertheless, we've for example never tried it out across different operating systems - who knows whether it works on Windows? We'd love to share the tips and tricks we found to work, and even more love feedback on how to improve this further.

  • The viewing angle in AGN SED models, a data-driven analysis

    Authors: Andrés Felipe Ramos Padilla, Lingyu Wang, Katarzyna Małek, Andreas Efstathiou, Guang Yang
    Submitted by aframosp    
      Mean reproducibility score:   9.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Most of the material is available through Jupyter notebooks in GitHub, and it should be easy to reproduce with the help of Binder. With the notebooks, you could experiment with different parameters to the ones analyzed in the paper. It also contains a large dataset of physical parameters of galaxies analysed in this work. We expect this work to be easily reproducible in the steps described in the repository.

  • Finding Efficient Trade-offs in Multi-Fidelity Response Surface Modeling

    Authors: Sander van Rijn, Sebastian Schmitt, Matthijs van Leeuwen, Thomas Bäck
    Submitted by sjvrijn    
      Mean reproducibility score:   9.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Because: - Two fellow PhDs working on different topics have been able to reproduce some figures by following the README instructions and I hope this extends to other people - I've tried to incorporate as many of the best practices as possible to make my code and data open and accessible - I've tried to make sure that my data is exactly reproducible with the specified random seed strategy - the paper suggests a method that should be useful to other researchers in my field, which is not useful unless my results are reproducible

  • Genomic Response to Vitamin D Supplementation in the Setting of a Randomized, Placebo-Controlled Trial

    Authors: Berlanga-Taylor, A. J., Plant, K., Dahl, A., Lau, E., Hill, M., Sims, D., Heger, A., et al.
    Submitted by hub-admin  
    Number of reviews:   1
    Why should we attempt to reproduce this paper?

    It was a null findings paper that disappointed many people. Could I have made a mistake in the coding?; I'm interested in using it as an example of reproducible research and learning from ReproHack. It's nerve wracking to submit for inspection from others so I also want to overcome that fear and be able to lead my students by example. I'll be available via the Slack group or other forms for communication as suggested by organisers. Please note it's only the gene expression and related data that's available on ArrayExpress.

    Tags: Python R
  • Investigation into the annotation of protocol sequencing steps in the sequence read archive

    Authors: Alnasir, Jamie, and Hugh P. Shanahan.
    Submitted by hub-admin  

    Why should we attempt to reproduce this paper?

    Metadata annotation is key to reproducibility in sequencing experiments. Reproducing this research using the scripts provided will also show the current level of annotation in years since 2015 when the paper was published.

    Tags: Python SQL
  • Analytical solutions for the isobaric evaporation of pure cryogens in storage tanks

    Authors: Felipe Huerta, Velisa Vesovic,
    DOI: 10.1016/j.ijheatmasstransfer.2019.118536
    Submitted by hub-admin    

    Why should we attempt to reproduce this paper?

    1. Because it contains customized numerical methods to implement analytical solutions for an engineering problem relevant to cryogenic storage. This will become increasingly relevant in the future with the increase in the use of liquid hydrogen and LNG as fuel. 2. The storage tank is implemented as a Class and there is an opportunity to understand the object oriented programming mindset of the authors. 3. In the provided Jupyter Notebook, thermodynamic data for nitrogen and methane are provided which enable the users the quick implementation. 4. To reproduce some of the figures and results, the storage tanks need to be modified with inputs available in the paper.

    Tags: Python Matlab
  • Deep Structural Causal Models for Tractable Counterfactual Inference

    Authors: Nick Pawlowski, Daniel C. Castro, Ben Glocker.
    Submitted by hub-admin    
      Mean reproducibility score:   5.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Some may argue that the field of machine learning is in a reproducibility crisis. It will be interesting to know how difficult it is for others to reproduce the results of a paper that proposed a quite complex methodology.

    Tags: Python
  • Plasmonic nanostructure design and characterization via Deep Learning

    Authors: Malkiel, I., Mrejen, M., Nagler, A. et al.
    DOI: 10.1038/s41377-018-0060-7
    Submitted by hub-admin    

    Why should we attempt to reproduce this paper?

    The current code is written in Torch, which is no longer actively maintained. Since deep learning in nanophotonics is an area of active interest (e.g. for the design of new metamaterials), it is important to update the code to use a more modern deep learning library such as tensorflow/keras

  • FlowFrontNet : Improving Carbon Composite Manufacturing with CNNs

    Authors: Stieber, S., Schröter, N., Schiendorfer, A., Hoffmann, A., & Reif, W.
    Submitted by hub-admin    

    Why should we attempt to reproduce this paper?

    To use data from a manufacturing process: RTM for carbon composite production.To see if you can handle large amounts of data: the 36 k injection runs contain a total of 5 m frames. Maybe it is possible for you to reach our performance on smaller parts of the data, which would be great.

    Tags: Python
  • pyKNEEr: An image analysis workflow for open and reproducible research on femoral knee cartilage

    Authors: Bonaretti S, Gold GE, Beaupre GS
    DOI: 10.1371/journal.pone.0226501
    Submitted by hub-admin    
      Mean reproducibility score:   6.5/10   |   Number of reviews:   2
    Why should we attempt to reproduce this paper?

    The paper describes pyKNEEr, a python package for open and reproducible research on femoral knee cartilage using Jupyter notebooks as a user interface. I created this paper with the specific intent to make both the workflows it describes and the paper itself open and reproducible, following guidelines from authorities in the field. Therefore, two things in the paper can be reproduced: 1) workflow results: Table 2 contains links to all the Jupyter notebooks used to calculate the results. Computations are long and might require a server, so if you want to run them locally, I recommend using only 2 or 3 images as inputs for the computations. Also, the paper should be sufficient, but if you need further introductory info, there are a documentation website: https://sbonaretti.github.io/pyKNEEr/ and a "how to" video: https://youtu.be/7WPf5KFtYi8 2) paper graphs: In the captions of figures 1, 4, and 5 you can find links to data repository, code (a Jupyter notebook), and the computational environment (binder) to fully reproduce the graph. These computations can be easily run locally and require a few seconds. All Jupyter notebooks automatically download data from Zenodo and provide dependencies, which should make reproducibility easier.

  • Dynamic redistribution of plasticity in a cerebellar spiking neural network reproducing an associative learning task perturbed by TMS

    Authors: Alberto Antonietti, Jessica Monaco, Egidio D'Angelo, Alessandra Pedrocchi, and Claudia Casellato
    Submitted by @_Aalph    

    Why should we attempt to reproduce this paper?

    Paper and codes+data have been published 4 years ago, will they still work? I always try to release data and codes to reproduce my papers, but I seldom receive feedback. It would be useful to have comments from a reproducers' team, in order to improve sharing for future research (I switched from MATLAB to Python already).

  • Good Me Bad Me: Prioritization of the Good-Self During Perceptual Decision-Making

    Authors: Hu, C.-P., Lan, Y., Macrae, C. N., & Sui, J.
    DOI: 10.1525/collabra.301
    Submitted by hub-admin    
      Mean reproducibility score:   7.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    It'll a great helpful to independently check the scientific record I've published, so that errors, if there are any, could be corrected. Also, I will learn how to share the data in a more accessible to other if you could give me feedback.

    Tags: Python R Matlab
  • Hyperparameter importance Across Datasets

    Authors: Jan N van Rijn and Frank Hutter
    DOI: 10.1145/3219819.3220058
    Submitted by hub-admin    
      Mean reproducibility score:   7.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    I tried hard to make this paper as reproducible as possible, but as techniques and dependencies become more complex, it is hard to make it 100% clear. Any form of feedback is more than welcome.

  • Statistical analysis of coverage error in simple global temperature estimators

    Authors: K Cowtan, P Jacobs, P Thorne, R Wilkinson
    Submitted by hub-admin    

    Why should we attempt to reproduce this paper?

    To see whether we did a good enough job in providing data and methods, and to check how the code has aged with respect to current libraries.

    Tags: Python
  • Quantitative analysis of spectroscopic Low Energy Electron Microscopy data: High-dynamic range imaging, drift correction and cluster analysis

    Authors: Jong T.A. de, Kok, D.N.L, Torren A.J.H. van der, Schopmans H., Tromp R.M., Molen S.J. van der & Jobst J.
    Submitted by hub-admin  
    Number of reviews:   2
    Why should we attempt to reproduce this paper?

    Low Energy Electron Microscopy (LEEM) is a somewhat specific form of electron microscopy used to study surfaces and 2D materials. In this paper we describe a set of data processing techniques applied to LEEM and adapted to the peculiarities of LEEM. This is combined with a parallelized Python implementation using Dask in separate notebooks. So if you are interested in microscopy, image analysis, clustering of experimental physics data or parallel Python, this paper should be interesting to you.

    Tags: Python
  • Determination of the fundamental absorption and optical bandgap of dielectric thin films from single optical transmittance measurements

    Authors: A. Tejada, L. Montañez, C. Torres, P. Llontop, L. Flores-Escalante, F. De Zela, A. Winnacker, and J. A. Guerra
    Submitted by hub-admin    

    Why should we attempt to reproduce this paper?

    We propose a simple method to retrieve optical constants from single optical transmittance measurements, in particular in the fundamental absorption region. The construction of needed envelopes is arbitrary and will depend on the user. However, the method should still be robust and deliver similar results.