Papers



Submit a Paper!

Browse ReproHack papers

  • Southern Ocean deep mixing band emerges from a competition between winter buoyancy loss and upper stratification strength

    Authors: Romain Caneill, Fabien Roquet, and Jonas Nycander
    DOI: 10.5194/egusphere-2023-2404
    Submitted by rcaneill      
      Mean reproducibility score:   2.6/10   |   Number of reviews:   8
    Why should we attempt to reproduce this paper?

    I tried as hard as possible to make it reproducible, which it is on my computer. I would be happy to see if this still works on other computers. Moreover, by allowing easy reproducibility, I hope that other people may easily build research on top of this work.

  • Revisiting the zonally asymmetric extratropical circulation of the Southern Hemisphere spring using complex empirical orthogonal functions

    Authors: Elio Campitelli, Leandro Díaz, Carolina Vera
    DOI: 10.1007/s00382-023-06780-0
    Submitted by eliocamp      
      Mean reproducibility score:   1.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    I used a lot of different tools and strategies to make this paper easily reproducible at different levels. There's Docker container for the highest level of reproducibility, and package versions are managed with renv. The data used in the paper is hosted on Zenodo to avoid long queue times when downloading from the Climate Data Store and future-proof for when it goes away and checksumed before using it.

    Tags: R Docker climate
  • The Polar Transition from Alpha to Beta Regions Set by a Surface Buoyancy Flux Inversion

    Authors: Romain Caneill Fabien Roquet Gurvan Madec Jonas Nycander
    DOI: 10.1175/JPO-D-21-0295.1
    Submitted by rcaneill      
      Mean reproducibility score:   0.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    I tried hard to make it reproducible, so hopefully this paper can serve as an example on how reproducibility can be achieved. I think that being reproducible with only few commands to type in a terminal is quite an achievment. At least in my field, where I usually see code published along with paper, but with almost no documentation on how to rerun it.

  • A multi-level analysis of data quality for formal software citation

    Authors: David Schindler, Tazin Hossain, Sascha Spors, Frank Krüger
    DOI: https://doi.org/10.48550/arXiv.2306.17535
    Submitted by frank.krueger    
      Mean reproducibility score:   9.0/10   |   Number of reviews:   2
    Why should we attempt to reproduce this paper?

    We spend a lot of time to make our analyses reproducible. A review would allow us to collect some information on whether we are successful with it.

  • What do analyses of city size distributions have in common?

    Authors: Clémentine Cottineau
    DOI: 10.1007/s11192-021-04256-8
    Submitted by clementinecottineau      
      Mean reproducibility score:   8.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    This article was meant to be entirely reproducible, with the data and code published alongside the article. It is however not embedded within a container (e.g. Docker). Will it past the reproducibility test tomorrow? next year? I'm curious.

  • Living HTA: Automating Health Technology Assessment with R

    Authors: Robert A. Smith, Paul P. Schneider, Wael Mohammed
    DOI: 10.12688/wellcomeopenres.17933.1
    Submitted by rasmith3    

    Why should we attempt to reproduce this paper?

    We think this is an interesting paper for anyone who wants to learn to build an API with the R package plumber. This is a novel method in health economics, but we believe will help improve the transparency of modelling methods in our field.

  • Does ethnic density influence community participation in mass participation physical activity events?

    Authors: Robert A. Smith, Paul P. Schneider, Alice Bullas, Steve Haake, Helen Quirk, Rami Cosulich1, Elizabeth Goyder
    DOI: 10.12688/wellcomeopenres.15657.2
    Submitted by rasmith3    
      Mean reproducibility score:   9.2/10   |   Number of reviews:   5
    Why should we attempt to reproduce this paper?

    The code and data are both on GitHub. The paper has been published in Wellcome Open Research and has been replicated by multiple other authors.

  • Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA

    Authors: Sahil Loomba, Alexandre de Figueiredo, Simon J. Piatek, Kristen de Graaf, Heidi J. Larson
    DOI: 10.1038/s41562-021-01056-1
    Submitted by samuelpawel      
      Mean reproducibility score:   7.0/10   |   Number of reviews:   4
    Why should we attempt to reproduce this paper?

    In the middle of the COVID-19 pandemic, this paper provided important evidence regarding the effect of misinformation on vaccination intent. Its analyses and conclusions were extremely important for decision makers. Therefore, it is also important that the analyses are reproducible.

  • Investigating the replicability of preclinical cancer biology

    Authors: Timothy M Errington, Maya Mathur, Courtney K Soderberg, Alexandria Denis, Nicole Perfito, Elizabeth Iorns, Brian A Nosek
    DOI: 10.7554/eLife.71601
    Submitted by samuelpawel      

    Why should we attempt to reproduce this paper?

    This papers represents an important milestone in meta-science, as it is one of the first large-scale replication projects outside the social sciences.

  • Machine learning a model for RNA structure prediction

    Authors: Nicola Calonaci, Alisha Jones, Francesca Cuturello, Michael Sattler, Giovanni Bussi
    DOI: 10.1093/nargab/lqaa090
    Submitted by giovannibussi      

    Why should we attempt to reproduce this paper?

    The method is trained on the data that were available, but it is meant to be re-trainable as soon as new data are published. It would be great to be really sure that even someone else will be able to do it. In case we receive any feedback, we would be really happy to improve our Github repository so as to make the reproduction easier!

  • Accelerating the prediction of large carbon clusters via structure search: Evaluation of machine-learning and classical potentials

    Authors: Bora Karasulu, Jean-Marc Leyssale, Patrick Rowe, Cedric Weber, Carla de Tomas
    DOI: 10.1016/j.carbon.2022.01.031
    Submitted by bkarasulu    
    Number of reviews:   1
    Why should we attempt to reproduce this paper?

    This paper presents a fine example of high-throughput computational materials screening studies, mainly focusing on the carbon nanoclusters of different sizes. In the paper, a set of diverse empirical and machine-learned interatomic potentials, which are commonly used to simulate carbonaceous materials, is benchmarked against the higher-level density functional theory (DFT) data, using a range of diverse structural features as the comparison criteria. Trying to reproduce the data presented here (even if you only consider a subset of the interaction potentials) will help you devise an understanding as to how you could approach a high-throughput structure prediction problem. Even though we concentrate here on isolated/finite nanoclusters, AIRSS (and other similar approaches like USPEX, CALYPSO, GMIN, etc.,) can also be used to predict crystal structures of different class of materials with applications in energy storage, catalysis, hydrogen storage, and so on.

  • Automatic learning of hydrogen-bond fixes in an AMBER RNA force field

    Authors: Thorben Fröhlking, Vojtěch Mlýnský, Michal Janeček, Petra Kührová, Miroslav Krepl, Pavel Banáš, Jiří Šponer, Giovanni Bussi
    Submitted by giovannibussi      

    Why should we attempt to reproduce this paper?

    We do care about reproducibility. In case we receive any feedback, we would be really happy to improve our Github repository and/or submitted manuscript so as to make the reproduction easier!

  • REMoDNaV: robust eye-movement classification for dynamic stimulation

    Authors: Asim H. Dar, Adina S. Wagner, Michael Hanke
    DOI: https://doi.org/10.3758/s13428-020-01428-x
    Submitted by adswa    
      Mean reproducibility score:   7.0/10   |   Number of reviews:   3
    Why should we attempt to reproduce this paper?

    In theory, reproducing this paper should only require a clone of a public Git repository, and the execution of a Makefile (detailed in the README of the paper repository at https://github.com/psychoinformatics-de/paper-remodnav). We've set up our paper to be dynamically generated, retrieving and installing the relevant data and software automatically, and we've even created a tutorial about it, so that others can reuse the same setup for their work. Nevertheless, we've for example never tried it out across different operating systems - who knows whether it works on Windows? We'd love to share the tips and tricks we found to work, and even more love feedback on how to improve this further.

  • Optimizing the Use of Carbonate Standards to Minimize Uncertainties in Clumped Isotope Data

    Authors: Ilja J. Kocken, Inigo A. Müller, Martin Ziegler
    DOI: 10.1029/2019GC008545
    Submitted by japhir      

    Why should we attempt to reproduce this paper?

    Even though the approach in the paper focuses on a specific measurement (clumped isotopes) and how to optimize which and how many standards we use, I hope that the problem is general enough that insight can translate to any kind of measurement that relies on machine calibration. I've committed to writing a literate program (plain text interspersed with code chunks) to explain what is going on and to make the simulations one step at a time. I really hope that this is understandable to future collaborators and scientists in my field, but I have not had any code review internally and I also didn't receive any feedback on it from the reviewers. I would love to see if what in my mind represents "reproducible code" is actually reproducible, and to learn what I can improve for future projects!

  • The viewing angle in AGN SED models, a data-driven analysis

    Authors: Andrés Felipe Ramos Padilla, Lingyu Wang, Katarzyna Małek, Andreas Efstathiou, Guang Yang
    Submitted by aframosp    
      Mean reproducibility score:   9.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Most of the material is available through Jupyter notebooks in GitHub, and it should be easy to reproduce with the help of Binder. With the notebooks, you could experiment with different parameters to the ones analyzed in the paper. It also contains a large dataset of physical parameters of galaxies analysed in this work. We expect this work to be easily reproducible in the steps described in the repository.

  • Finding Efficient Trade-offs in Multi-Fidelity Response Surface Modeling

    Authors: Sander van Rijn, Sebastian Schmitt, Matthijs van Leeuwen, Thomas Bäck
    Submitted by sjvrijn    
      Mean reproducibility score:   9.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Because: - Two fellow PhDs working on different topics have been able to reproduce some figures by following the README instructions and I hope this extends to other people - I've tried to incorporate as many of the best practices as possible to make my code and data open and accessible - I've tried to make sure that my data is exactly reproducible with the specified random seed strategy - the paper suggests a method that should be useful to other researchers in my field, which is not useful unless my results are reproducible

  • No Effect of Nature Representations on State Anxiety, Actual and Perceived Noise

    Authors: Korbmacher, M., & Wright, L.
    DOI: 10.31234/osf.io/8gtyq
    Submitted by hub-admin    
      Mean reproducibility score:   5.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Basic analyses, which are easy to understand and reproduce + the paper contains multiple imputation, which can be interesting; ALL materials are available

    Tags: R
  • Genomic Response to Vitamin D Supplementation in the Setting of a Randomized, Placebo-Controlled Trial

    Authors: Berlanga-Taylor, A. J., Plant, K., Dahl, A., Lau, E., Hill, M., Sims, D., Heger, A., et al.
    Submitted by hub-admin  
    Number of reviews:   1
    Why should we attempt to reproduce this paper?

    It was a null findings paper that disappointed many people. Could I have made a mistake in the coding?; I'm interested in using it as an example of reproducible research and learning from ReproHack. It's nerve wracking to submit for inspection from others so I also want to overcome that fear and be able to lead my students by example. I'll be available via the Slack group or other forms for communication as suggested by organisers. Please note it's only the gene expression and related data that's available on ArrayExpress.

    Tags: Python R
  • Investigation into the annotation of protocol sequencing steps in the sequence read archive

    Authors: Alnasir, Jamie, and Hugh P. Shanahan.
    Submitted by hub-admin  

    Why should we attempt to reproduce this paper?

    Metadata annotation is key to reproducibility in sequencing experiments. Reproducing this research using the scripts provided will also show the current level of annotation in years since 2015 when the paper was published.

    Tags: Python SQL
  • Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: An observational study

    Authors: Hardwicke, T. E., Bohn, M., MacDonald, K., Hembacher, E., Nuijten, M. B., Peloquin, B. N., deMayo, B., Long, B., Yoon, E. J., & Frank, M. C.
    DOI: 10.1098/rsos.201494
    Submitted by hub-admin    
      Mean reproducibility score:   9.7/10   |   Number of reviews:   3
    Why should we attempt to reproduce this paper?

    This is perhaps an interesting 'meta' example for ReproHack as in this study we attempted to reproduce analyses reporrted in 25 published articles. So it seems even more important that our own analyses are reproducible! We tried our best to adhere to best practices in this regard, so we would be very keen to know if anyone has problems reproducing our analyses and/or learning how we can make the process easier. A couple of things to note: 1. In addition to the links to the data and analysis scripts provided above, we also have a Code Ocean container for this article (https://doi.org/10.24433/CO.1796004.v3), which should theoretically allow you to reproduce the analyses with the click of a single button (we hope!). 2. In addition to the main research analyses (for which I've provided links above), we also have data, scripts, and Code Ocean containers for each of the reprodubility attempts for the 25 articles we looked at. I don't know if you will also want to look at this level of the analyses, but if you do then take a look at Supplementary Information section E here: https://royalsocietypublishing.org/doi/suppl/10.1098/rsos.201494 For each reproducibility attempt, there is a short 'vignette' describing the outcome, and a link to data/scripts on the OSF and a Code Ocean container.

    Tags: R

Search for papers

Filter by tags

Python R GDAL GEOS GIS Shiny PROJ Galaxies Astronomy HPC Databases Binder Social Science Stata make Computer Science Jupyter Notebook tidyverse emacs literate earth sciences clumped isotopes org-mode geology eyetracking LaTeX Git ArcGIS Docker Drake SVN knitr C Matlab Mathematica Meta-analysis swig miniconda tensorflow keras Pandas SQL neuroscience robotics deep learning planner reiforcement learning Plasma physics Hybrid-PIC EPOCH Laser Gamma-ray X-ray radiation Petawatt Fortran plasma PIC physics Monte Carlo Atomistic Simulation LAMMPS Electron Transport DFT descriptors interatomic potentials machine learning Molecular Dynamics Python scripting AIRSS structure prediction density functional theory high-throughput machine-learning RNA bioinformatics CFD Fluid Dynamics OpenFOAM C++ DNS Mathematics Droplets Basilisk Particle-In-Cell psychology Stan Finance SAS Replication crisis Economics Malaria consumer behavior number estimation mental arithmetic psychophysics Archaeology Precipitation Epidemiology Parkrun Health Health Economics HTA plumber science of science Zipf networks city size distribution urbanism literature review Preference Visual Questionnaire Mann-Whitney Correlation Conceptual replication Cognitive psychology Multinomial processing tree (MPT) modeling #urbanism #R k-means cluster analysis city-regions Urban Knowledge Systems Topic modelling Planning Support Systems Software Citation Quarto snakemake Numerical modelling Ocean climate physical oceanography apptainer oceanography All tags Clear tags

Key

  Associated with an event
  Available for general review
  Public reviews welcome