Papers



Submit a Paper!

Browse ReproHack papers

  • Encapsulated Nanowires: Boosting Electronic Transport in Carbon Nanotubes

    Authors: Andrij Vasylenko, Jamie Wynn, Paulo Medeiros, Andrew J Morris, Jeremy Sloan, David Quigley
    DOI: 10.1103/PhysRevB.95.121408
    Submitted by dquigley      
      Mean reproducibility score:   5.0/10   |   Number of reviews:   2
    Why should we attempt to reproduce this paper?

    DFT calculations are in principle reproducible between different codes, but differences can arise due to poor choice of convergence tolerances, inappropriate use of pseudopotentials and other numerical considerations. An independent validation of the key quantities needed to compute electrical conductivity would be valuable. In this case we have published our input files for calculating the four quantities needed to parametrise the transport simulations from which we compute the electrical conductivity. These are specifically electronic band structure, phonon dispersions, electron-phonon coupling constants and third derivatives of the force constants. Each in turn in more sensitive to convergence tolerances than the last, and it is the final quantity on which the conclusions of the paper critically depend. Reference output data is provided for comparison at the data URL below. We note that the pristine CNT results (dark red line) in figure 3 are an independent reproduction of earlier work and so we are confident the Boltzmann transport simulations are reproducible. The calculated inputs to these from DFT (in the case of Be encapsulation) have not been independently reproduced to our knowledge.

  • New Insight into the Stability of CaCO3 Surfaces and Nanoparticles via Molecular Simulation

    Authors: A. Matthew Bano, P. Mark Rodger, and David Quigley
    DOI: 10.1021/la501409j
    Submitted by dquigley      

    Why should we attempt to reproduce this paper?

    The negative surface enthalpies in figure 5 are surprising. At least one group has attempted to reproduce these using a different code and obtained positive enthalpies. This was attributed to the inability of that code to independently relax the three simulation cell vectors resulting in an unphysical water density. This demonstrates how sensitive these results can be to the particular implementation of simulation algorithms in different codes. Similarly the force field used is now very popular. Its functional form and full set of parameters can be found in the literature. However differences in how different simulation codes implement truncation, electrostatics etc can lead to significant difference in results such as these. It would be a valuable exercise to establish if exactly the same force field as that used here can be reproduced from only its specification in the literature. The interfacial energies of interest should be reproducible with simulations on modest numbers of processors (a few dozen) with run times for each being 1-2 days. Each surface is an independent calculation and so these can be run concurrently during the ReproHack.

  • Thermodynamics of stacking disorder in ice nuclei

    Authors: David Quigley
    DOI: 10.1063/1.4896376
    Submitted by dquigley      
      Mean reproducibility score:   3.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    The results of this paper have been used in multiple subsequent studies as a benchmark against which other methods of performing the same calculation have been tested. Other groups have challenged the results as suffering from finite size effects, in particular the calculations on mixtures of cubic and hexagonal ice. Should there be time during in the event, participants could check this by performing calculations on larger unit cells. Each individual calculation should converge adequately within 96 hours making it amenable to a HPC ReproHack. Given modern HPC hardware many such calculations could be run concurrently on a single HPC node.

  • Highly efficient conversion of laser energy to hard X-rays in high intensity laser-solid simulations

    Authors: S. Morris, A. Robinson, C. Ridgers
    DOI: 10.1063/5.0055398
    Submitted by Stuart_Morris      

    Why should we attempt to reproduce this paper?

    There are many applications to multi-MeV X-rays. Their penetrative properties make them good for scanning dense objects for industry, and their ionising properties can destroy tumours in radiotherapy. They are also around the energy of nuclear transitions, so they can trigger nuclear reactions to break down nuclear waste into medical isotopes, or to reveal smuggled nuclear-materials for port security. Laser-driven X-ray generation offers a compact and efficient way to create a bright source of X-rays, without having to construct a large synchrotron. To fully utilise this capability, work on optimising the target design and understanding the underlying X-ray mechanisms are essential. The hybrid-PIC code is in a unique position to model the full interaction, so its ease-of-use and reproducibility are crucial for this field to develop.

  • PlanGAN: Model-based Planning With Sparse Rewards and Multiple Goals

    Authors: Henry Charlesworth and Giovanni Montana
    Submitted by gmontana74      
      Mean reproducibility score:   10.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    This paper proposes a probabilistic planner that can solve goal-conditional tasks such as complex continuous control problems. The approach reaches state-of-the-art performance when compared to current deep reinforcement learning algorithms. However, the method relies on an ensemble of deep generative models and is computationally intensive. It would be interesting to reproduce the results presented in this paper on their robotic manipulation and navigation problems as these are very challenging problems that current reinforcement learning methods cannot easily solve (and when they do, they require a significantly larger number of experiences). Can the results be reproduced out-of-the-box with the provided code?

  • REMoDNaV: robust eye-movement classification for dynamic stimulation

    Authors: Asim H. Dar, Adina S. Wagner, Michael Hanke
    DOI: https://doi.org/10.3758/s13428-020-01428-x
    Submitted by adswa    
      Mean reproducibility score:   7.0/10   |   Number of reviews:   3
    Why should we attempt to reproduce this paper?

    In theory, reproducing this paper should only require a clone of a public Git repository, and the execution of a Makefile (detailed in the README of the paper repository at https://github.com/psychoinformatics-de/paper-remodnav). We've set up our paper to be dynamically generated, retrieving and installing the relevant data and software automatically, and we've even created a tutorial about it, so that others can reuse the same setup for their work. Nevertheless, we've for example never tried it out across different operating systems - who knows whether it works on Windows? We'd love to share the tips and tricks we found to work, and even more love feedback on how to improve this further.

  • Optimizing the Use of Carbonate Standards to Minimize Uncertainties in Clumped Isotope Data

    Authors: Ilja J. Kocken, Inigo A. Müller, Martin Ziegler
    DOI: 10.1029/2019GC008545
    Submitted by japhir      

    Why should we attempt to reproduce this paper?

    Even though the approach in the paper focuses on a specific measurement (clumped isotopes) and how to optimize which and how many standards we use, I hope that the problem is general enough that insight can translate to any kind of measurement that relies on machine calibration. I've committed to writing a literate program (plain text interspersed with code chunks) to explain what is going on and to make the simulations one step at a time. I really hope that this is understandable to future collaborators and scientists in my field, but I have not had any code review internally and I also didn't receive any feedback on it from the reviewers. I would love to see if what in my mind represents "reproducible code" is actually reproducible, and to learn what I can improve for future projects!

  • The viewing angle in AGN SED models, a data-driven analysis

    Authors: Andrés Felipe Ramos Padilla, Lingyu Wang, Katarzyna Małek, Andreas Efstathiou, Guang Yang
    Submitted by aframosp    
      Mean reproducibility score:   9.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Most of the material is available through Jupyter notebooks in GitHub, and it should be easy to reproduce with the help of Binder. With the notebooks, you could experiment with different parameters to the ones analyzed in the paper. It also contains a large dataset of physical parameters of galaxies analysed in this work. We expect this work to be easily reproducible in the steps described in the repository.

  • Finding Efficient Trade-offs in Multi-Fidelity Response Surface Modeling

    Authors: Sander van Rijn, Sebastian Schmitt, Matthijs van Leeuwen, Thomas Bäck
    Submitted by sjvrijn    
      Mean reproducibility score:   9.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Because: - Two fellow PhDs working on different topics have been able to reproduce some figures by following the README instructions and I hope this extends to other people - I've tried to incorporate as many of the best practices as possible to make my code and data open and accessible - I've tried to make sure that my data is exactly reproducible with the specified random seed strategy - the paper suggests a method that should be useful to other researchers in my field, which is not useful unless my results are reproducible

  • No Effect of Nature Representations on State Anxiety, Actual and Perceived Noise

    Authors: Korbmacher, M., & Wright, L.
    DOI: 10.31234/osf.io/8gtyq
    Submitted by hub-admin    
      Mean reproducibility score:   5.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Basic analyses, which are easy to understand and reproduce + the paper contains multiple imputation, which can be interesting; ALL materials are available

    Tags: R
  • Genomic Response to Vitamin D Supplementation in the Setting of a Randomized, Placebo-Controlled Trial

    Authors: Berlanga-Taylor, A. J., Plant, K., Dahl, A., Lau, E., Hill, M., Sims, D., Heger, A., et al.
    Submitted by hub-admin  
    Number of reviews:   1
    Why should we attempt to reproduce this paper?

    It was a null findings paper that disappointed many people. Could I have made a mistake in the coding?; I'm interested in using it as an example of reproducible research and learning from ReproHack. It's nerve wracking to submit for inspection from others so I also want to overcome that fear and be able to lead my students by example. I'll be available via the Slack group or other forms for communication as suggested by organisers. Please note it's only the gene expression and related data that's available on ArrayExpress.

    Tags: Python R
  • Investigation into the annotation of protocol sequencing steps in the sequence read archive

    Authors: Alnasir, Jamie, and Hugh P. Shanahan.
    Submitted by hub-admin  

    Why should we attempt to reproduce this paper?

    Metadata annotation is key to reproducibility in sequencing experiments. Reproducing this research using the scripts provided will also show the current level of annotation in years since 2015 when the paper was published.

    Tags: Python SQL
  • Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: An observational study

    Authors: Hardwicke, T. E., Bohn, M., MacDonald, K., Hembacher, E., Nuijten, M. B., Peloquin, B. N., deMayo, B., Long, B., Yoon, E. J., & Frank, M. C.
    DOI: 10.1098/rsos.201494
    Submitted by hub-admin    
      Mean reproducibility score:   9.7/10   |   Number of reviews:   3
    Why should we attempt to reproduce this paper?

    This is perhaps an interesting 'meta' example for ReproHack as in this study we attempted to reproduce analyses reporrted in 25 published articles. So it seems even more important that our own analyses are reproducible! We tried our best to adhere to best practices in this regard, so we would be very keen to know if anyone has problems reproducing our analyses and/or learning how we can make the process easier. A couple of things to note: 1. In addition to the links to the data and analysis scripts provided above, we also have a Code Ocean container for this article (https://doi.org/10.24433/CO.1796004.v3), which should theoretically allow you to reproduce the analyses with the click of a single button (we hope!). 2. In addition to the main research analyses (for which I've provided links above), we also have data, scripts, and Code Ocean containers for each of the reprodubility attempts for the 25 articles we looked at. I don't know if you will also want to look at this level of the analyses, but if you do then take a look at Supplementary Information section E here: https://royalsocietypublishing.org/doi/suppl/10.1098/rsos.201494 For each reproducibility attempt, there is a short 'vignette' describing the outcome, and a link to data/scripts on the OSF and a Code Ocean container.

    Tags: R
  • Analytical solutions for the isobaric evaporation of pure cryogens in storage tanks

    Authors: Felipe Huerta, Velisa Vesovic,
    DOI: 10.1016/j.ijheatmasstransfer.2019.118536
    Submitted by hub-admin    

    Why should we attempt to reproduce this paper?

    1. Because it contains customized numerical methods to implement analytical solutions for an engineering problem relevant to cryogenic storage. This will become increasingly relevant in the future with the increase in the use of liquid hydrogen and LNG as fuel. 2. The storage tank is implemented as a Class and there is an opportunity to understand the object oriented programming mindset of the authors. 3. In the provided Jupyter Notebook, thermodynamic data for nitrogen and methane are provided which enable the users the quick implementation. 4. To reproduce some of the figures and results, the storage tanks need to be modified with inputs available in the paper.

    Tags: Python Matlab
  • Deep Structural Causal Models for Tractable Counterfactual Inference

    Authors: Nick Pawlowski, Daniel C. Castro, Ben Glocker.
    Submitted by hub-admin    
      Mean reproducibility score:   5.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Some may argue that the field of machine learning is in a reproducibility crisis. It will be interesting to know how difficult it is for others to reproduce the results of a paper that proposed a quite complex methodology.

    Tags: Python
  • Plasmonic nanostructure design and characterization via Deep Learning

    Authors: Malkiel, I., Mrejen, M., Nagler, A. et al.
    DOI: 10.1038/s41377-018-0060-7
    Submitted by hub-admin    

    Why should we attempt to reproduce this paper?

    The current code is written in Torch, which is no longer actively maintained. Since deep learning in nanophotonics is an area of active interest (e.g. for the design of new metamaterials), it is important to update the code to use a more modern deep learning library such as tensorflow/keras

  • The role of conidia in the dispersal of Ascochyta rabiei

    Authors: Khaliq, I., Fanning, J., Melloy, P. et al.
    DOI: 10.1007/s10658-020-02126-2
    Submitted by hub-admin    

    Why should we attempt to reproduce this paper?

    I suggested a few papers last year. I’m hoping that we’ve improved our reproducibility with this one, this year. We’ve done our best to package it up both in Docker and as an R package. I’d be curious to know what the best way to reproduce it is found to be. Working through vignettes or spinning up a Docker instance. Which is the preferred method?

    Tags: R Docker
  • Unveiling the diversity of spatial data infrastructures in Latin America: evidence from an exploratory inquiry

    Authors: Luis M. Vilches-Blázquez & Daniela Ballari
    DOI: 10.1080/15230406.2020.1772113
    Submitted by hub-admin    
      Mean reproducibility score:   10.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    It is kind of an easy reproducible code. It reads the data, makes few descriptive statistical analysis and plots figures using ggplot2.

    Tags: R
  • Evolutionary and food supply implications of ongoing maize domestication by Mexican campesinos

    Authors: Bellon, M. R., Mastretta-Yanes, A., Ponce-Mendoza, A., Ortiz-Santamaría, D., Oliveros-Galindo, O., Perales, H., … Sarukhán, J.
    DOI: 10.1098/rspb.2018.1049
    Submitted by hub-admin    
      Mean reproducibility score:   6.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    Cleaning the databases used for this study was one of the most challenging aspects of it, so making it public is the best way to make the more out of it. We made an effort to document all analyses and data wrangling steps. We are interested to know if it is truly reproducible so that we can follow this same scheme for further projects, or adjust accordingly.

    Tags: R
  • FlowFrontNet : Improving Carbon Composite Manufacturing with CNNs

    Authors: Stieber, S., Schröter, N., Schiendorfer, A., Hoffmann, A., & Reif, W.
    Submitted by hub-admin    

    Why should we attempt to reproduce this paper?

    To use data from a manufacturing process: RTM for carbon composite production.To see if you can handle large amounts of data: the 36 k injection runs contain a total of 5 m frames. Maybe it is possible for you to reach our performance on smaller parts of the data, which would be great.

    Tags: Python

Search for papers

Filter by tags

Python R GDAL GEOS GIS Shiny PROJ Galaxies Astronomy HPC Databases Binder Social Science Stata make Computer Science Jupyter Notebook tidyverse emacs literate earth sciences clumped isotopes org-mode geology eyetracking LaTeX Git ArcGIS Docker Drake SVN knitr C Matlab Mathematica Meta-analysis swig miniconda tensorflow keras Pandas SQL neuroscience robotics deep learning planner reiforcement learning Plasma physics Hybrid-PIC EPOCH Laser Gamma-ray X-ray radiation Petawatt Fortran plasma PIC physics Monte Carlo Atomistic Simulation LAMMPS Electron Transport DFT descriptors interatomic potentials machine learning Molecular Dynamics Python scripting AIRSS structure prediction density functional theory high-throughput machine-learning RNA bioinformatics CFD Fluid Dynamics OpenFOAM C++ DNS Mathematics Droplets Basilisk Particle-In-Cell psychology Stan Finance SAS Replication crisis Economics Malaria consumer behavior number estimation mental arithmetic psychophysics Archaeology Precipitation Epidemiology Parkrun Health Health Economics HTA plumber science of science Zipf networks city size distribution urbanism literature review Preference Visual Questionnaire Mann-Whitney Correlation Conceptual replication Cognitive psychology Multinomial processing tree (MPT) modeling #urbanism #R k-means cluster analysis city-regions Urban Knowledge Systems Topic modelling Planning Support Systems Software Citation Quarto snakemake Numerical modelling Ocean climate physical oceanography apptainer oceanography All tags Clear tags

Key

  Associated with an event
  Available for general review
  Public reviews welcome