Papers



Submit a Paper!

Browse ReproHack papers

  • What do analyses of city size distributions have in common?

    Authors: Clémentine Cottineau
    DOI: 10.1007/s11192-021-04256-8
    Submitted by clementinecottineau      

    Why should we attempt to reproduce this paper?

    This article was meant to be entirely reproducible, with the data and code published alongside the article. It is however not embedded within a container (e.g. Docker). Will it past the reproducibility test tomorrow? next year? I'm curious.

  • Living HTA: Automating Health Technology Assessment with R

    Authors: Robert A. Smith, Paul P. Schneider, Wael Mohammed
    DOI: 10.12688/wellcomeopenres.17933.1
    Submitted by rasmith3    

    Why should we attempt to reproduce this paper?

    We think this is an interesting paper for anyone who wants to learn to build an API with the R package plumber. This is a novel method in health economics, but we believe will help improve the transparency of modelling methods in our field.

  • Does ethnic density influence community participation in mass participation physical activity events?

    Authors: Robert A. Smith, Paul P. Schneider, Alice Bullas, Steve Haake, Helen Quirk, Rami Cosulich1, Elizabeth Goyder
    DOI: 10.12688/wellcomeopenres.15657.2
    Submitted by rasmith3    
      Mean reproducibility score:   10.0/10   |   Number of reviews:   2
    Why should we attempt to reproduce this paper?

    The code and data are both on GitHub. The paper has been published in Wellcome Open Research and has been replicated by multiple other authors.

  • Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA

    Authors: Sahil Loomba, Alexandre de Figueiredo, Simon J. Piatek, Kristen de Graaf, Heidi J. Larson
    DOI: 10.1038/s41562-021-01056-1
    Submitted by samuelpawel      
      Mean reproducibility score:   8.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    In the middle of the COVID-19 pandemic, this paper provided important evidence regarding the effect of misinformation on vaccination intent. Its analyses and conclusions were extremely important for decision makers. Therefore, it is also important that the analyses are reproducible.

  • Investigating the replicability of preclinical cancer biology

    Authors: Timothy M Errington, Maya Mathur, Courtney K Soderberg, Alexandria Denis, Nicole Perfito, Elizabeth Iorns, Brian A Nosek
    DOI: 10.7554/eLife.71601
    Submitted by samuelpawel      

    Why should we attempt to reproduce this paper?

    This papers represents an important milestone in meta-science, as it is one of the first large-scale replication projects outside the social sciences.

  • Enhanced proton acceleration in an applied longitudinal magnetic field

    Authors: Alexey Arefiev, Toma Toncian, Gennady Fiksel
    DOI: 10.1088/1367-2630/18/10/105011
    Submitted by aarefiev    

    Why should we attempt to reproduce this paper?

    This paper provides a good learning example for intense light-matter interactions in an applied magnetic field.

    Tags: EPOCH
  • Beyond the ponderomotive limit: Direct laser acceleration of relativistic electrons in sub-critical plasmas

    Authors: A Arefiev, V Khudik, A Robinson, G Shvets, L Willingale, M Schollmeier
    DOI: 10.1063/1.4946024
    Submitted by aarefiev    
      Mean reproducibility score:   4.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    This is a review paper that discusses a ubiquitous electron acceleration mechanism. Reproducing the discussed regimes can serve as a good learning platform.

    Tags: EPOCH
  • Laser-assisted propagation of a relativistic electron bunch in air

    Authors: R M G M Trines, A P L Robinson, J R Wilkinson, J N Kirk, D S Hills, R M Deas, S Morris, T Goffrey, K Bennett, T D Arber
    DOI: 10.1088/1361-6587/ac0b9d
    Submitted by Stuart_Morris      

    Why should we attempt to reproduce this paper?

    Most electron beam physics is considered in the context of a vacuum, but there are applications to long-range electron beam transmission in air. As particle acceleration sources become more compact, we may have the chance to take particle beams out to the real world. The example provided in the paper describes that of x-ray backscatter detectors, where significantly stronger signals could be achieved by scanning objects with electron beams. This paper forms the basis for a potential new mode of particle-beam research, and it is important to ensure the reproducibility of this work for groups who wish to explore the applications of this new technology.

  • Droplet impact onto a spring-supported plate: analysis and simulations

    Authors: Michael J. Negus, Matthew R. Moore, James M. Oliver, Radu Cimpeanu
    DOI: https://doi.org/10.1007/s10665-021-10107-5
    Submitted by MNegus      
      Mean reproducibility score:   8.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    The direct numerical simulations (DNS) for this paper were conducted using Basilisk (http://basilisk.fr/). As Basilisk is a free software program written in C, it can be readily installed on any Linux machine, and it should be straightforward to then run the driver code to re-produce the DNS from this paper. Given this, the numerical solutions presented in this paper are a result of many high-fidelity simulations, which each took approximately 24 CPU hours running between 4 to 8 cores. Hence the difficulty in reproducing the results should mainly be in the amount of computational resources it would take, so HPC resources will be required. The DNS in this paper were used to validate the presented analytical solutions, as well as extend the results to a longer timescale. Reproducing these numerical results will build confidence in these results, ensuring that they are independent of the system architecture they were produced on.

  • Machine learning a model for RNA structure prediction

    Authors: Nicola Calonaci, Alisha Jones, Francesca Cuturello, Michael Sattler, Giovanni Bussi
    DOI: 10.1093/nargab/lqaa090
    Submitted by giovannibussi      

    Why should we attempt to reproduce this paper?

    The method is trained on the data that were available, but it is meant to be re-trainable as soon as new data are published. It would be great to be really sure that even someone else will be able to do it. In case we receive any feedback, we would be really happy to improve our Github repository so as to make the reproduction easier!

  • Accelerating the prediction of large carbon clusters via structure search: Evaluation of machine-learning and classical potentials

    Authors: Bora Karasulu, Jean-Marc Leyssale, Patrick Rowe, Cedric Weber, Carla de Tomas
    DOI: 10.1016/j.carbon.2022.01.031
    Submitted by bkarasulu    
    Number of reviews:   1
    Why should we attempt to reproduce this paper?

    This paper presents a fine example of high-throughput computational materials screening studies, mainly focusing on the carbon nanoclusters of different sizes. In the paper, a set of diverse empirical and machine-learned interatomic potentials, which are commonly used to simulate carbonaceous materials, is benchmarked against the higher-level density functional theory (DFT) data, using a range of diverse structural features as the comparison criteria. Trying to reproduce the data presented here (even if you only consider a subset of the interaction potentials) will help you devise an understanding as to how you could approach a high-throughput structure prediction problem. Even though we concentrate here on isolated/finite nanoclusters, AIRSS (and other similar approaches like USPEX, CALYPSO, GMIN, etc.,) can also be used to predict crystal structures of different class of materials with applications in energy storage, catalysis, hydrogen storage, and so on.

  • Automatic learning of hydrogen-bond fixes in an AMBER RNA force field

    Authors: Thorben Fröhlking, Vojtěch Mlýnský, Michal Janeček, Petra Kührová, Miroslav Krepl, Pavel Banáš, Jiří Šponer, Giovanni Bussi
    Submitted by giovannibussi      

    Why should we attempt to reproduce this paper?

    We do care about reproducibility. In case we receive any feedback, we would be really happy to improve our Github repository and/or submitted manuscript so as to make the reproduction easier!

  • Molecular Dynamics of Solids at Constant Pressure and Stress Using Anisotropic Stochastic Cell Rescaling

    Authors: Vittorio Del Tatto, Paolo Raiteri, Mattia Bernetti, Giovanni Bussi
    DOI: 10.3390/app12031139
    Submitted by giovannibussi      

    Why should we attempt to reproduce this paper?

    We do care about reproducibility. In case we receive any feedback, we would be really happy to improve our Github repository so as to make the reproduction easier!

  • Synergistic coupling in ab initio-machine learning simulations of dislocations

    Authors: Petr Grigorev, Alexandra M. Goryaeva, Mihai-Cosmin Marinica, James R. Kermode, Thomas D. Swinburnea
    DOI: https://arxiv.org/abs/2111.11262
    Submitted by jameskermode      

    Why should we attempt to reproduce this paper?

    Systematically improvable machine learning potentials could have a significant impact on the range of properties that can be modelled, but the toolchain associated with using them presents a barrier to entry for new users. Attempting to reproduce some of our results will help us improve the accessibility of the approach.

  • Sensitivity and dimensionality of atomic environment representations used for machine learning interatomic potentials

    Authors: Berk Onat, Christoph Ortner and James Kermode
    DOI: 10.1063/5.0016005
    Submitted by jameskermode      

    Why should we attempt to reproduce this paper?

    Popular descriptors for machine learning potentials such as the Behler-Parinello atom centred symmetry functions (ACSF) or the Smooth Overlap of Interatomic Potentials (SOAP) are widely used but so far not much attention has been paid to optimising how many descriptor components need to be included to give good results.

  • Encapsulated Nanowires: Boosting Electronic Transport in Carbon Nanotubes

    Authors: Andrij Vasylenko, Jamie Wynn, Paulo Medeiros, Andrew J Morris, Jeremy Sloan, David Quigley
    DOI: 10.1103/PhysRevB.95.121408
    Submitted by dquigley      
      Mean reproducibility score:   5.0/10   |   Number of reviews:   2
    Why should we attempt to reproduce this paper?

    DFT calculations are in principle reproducible between different codes, but differences can arise due to poor choice of convergence tolerances, inappropriate use of pseudopotentials and other numerical considerations. An independent validation of the key quantities needed to compute electrical conductivity would be valuable. In this case we have published our input files for calculating the four quantities needed to parametrise the transport simulations from which we compute the electrical conductivity. These are specifically electronic band structure, phonon dispersions, electron-phonon coupling constants and third derivatives of the force constants. Each in turn in more sensitive to convergence tolerances than the last, and it is the final quantity on which the conclusions of the paper critically depend. Reference output data is provided for comparison at the data URL below. We note that the pristine CNT results (dark red line) in figure 3 are an independent reproduction of earlier work and so we are confident the Boltzmann transport simulations are reproducible. The calculated inputs to these from DFT (in the case of Be encapsulation) have not been independently reproduced to our knowledge.

  • New Insight into the Stability of CaCO3 Surfaces and Nanoparticles via Molecular Simulation

    Authors: A. Matthew Bano, P. Mark Rodger, and David Quigley
    DOI: 10.1021/la501409j
    Submitted by dquigley      

    Why should we attempt to reproduce this paper?

    The negative surface enthalpies in figure 5 are surprising. At least one group has attempted to reproduce these using a different code and obtained positive enthalpies. This was attributed to the inability of that code to independently relax the three simulation cell vectors resulting in an unphysical water density. This demonstrates how sensitive these results can be to the particular implementation of simulation algorithms in different codes. Similarly the force field used is now very popular. Its functional form and full set of parameters can be found in the literature. However differences in how different simulation codes implement truncation, electrostatics etc can lead to significant difference in results such as these. It would be a valuable exercise to establish if exactly the same force field as that used here can be reproduced from only its specification in the literature. The interfacial energies of interest should be reproducible with simulations on modest numbers of processors (a few dozen) with run times for each being 1-2 days. Each surface is an independent calculation and so these can be run concurrently during the ReproHack.

  • Thermodynamics of stacking disorder in ice nuclei

    Authors: David Quigley
    DOI: 10.1063/1.4896376
    Submitted by dquigley      
      Mean reproducibility score:   3.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    The results of this paper have been used in multiple subsequent studies as a benchmark against which other methods of performing the same calculation have been tested. Other groups have challenged the results as suffering from finite size effects, in particular the calculations on mixtures of cubic and hexagonal ice. Should there be time during in the event, participants could check this by performing calculations on larger unit cells. Each individual calculation should converge adequately within 96 hours making it amenable to a HPC ReproHack. Given modern HPC hardware many such calculations could be run concurrently on a single HPC node.

  • Highly efficient conversion of laser energy to hard X-rays in high intensity laser-solid simulations

    Authors: S. Morris, A. Robinson, C. Ridgers
    DOI: 10.1063/5.0055398
    Submitted by Stuart_Morris      

    Why should we attempt to reproduce this paper?

    There are many applications to multi-MeV X-rays. Their penetrative properties make them good for scanning dense objects for industry, and their ionising properties can destroy tumours in radiotherapy. They are also around the energy of nuclear transitions, so they can trigger nuclear reactions to break down nuclear waste into medical isotopes, or to reveal smuggled nuclear-materials for port security. Laser-driven X-ray generation offers a compact and efficient way to create a bright source of X-rays, without having to construct a large synchrotron. To fully utilise this capability, work on optimising the target design and understanding the underlying X-ray mechanisms are essential. The hybrid-PIC code is in a unique position to model the full interaction, so its ease-of-use and reproducibility are crucial for this field to develop.

  • PlanGAN: Model-based Planning With Sparse Rewards and Multiple Goals

    Authors: Henry Charlesworth and Giovanni Montana
    Submitted by gmontana74      
      Mean reproducibility score:   10.0/10   |   Number of reviews:   1
    Why should we attempt to reproduce this paper?

    This paper proposes a probabilistic planner that can solve goal-conditional tasks such as complex continuous control problems. The approach reaches state-of-the-art performance when compared to current deep reinforcement learning algorithms. However, the method relies on an ensemble of deep generative models and is computationally intensive. It would be interesting to reproduce the results presented in this paper on their robotic manipulation and navigation problems as these are very challenging problems that current reinforcement learning methods cannot easily solve (and when they do, they require a significantly larger number of experiences). Can the results be reproduced out-of-the-box with the provided code?