Review of
"Southern Ocean deep mixing band emerges from a competition between winter buoyancy loss and upper stratification strength"

Review of "Southern Ocean deep mixing band emerges from a competition between winter buoyancy loss and upper stratification strength"

Submitted by jwallwork  

March 12, 2024, 5:16 p.m.

Lead reviewer

jwallwork

Review team members

jakharkaran

Review Body

Reproducibility

Did you manage to reproduce it?
Fully Reproducible
Reproducibility rating
How much of the paper did you manage to reproduce?
7 / 10
Briefly describe the procedure followed/tools used to reproduce it
  • Downloaded code as a tarball.
  • Installed Apptainer using PPA approach.
  • Downloaded zipped data from Zenodo using the approach provided.
  • Generated plots using the Snakemake approach provided.
Briefly describe your familiarity with the procedure/tools used by the paper.
  • Lots of experience with Python and Pip.
  • No previous experience with Poetry (recommended installation approach).
  • No previous experience with Apptainer (alternative approach used).
  • No previous experience with Snakemake.
Which type of operating system were you working in?
Linux/FreeBSD or other Open Source Operating system
What additional software did you need to install?
  • Apptainer
What software did you use
  • Apptainer
  • Snakemake
What were the main challenges you ran into (if any)?
  • Difficulties installing using the default approach, in particular using Pyproj 3.2.1, which led to a Cython compilation error.
  • Problems with downloading all of the data: (a) the descriptions for providing credentials for two of the data sets are a little unclear (see later), (b) insufficient time to download all of the data sets.
What were the positive features of this approach?
  • The alternative installation approach (Apptainer) worked seamlessly.
  • Great to be able to download all data and generate all plots using single commands.
Any other comments/suggestions on the reproducibility approach?

Documentation

Documentation rating
How well was the material documented?
9 / 10
How could the documentation be improved?
  • If possible, a fix for the recommended installation approach, so that use of containers can be avoided. The author mentioned a potential fix for this by using a particular Cython version.
  • Clearer instructions on how to get authorisation to download the ARMOR3D and ECCO data sets - what service do the username and password mentioned relate to and would one set these up?
What do you like about the documentation?
  • Great to have alternative installation approaches available.
  • It was nice to be able to download a subset of the data from Zenodo, which enabled me to generate most of the plots from the paper.
After attempting to reproduce, how familiar do you feel with the code and methods used in the paper?
4 / 10
Any suggestions on how the analysis could be made more transparent?

Still unfamiliar with how Apptainer and Snakemake work because I've never used them before today!


Reusability

Reusability rating
Rate the project on reusability of the material
10 / 10
Permissive Data license included:  
Permissive Code license included:  

Any suggestions on how the project could be more reusable?


Any final comments
  • Overall, the process went smoothly (once I'd decided to take the Apptainer approach). Nice work!
  • I am confident that the only reasons I was unable to generate the remaining figures were (a) time constraints, (b) inability to access the ARMOR3D and ECCO data sets.