Review of
"Thermodynamics of stacking disorder in ice nuclei"

Review of "Thermodynamics of stacking disorder in ice nuclei"

Submitted by aravinthen  

April 1, 2022, 9:39 a.m.

Lead reviewer

aravinthen

Review Body

Reproducibility

Did you manage to reproduce it?
Partially Reproducible
Reproducibility rating
How much of the paper did you manage to reproduce?
3 / 10
Briefly describe the procedure followed/tools used to reproduce it

The code with which the results of the paper were obtained was written in Fortran and came with examples upon which it could be easily tested. I ran the code with one of the examples and reproduced a reasonable value for one of the systems described in the paper.

Briefly describe your familiarity with the procedure/tools used by the paper.

I had basic familiarity with Fortran, enough to read and understand the code (the code was written in a way that anybody would have little difficulty deciphering it). However, I had no familiarity with the technique itself.

Which type of operating system were you working in?
High Performance Computing Cluster
What additional software did you need to install?

Nothing else.

What software did you use

Fortran and OpenMP/MPI for the provided code, Python for the work required to show reproducibility.

What were the main challenges you ran into (if any)?

I was unsure at first on how I should run the code: I don’t frequently use high-performance codes in my own research and was unfamiliar with job submission. The rest of the people in my group did not have this problem.

The makefile had to be updated in a few places for the code to be built: it initially included a module that was no longer part of the software. This was fixed by simply removing that module from the makefile.

A single line of the code had to be modified to work on the HPC cluster: exit code “15” had to be changed to “SIGTERM”.

What were the positive features of this approach?

I appreciated how self-contained the code was - all I had to do was load two modules and make two reasonably intuitive changes: the rest of the code basically ran itself. We were fortunate in this regard when compared to the other groups who had to spend hours loading dependencies and building software.

Any other comments/suggestions on the reproducibility approach?

A few lines of detail regarding the makefile would have been helpful, as well as instructions on how the code should be run.


Documentation

Documentation rating
How well was the material documented?
2 / 10
How could the documentation be improved?

Existing in the first place!

The code was so thoroughly commented that it could essentially be considered self-documenting software.

What do you like about the documentation?

I read all of it in no time at all.

The comments within the code, however, were incredibly useful. It was also very consistently structured, so navigating the code base was effortless.

After attempting to reproduce, how familiar do you feel with the code and methods used in the paper?
5 / 10
Any suggestions on how the analysis could be made more transparent?

I feel as though I understand the code, but I do not fully understand the technique that the paper uses. This is probably due to my near complete unfamiliarity with the field.


Reusability

Reusability rating
Rate the project on reusability of the material
7 / 10
Permissive Data license included:  
Permissive Code license included:  

Any suggestions on how the project could be more reusable?

A wider range of structure files included within the examples would have been very helpful.



Any final comments