Review of
"Comparisons of Citizen Science Data-Gathering Approaches to Evaluate Urban Butterfly Diversity"

Review of "Comparisons of Citizen Science Data-Gathering Approaches to Evaluate Urban Butterfly Diversity"

Submitted by Gabriele  

March 15, 2022, 1:50 p.m.

Lead reviewer

ludmillafi

Review team members

Gabriele

Review Body

Reproducibility

Did you manage to reproduce it?
Fully Reproducible
Reproducibility rating
How much of the paper did you manage to reproduce?
10 / 10
Briefly describe the procedure followed/tools used to reproduce it

We downloaded the files stored in Zenodo (link in paper), opened the .Rproj locally (in RStudio) and simply ran the RNotebook files following the order that they appeared in the main text.

Briefly describe your familiarity with the procedure/tools used by the paper.

We are both fairly familiar with R, and general structure of a reproducible project (having .Rproj, README files). One of us was not familiar with the field, but that did not hamper the reproduction of the study.

Which type of operating system were you working in?
Windows Operating System
What additional software did you need to install?

No software, but a couple of R packages.

What software did you use

R (and RStudio).

What were the main challenges you ran into (if any)?

We had minor challenges. These relate to identifying how the content of the notebooks related to specific sections of the main text. However, the notebooks were so well documented that we managed to do it by reading the contents.

What were the positive features of this approach?
  • The data and scripts are organized in a Zenodo repository, with a fairly informative README file
  • Results of statistical analyses are directly reported from the code, in text form, which is great because it avoid any human error in the transfer
  • Tables are tidy, with clear column names
Any other comments/suggestions on the reproducibility approach?
  • The main text could reference the work done in the notebooks more, specially because the later are so well documented
  • There should be a list of the R packages necessary to reproduce the work
  • All packages should be loaded at once, rather than repeatedly in all notebooks (probably use a script for it). There is no package that is problematic, which would justify having it loaded only if absolutely necessary.
  • There should be be list of the packages and OS versions used when compiling the notebooks (possible with the R function Sys.info())
  • The file bioscan-functions.R should be in the scripts folder

Documentation

Documentation rating
How well was the material documented?
10 / 10
How could the documentation be improved?
  • README.md or, at least, the main text should inform the order to run the RNotebooks (.Rmd files)
What do you like about the documentation?
  • Functions are very well documented, following R guidelines. As are the notebooks, all the reasoning of the researchers is explained in them.
After attempting to reproduce, how familiar do you feel with the code and methods used in the paper?
10 / 10
Any suggestions on how the analysis could be made more transparent?

None, actually the text is a good example of how to document code and make the reasoning behind analyses very explicit.


Reusability

Reusability rating
Rate the project on reusability of the material
10 / 10
Permissive Data license included:  
Permissive Code license included:  

Any suggestions on how the project could be more reusable?
  • Include the MIT licensing used in the author's github page in the Zenodo repository. The one currently in Zenodo is not clear (link seems to be broken)
  • The pipeline of downloading and pre-processing of open data could be (from iNaturalist) could be made more explicit


Any final comments