Review of
"A multi-level analysis of data quality for formal software citation"

Review of "A multi-level analysis of data quality for formal software citation"

Submitted by ele_abu_101  

Nov. 23, 2023, 3:16 p.m.

Lead reviewer


Review Body


Did you manage to reproduce it?
Fully Reproducible
Reproducibility rating
How much of the paper did you manage to reproduce?
10 / 10
Briefly describe the procedure followed/tools used to reproduce it

I downloaded relevant files from the github to my local computer. I installed Quarto. I ensured that my project directory was properly setup to ensure that it referenced the location where relevant data was stored.

All relevant libraries were installed and loaded. The 'easyalluvial' package has been removed from CRAN, so I had to install it from an archive zipped file.

Briefly describe your familiarity with the procedure/tools used by the paper.

I had not used Quarto until now. However, once I found it is similar to RMarkdown, which I am very familiar with, I got a handle on the quarto document. I enjoyed the experience.

Which type of operating system were you working in?
Windows Operating System
What additional software did you need to install?

Quarto and easyalluvial

What software did you use

R studio

What were the main challenges you ran into (if any)?

I made some enquiries on loading the quarto document. I was careful to configure my working directory accordingly. In particular, I ensured relevant data could be accessed by the codes.

What were the positive features of this approach?

This ensured all codes could run hitch-free to reproduce the required results.

Any other comments/suggestions on the reproducibility approach?


Documentation rating
How well was the material documented?
10 / 10
How could the documentation be improved?

The documentation is sufficient for anyone adept at using R.

What do you like about the documentation?

A html compilation of the same document is provided, which contains relevant details for an interested reader/user of the codes.

After attempting to reproduce, how familiar do you feel with the code and methods used in the paper?
9 / 10
Any suggestions on how the analysis could be made more transparent?


Reusability rating
Rate the project on reusability of the material
10 / 10
Permissive Data license included:  
Permissive Code license included:  

Any suggestions on how the project could be more reusable?

Aside from the quarto document, multiple formats of the code could have been provided, including raw r scripts, and an R markdown file. Nonetheless, the refusal to do this has motivated me to learn how to use a new documentation program (quarto) which apparently is more robust than the R Markdown I am familiar with.

Any final comments

The paper has excellent plots, well laid out in the quarto markdown document.