Review of
"Mental Health and Social Contact During the COVID-19 Pandemic: An Ecological Momentary Assessment Study"

Review of "Mental Health and Social Contact During the COVID-19 Pandemic: An Ecological Momentary Assessment Study"

Submitted by AnjaEggert  

Aug. 31, 2022, 9:40 a.m.

Lead reviewer

AnjaEggert

Review Body

Reproducibility

Did you manage to reproduce it?
Partially Reproducible
Reproducibility rating
How much of the paper did you manage to reproduce?
7 / 10
Briefly describe the procedure followed/tools used to reproduce it

Followed link on PsyArXiv to https://osf.io/mvdpe/, there download of zip-file containing all files (preprint itself, data, code, figures etc.)

Briefly describe your familiarity with the procedure/tools used by the paper.

very familiar with R :)

Which type of operating system were you working in?
Windows Operating System
What additional software did you need to install?

R packages:

  • mlVAR, summarytools, lm.beta, Rmpfr
What software did you use

sessionInfo() R version 4.2.0 (2022-04-22 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19043)

other attached packages: lm.beta_1.6-2, summarytools_1.0.1 tidyselect_1.1.2, tidyr_1.2.0, lme4_1.1-30, Matrix_1.4-1, viridis_0.6.2, viridisLite_0.4.0, reshape_0.8.9, bootnet_1.5, qgraph_1.9.2, mlVAR_0.5, dplyr_1.0.9, ggplot2_3.3.6

What were the main challenges you ran into (if any)?

No main challenges as the files, the folder structure, the names of files and folders are very clear. The one main point are missing data for fitting the mlVAR network models (orthogonal and correlated estimation).

What were the positive features of this approach?

Folder structure, the names of files and folders are very clear.

Any other comments/suggestions on the reproducibility approach?
  • Timeseries.R: using geom_jitter() for the plots (Fig. 3) causes different position of the dots - this could be solved with set.seed()
  • Network models.R: I cannot fit mlVAR network models: orthogonal and correlated estimation as data sets network_orthogonal.RData and network_correlated.RData are not available. Therefore I cannot reproduce Fig. 4
  • Compare pre-post.R: many descriptive measures are calculated here, most written in the 1st result section. It is difficult to match the numbers generated in the script with the text, also because not all of the measures/figures are given in the paper.
  • COVID cases NL.R: no remarks.
  • Multilevel regression.R:

Documentation

Documentation rating
How well was the material documented?
8 / 10
How could the documentation be improved?

  • It would be helpful to write a README to give general information.

  • Generally, version numbers of packages should be given. I suggest to use session.Info().

  • Even though folders and files are clearly organized, it would be helpful to "adjust" the code more to the final paper, i.e. figure numbers in the paper could be given in the code.

What do you like about the documentation?

Clearly organized files and code.

After attempting to reproduce, how familiar do you feel with the code and methods used in the paper?
10 / 10
Any suggestions on how the analysis could be made more transparent?

Reusability

Reusability rating
Rate the project on reusability of the material
7 / 10
Permissive Data license included:  
Permissive Code license included:  

Any suggestions on how the project could be more reusable?

CC-By Attribution 4.0 International



Any final comments

I thank the authors very much for contributing their paper to ReproHack. I enjoyed reading the paper and reproducing the data analysis.