Review of
"Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA"

Review of "Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA"

Submitted by a.aslam2@leeds.ac.uk  

Nov. 23, 2023, 3:48 p.m.

Lead reviewer

a.aslam2@leeds.ac.uk

Review Body

Reproducibility

Did you manage to reproduce it?
Partially Reproducible
Reproducibility rating
How much of the paper did you manage to reproduce?
4 / 10
Briefly describe the procedure followed/tools used to reproduce it

Read/Overview of Paper

Gone through code

Ran two Jupyter Notebooks successfully, though Raw data was absent

Tried third notebook and were not successful

Briefly describe your familiarity with the procedure/tools used by the paper.

Team of four with experience (+7 years) with Python, Notebooks, environments and statistics.

Familiar with pandas and other libraries. Not familiar with PyStan or Baysian statistics.

Which type of operating system were you working in?
Windows Operating System
What additional software did you need to install?

pystan (but didnt work) cython C++

What software did you use

Jupyter Notebook vscode Anaconda

What were the main challenges you ran into (if any)?
  1. Raw data files are missing

  2. Installing pystan. When searching online: -- Install PyStan with pip install pystan. PyStan runs on Linux and macOS. You will also need a C++ compiler such as gcc ≥9.0 or clang ≥10.0. The following block of code shows how to use PyStan with a model which studied coaching effects across eight schools (see Section 5.5 of Gelman et al (2003)). -- PyStan on Windows requires Python 2.7/3. x and a working C++ compiler. If you have already installed Python and the MingW-w64 C++ compiler, running pip install pystan will install PyStan

Tried https://pystan2.readthedocs.io/en/latest/windows.html

What were the positive features of this approach?

Notebooks are quite informative

Got to learn how to do a good survey with data analysis

Any other comments/suggestions on the reproducibility approach?

Documentation

Documentation rating
How well was the material documented?
8 / 10
How could the documentation be improved?

could include - raw data or remove section from notebook that require raw data - requirements/config file

What do you like about the documentation?

Documentation, Clear

After attempting to reproduce, how familiar do you feel with the code and methods used in the paper?
6 / 10
Any suggestions on how the analysis could be made more transparent?

We didnt manage to reproduce so cant comment, please see above for other suggestions which can help in reproducibility.


Reusability

Reusability rating
Rate the project on reusability of the material
7 / 10
Permissive Data license included:  
Permissive Code license included:  

Any suggestions on how the project could be more reusable?

Please see above comments



Any final comments

Thank you.