Review of
"Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA"

Review of "Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA"

Submitted by lluisb3  

June 24, 2024, 1:42 p.m.

Lead reviewer

lluisb3

Review Body

Reproducibility

Did you manage to reproduce it?
Partially Reproducible
Reproducibility rating
How much of the paper did you manage to reproduce?
8 / 10
Briefly describe the procedure followed/tools used to reproduce it

I had to create my conda environment to run the code. I used pipreqs to scan the project that outputs the following requirements.txt file:

matplotlib==3.9.0 numpy==2.0.0 pandas==2.2.2 plotly==5.22.0

pystan==3.9.1

scipy==1.13.1 seaborn==0.13.2

As mentioned in the paper, I commented pystan and installed it afterward with version 2.17.1.0. Cython was needed for the installation as well.

Briefly describe your familiarity with the procedure/tools used by the paper.

I am not very familiar with the statistical procedures/tools used in the paper. I am familiar with Python programming language.

Which type of operating system were you working in?
Linux/FreeBSD or other Open Source Operating system
What additional software did you need to install?

I used Miniconda to create my conda environment.

What software did you use

I used Miniconda and vscode to run the Jupyter notebooks. In the environment, I installed Python 3.9 and the dependencies listed by pipreqs with specific version for pystan and Cython

What were the main challenges you ran into (if any)?

The main challenge was creating the environment to run the code.

What were the positive features of this approach?

The Jupiter notebooks format was an easy setup to reproduce the results.

Any other comments/suggestions on the reproducibility approach?

I will suggest providing a requeriments.txt file with the specific versions used in the project. Ideally providing a docker image.


Documentation

Documentation rating
How well was the material documented?
8 / 10
How could the documentation be improved?

Providing a requirements.txt file with the necessary dependencies.

What do you like about the documentation?

I like the simplicity and the fact that the folder structure in the project is well-defined and explained.

After attempting to reproduce, how familiar do you feel with the code and methods used in the paper?
7 / 10
Any suggestions on how the analysis could be made more transparent?

Reusability

Reusability rating
Rate the project on reusability of the material
8 / 10
Permissive Data license included:  
Permissive Code license included:  

Any suggestions on how the project could be more reusable?


Any final comments