Review of
"Don't Hold My Data Hostage - A Case For Client Protocol Redesign"

Review of "Don't Hold My Data Hostage - A Case For Client Protocol Redesign"

Submitted by gmuradyan  

June 30, 2024, 2:07 p.m.

Lead reviewer

gmuradyan

Review team members

AliY2000

Review Body

Reproducibility

Did you manage to reproduce it?
Not Reproducible
Reproducibility rating
How much of the paper did you manage to reproduce?
0 / 10
Briefly describe the procedure followed/tools used to reproduce it

To reproduce the data, the reviewer must download a virtual machine image and run the tests andbenchmarks. As it is not mentioned which specific program to use to run the image, we used 'Oracle VMVirtualbox'. After several attempts to import the downloaded image using Virtualbox, we were unsuccessfuldue to the E_INVALIDARG (0X80070057) error.

Briefly describe your familiarity with the procedure/tools used by the paper.

Working with virtual machine images is moderately familiar to us.

Which type of operating system were you working in?
Windows Operating System
What additional software did you need to install?

Oracle VM Virtualbox

What software did you use

Oracle VM Virtualbox

What were the main challenges you ran into (if any)?

The error "E_INVALIDARG (0X80070057)" when importing the image with Oracle VM Virtualbox

What were the positive features of this approach?

According to the authors, the reviewer only needs to download and run the image to reproduce the benchmark results,which we believe is a well-designed, low-error and automated approach.

Any other comments/suggestions on the reproducibility approach?

Please fix the error or mention which specific application should be used to run the image.


Documentation

Documentation rating
How well was the material documented?
6 / 10
How could the documentation be improved?

Documentation lacks information on which application to use for VM image

What do you like about the documentation?

The documentation is detailed enough and seems sufficient to reproduce the benchmark results.

After attempting to reproduce, how familiar do you feel with the code and methods used in the paper?
0 / 10
Any suggestions on how the analysis could be made more transparent?

We cannot comment on the analysis as we have not been able to reproduce it.


Reusability

Reusability rating
Rate the project on reusability of the material
0 / 10
Permissive Data license included:  
Permissive Code license included:  

Any suggestions on how the project could be more reusable?

Include licence for code



Any final comments