Lab: Peer Review

due Fri, Apr 16 at 11:59 PM

Critically reviewing others’ work is a crucial part of the scientific process, and STA 199 is no exception. On Mon Apr 12 you will all be give read access to another project repo and have until Fri Apr 16 8:00 AM to provide a detailed critique of the written report and data analysis. This review is intended to help you create a high quality final project, as well as give you experience reading and constructively critiquing the work of others.

Before attending lab

Before attending lab, carefully read the guidelines for the final written report.

Then, search your GitHub repositories for project with the suffix given by the team name for the project you will be reviewing. Peer review assignments are available on Sakai here. Open the repo and follow the usual steps to clone into RStudio. You have read access to this repository but will not be able to push any changes.

Carefully read the project and consider the questions below. For each, answer “Yes”, “Somewhat”, or “No” and address any issues if you answer “Somewhat” or “No”.

Introduction and Data

• Is the research question and goal of the report clearly stated?
• Does the introduction provide appropriate background context and motivation for a general reader?
• Is the source of the data stated with an appropriate citation?
• Is it clear when and how the data was collected?
• Is data manipulation described clearly (missing data, creation of new variables, etc)?
• Are the cases and relevant variables described?

Methodology

• Do the visualizations correspond to the stated research question?
• Are visualizations effective and do they follow the visualization principles we have discussed in STA 199 (including elements like titles, labels, appropriate for the type of data, etc)?
• Is the choice of statistical method justified?

Results

• Are the chosen techniques for answering the research question appropriate for the research context and type of data?
• Is the research question answered effectively?

Discussion

• Is the answer to the research question summarized and supported by statistical arguments?
• Are limitations of the analysis clearly outlined?

General

• Is the writing clear (including elements like spelling, grammar, etc)? Are you able to follow what is being done?
• Is the coding clear? Are you able to follow precisely what is being done?
• Are you able to reproduce all aspects of the report, including output, visualizations, etc? Have the reproducibility principles we have discussed in STA 199 been followed?
• Is the report well-formatted and readable (including layout but also only reporting relevant output, with no extraneous code, visuals, etc)?
• Have they appropriate outlined the next steps with gaps clearly defined?
• Any suggestions for them moving forward?

Final Considerations

• What is one question you have for the group after reading their analysis?
• What is one thing the group has done well?

After you have finished your close reading of the report and consideration of the questions above it is time to provide feedback.

Go to the project repo on GitHub and click “Issues”. For each comment or critique on the written report and data analysis, click “New Issue”. Provide an informative issue title and then a description outlining your concerns, comments, and any suggested changes. Click “Submit new issue”. Do this for every major comment or critique. Do not submit a single issue with all of your feedback.

You have until Fri Apr 16 8:00 AM to provide your detailed critique of the written report and data analysis via issues in GitHub.

During lab

All teams attending synchronous labs are required to attend lab on Fri Apr 16. For the first portion of lab, you will meet with the team you peer-reviewed to discuss comments, ask for clarification, and receive “in-person” feedback on your report. For students taking the course asynchronously, set a time outside of class to meet or coordinate over email.

The remainder of lab time is devoted to project work with TA support.

After lab

After you have received feedback, examine the GitHub issues submitted by the other team. Consider their comments and critiques, make changes as necessary, then respond, and mark the issue as closed.

Submission

The peer review will be graded on the extent to which it comprehensively and constructively addresses the components of the partner team’s report: the research context and motivation, exploratory data analysis, reproducibility, and any inference, modeling, or conclusions.

You will be graded based on the submitted issues on GitHub.

All team members must meaningfully contribute to the peer review via submitting issues