As a general rule, this course is graded on a “Check Plus” / “Check” / “Check Minus” / “Fail” grading scale. These roughly correspond to a 4.0, 3.5, 3.0, and 1.0, respectively.

**Check Plus**: The work represents a*sophisticated*understanding of the concepts and a sophisticated execution of the techniques.**Check**: The work represents a*competent*understanding of the concepts and a competent execution of the techniques.**Check Minus**: The work is*not*yet at a the level of a compentent understanding or an compentent execution.**Fail**: The work is missing key parts or was otherwise incomplete.

Below, I roughly describe criteria for what sophisticated, compentent, and not competent are for different aspects of the assignments in this course.

**Sophisticated** statistics use appropriate statistics for the argument being made. The statistics used help make
patterns in the data clear to the reader. Appropriate units are reported for all statistics, and are interpreted in
terms of the research questions and real-world significance. Often, multiple different statistics are used to show how
different analyses show different aspects of the data, and these are explained clearly. Assumptions are understoond, and
arguments are made about how the data either meets the assumptions, or how the assumption violation might affect (or not
affect) the results. Uses simpler statistics when appropriate to make arguments clearer, but backs them up with more
sophisticated statistics as necessary.

**Competent** statistics uses an appropriate statistic for the type of data (e.g. t-test for two-group continuous data). It
correctly and validly presents the results of the statistic and interprets it in light of the reseearch question(s).
The data and use meets appropriate assumptions for the statistic. Examines the effect size appropriately rather than
just focusing on statistical significance.

**Not yet competent** statistics do not match statistic choice to the data. These statistics throw away or ignore
important parts of the data (for example, binning a continuous variable to do a chi-square test) without having a good
reason. Choosing statistics based on their ability to provide “stars” or p-values. Following statistical instructions
because some book or guide tells you do without understanding why.

**Sophisticated** use of hypothesis tests is as a secondary argument. The main argument in the paper should be about
the real-world effect sizes, and the hypothesis tests are either not used, or only used to discuss the amount of noise
in the data. Units are always taken into account, and p-values are never compared to each other nor discussed as “very
significant” or “marginally significant”.

**Competent** use of hypothesis tests combines hypothesis tests with effect sizes to make arguments. While a hypothesis
test might help identify whether the analyst should pay attention to a result, the analyst also examines the effect size
to identify the meaning of the difference. Units are always taken into account, and p-values are never compared to each
other nor discussed as “very significant” or “marginally significant”.

**Not yet competent** use of hypothesis tests only focuses on the statistical significance (or “stars”), and ignores the
actual effect size. The absolute value of the p-value is given outsized meaning (such as “very significant” or
“marginally significant”), and p-values may be compared with one another (or a “significant” finding compared with a
“non-significant” finding).

**Sophisticated** modeling uses appropriate transformations and explanations to help the reader understand the results.
Categorical variables have their baseline level carefully chosen for clear analyses, and it is always recognized that
coefficients are comparisons against that baseline. Binary variables are named appropriately for easy interpretation.
Continuous variables have clear and understandable units, and are interpreted correctly in real-world units. All model
parameters are reported, and the model is parsimonious. There is a discussion of control variables, and what it means
to control for those variables.

**Competent** modeling reports on complete regression models, and includes coefficients for all model parameters.
Categorical variables are correctly interpreted as comparisons against a baseline. Continuous variables are reported
with units. Interactions are correctly interpreted, and main effects are interpreted correctly relative to other
predictors.

**Not yet competent** modeling talks about how variables are “significant” rather than the estimates. Categorical
variables are described as if they are significant themselves, without reference to a baseline level. Continuous
variables are discussed without identifying the units of the variable. Extra variables are included in the model but
are not interpreted. The control provided by other variables is ignored.

**Sophisticated** analysis includes a reproducible file that combines code to produce statistical results, the actual
statistical results themselves, and text interpreting those results in one easy-to-use place. Re-compiling or re-running
the statistics produces the same results, even when run on other people’s computers. Includes all relevant calculations
in an way that is easy to match up with the writeup, and possibly includes additional relevant statistics with
explanation that didn’t make it into the paper, but does not include irrelevant analyses.

**Competent** analysis can be reproduced, but not necessarily easily. A script or set of instructions exists to
reproduce the analysis, and it works correctly on the computer of the analyst (but, unknown whether it produces the same
results on other computers with possibly different settings). The output includes all relevant analyses, but requires
looking through the file to match up results with the writeup.

**Not yet competent** analyses cannot be easily reproduced; they require the analyst to figure out how to run the
analysis again and do not document exactly how the statistics were calculated the original time. They may also include
additional incorrect or irrelevant analyses in the reproducible file without labeling these analyses as potentially
misleading.

**Sophisticated** writeups make a clear argument that is backed up with evidence, and are clear and easy to understand.
They are of a professional quality; they contain no spelling errors and are gramatically correct. All tables and graphs
are easy to read (including appropriate font size) and all numbers are clearly explained. Additionally, all tables and
graphs include a discussion of what the reader should learn from them. Tables and graphs are explained but not
duplicating material in the text. The text completely answers all relevant questions with evidence, but without
unnecessary repetition of content or irrelevant text. Organized to clearly make argument. It is writen in first person
active voice.

**Competent** writeups contain a clearly developed ideas in a logical sequence. They use graphs and tables to provide
data as appropriate, and they are easy to read and provide relevant data. Very few spelling or gramatical errors, but
still easy to understand. Professional appearance. May contain some unnecessary repetition or irrelevant text, but that
doesn’t hinder understanding the main point. Numbers are explained clearly and include appropriate units.

**Not yet competent** writeups are disorganized and difficult to follow; they do not make a clear argument or back up
that argument with clear evidence. They can appear unprofessional, with numerous spelling errors or grammatical
mistakes. Tables or graphs are confusing and difficult to read (small font, too much in them, etc.). Doesn’t interpret
or explain graphs, tables, or statistics but instead leaves it up to the reader to figure out what they mean. It is
written in third person passive voice.