1. specify() the Response Variable

In this case, our response variable is Change. The specify() function returns a tibble of the specified variables as an “infer” class. For now, we’ll just view the output of specify() using head().

Students_wide %>%
  specify(response = Change) %>%
  head()

## Response: Change (numeric)
## # A tibble: 6 x 1
##   Change
##    <dbl>
## 1    0.6
## 2   -0.6
## 3    2.5
## 4    1.3
## 5    0.2
## 6    2.1

2. Declare the Null Hypothesis with hypothesize()

Our null and alternative hypotheses are \[ H_0: \mu_{\text{Change in GPA}} = 0 \]

\[ H_a: \mu_{\text{Change in GPA}} > 0 \]

Because we are looking at the mean of one variable, we will set null = "point" and mu = 0.

Students_wide %>%
  specify(response = Change) %>%
  hypothesize(null = "point", mu = 0) %>%
  head()

## Response: Change (numeric)
## Null Hypothesis: point
## # A tibble: 6 x 1
##   Change
##    <dbl>
## 1    0.6
## 2   -0.6
## 3    2.5
## 4    1.3
## 5    0.2
## 6    2.1

3. generate() the Null Distribution

Using a bootstrap sample, the generate() function returns a tibble that contains generated data that reflects the null hypothesis.

Students_wide %>%
  specify(response = Change) %>%
  hypothesize(null = "point", mu = 0) %>%
  generate(reps = 1000, type = "bootstrap") %>%
  head() %>%
  pander()

4. calculate() the Observed t-statistics and compare to Null Distribution

First, we’ll want to calculate our observed statistic:

observed_statistic <- Students_wide %>%
  specify(response = Change) %>%
  hypothesize(null = "point", mu = 0) %>%
  calculate(stat = "t")

observed_statistic %>% pander()

The observed statistic from our data is 11.83 We want to know how likely it would be to get a test stat this extreme if the null hypothesis was true. We can do this by calculating this t stat for the null distribution. We can also visualize this by using the visualize() and shade_p_value() functions.

Students_wide %>%
  specify(response = Change) %>%
  hypothesize(null = "point", mu = 0) %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "t") %>%
  
  visualize()  + 
  shade_p_value(observed_statistic,
                direction = "greater")

It looks like our observed stat would be very unlikely if the true mean GPA improvement was 0 points.

5. get_p_value() from the test statistic

Students_wide %>%
  specify(response = Change) %>%
  hypothesize(null = "point", mu = 0) %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "t") %>%
  get_p_value(obs_stat = observed_statistic,
              direction = "greater") %>%
  pander()

The p-value was very very small and rounded to 0. This means that the probability of seeing a test statistic as extreme or more extreme as our observed test statistic (11.86) is approximately 0, which which is below our significance level of 0.05. There is significant evidence to reject the null hypothesis in favor of our alternative hypothesis that the average student increased their semester GPA from Pre 1 to Post 1.

Wrapper Function t_test()

Another way to perform this t-test is the use the wrapper function provided in the infer package. This function is much quicker and simpler, but does not allow for quick visualization like the above workflow.

To use t_test(), we’ll pass in our response variable, mu = 0, and alternative = greater. It’ll perform the test in the background and return the test statistic, degrees of freedom, p_value, direction of alternative hypothesis, and a 95% confidence interval.

Students_wide %>%
  t_test(response = Change, 
         mu = 0, 
         alternative = "greater") %>%
  pander()

We see the same results as before, along with the exact value for our p-value (5.155e-28) and our 95% confidence interval. It can be said with 95% confidence that a Tier 3 student would improve their GPA from Pre_1 to Post_1 by 0.74 points or more.

1. calculate() our observed statistic

We’ll need to specify our explanatory variable too, which is Group. We also want to provide an order to subtract the means in so that a positive difference means that Mentored students improve more than Control students.

observed_statistic <- Students_wide %>%
  specify(response = Change, 
          explanatory = Group) %>%
  calculate(stat = "t",
            order = c("Mentored", "Control"))

observed_statistic %>% pander()

The observed statistic from our data is 3.539. We want to know how likely it would be to get a test stat this extreme if the null hypothesis was true and there was no relationship between what group a student is in and how much they improve. We can do this by generating the null distribution. We can also visualize this by using the visualize() and shade_p_value() functions.

2. generate() a null distribution

Because this is now an independent samples t-Test, we need to declare our new null hypothesis with hypothesize(null = "independence"). We also need to change the sample type to be type = "permute"

Students_wide %>%
  specify(response = Change, 
          explanatory = Group) %>%
  hypothesize(null = "independence") %>%
  generate(reps = 1000, type = "permute") %>%
  calculate(stat = "t",
            order = c("Mentored", "Control")) %>%
  visualize()  + 
  shade_p_value(observed_statistic,
                direction = "greater")

It looks like our observed stat would be very unlikely if the null hypothesis was true and there was no relationship between what group a student is in and how much they improve.

3. get_p_value() from the test statistic

Students_wide %>%
  specify(response = Change, 
          explanatory = Group) %>%
  hypothesize(null = "independence") %>%
  generate(reps = 1000, type = "permute") %>%
  calculate(stat = "t",
            order = c("Mentored", "Control")) %>%
  get_p_value(obs_stat = observed_statistic,
              direction = "greater") %>%
  pander()

The p-value was very very small and rounded to 0. This means that the probability of seeing a test statistic as extreme or more extreme as our observed test statistic (3.351) is approximately 0, which which is below our significance level of 0.05. There is significant evidence to reject the null hypothesis in favor of our alternative hypothesis that the mean change in GPA is higher for Mentored students than Control Group students.