Health Care and Legislation, Policies

Chapter 7
Hypothesis Testing Procedures

Learning Objectives (1 of 3)
Define null and research hypothesis, test statistic, level of significance, and decision rule
Distinguish between Type I and Type II errors and discuss the implications of each
Explain the difference between one- and two-sided tests of hypothesis
Estimate and interpret p-values

Learning Objectives (2 of 3)
Explain the relationship between confidence interval estimates and p-values in drawing inferences
Perform analysis of variance by hand
Appropriately interpret the results of analysis of variance tests
Distinguish between one- and two-factor analysis of variance tests

Learning Objectives (3 of 3)
Perform chi-square tests by hand
Appropriately interpret the results of chi-square tests
Identify the appropriate hypothesis testing procedures based on type of outcome variable and number of samples

Hypothesis Testing
Research hypothesis is generated about unknown population parameter.
Sample data are analyzed and determined to support or refute the research hypothesis.

Hypothesis Testing Procedures

Step 1
Null hypothesis (H0):
No difference, no change

Research hypothesis (H1):
What investigator believes to be true

Hypothesis Testing Procedures

Step 2
Collect sample data and determine whether sample data support research hypothesis or not.
For example, in test for m, evaluate .

Hypothesis Testing Procedures

Step 3
Set up decision rule to decide when to believe null versus research hypothesis.
Depends on level of significance, a = P(Reject H0|H0 is true)

Hypothesis Testing Procedures

Steps 4 and 5
Summarize sample information in test statistic (e.g., Z value).
Draw conclusion by comparing test statistic to decision rule.
Provide final assessment as to whether H1 is likely true given the observed data.

p-values
p-values represent the exact significance of the data.
Estimate p-values when rejecting H0 to summarize significance of the data (can approximate with statistical tables, can get exact value with statistical computing package).
p-value is the smallest a where we still reject H0.

Hypothesis Testing Procedures
Set up null and research hypotheses, select a.
Select test statistic.
Set up decision rule.
Compute test statistic.
Draw conclusion and summarize significance.

Errors in Hypothesis Tests

Hypothesis Testing for m
Continuous outcome
One sample

H0: m = m0
H1: m > m0, m < m0, m ≠ m0 Test statistic: n ≥ 30 (Find critical value in Table 1C, n < 30 Table 2, df = n – 1) The National Center for Health (NCHS) reports the mean total cholesterol for adults is 203. Is the mean total cholesterol in Framingham Heart Study participants significantly different? In 3310 participants the mean is 200.3 with a standard deviation of 36.8. Example 7.2. Hypothesis Testing for m (1 of 4) 1. H0: m = 203 H1: m ≠ 203 a = 0.05 2. Test statistic: 3. Decision rule: Reject H0 if z ≥ 1.96 or if z ≤ –1.96 Example 7.2. Hypothesis Testing for m (2 of 4) 4. Compute test statistic: Conclusion. Reject H0 because –4.22 < –1.96. We have statistically significant evidence at a = 0.05 to show that the mean total cholesterol is different in Framingham Heart Study participants. Example 7.2. Hypothesis Testing for m (3 of 4) Example 7.2. Hypothesis Testing for m (4 of 4) Significance of the findings: Z = –4.22 Table 1C. Critical Values for Two-Sided Tests a Z 0.20 1.282 0.10 1.645 0.05 1.960 0.010 2.576 0.001 3.291 0.0001 3.819 p < 0.0001 New Scenario Outcome is dichotomous (p = population proportion). Result of surgery (success, failure) Cancer remission (yes/no) One study sample Data On each participant, measure outcome (yes/no) n, x = number of positive responses, Hypothesis Testing for p Dichotomous outcome One sample H0: p = p0 H1: p > p0, p < p0, p ≠ p0 Test statistic: (Find critical value in Table 1C) The NCHS reports that the prevalence of cigarette smoking among adults in 2002 is 21.1%. Is the prevalence of smoking lower among participants in the Framingham Heart Study? In 3536 participants, 482 reported smoking. Example 7.4. Hypothesis Testing for p (1 of 3) 1. H0: p = 0.211 H1: p < 0.211 a = 0.05 Test statistic: 3. Decision rule: Reject H0 if z ≤ –1.645 Example 7.4. Hypothesis Testing for p (2 of 3) Example 7.4. Hypothesis Testing for p (3 of 3) 4. Compute test statistic: 5. Conclusion. Reject H0 because –10.93 < –1.645. We have statistically significant evidence at a = 0.05 to show that the prevalence of smoking is lower among the Framingham Heart Study participants. (p < 0.0001) Hypothesis Testing for Categorical and Ordinal Outcomes* Categorical or ordinal outcome One sample H0: p1 = p10, p2 = p20,…,pk = pk0 H1: H0 is false Test statistic: (Find critical value in Table 3, df = k – 1) * c2 goodness-of-fit test Chi-Square Tests 2 tests are based on the agreement between expected (under H0) and observed (sample) frequencies. Test statistic: Chi-Square Distribution If H0 is true c2 will be close to 0; if H0 is false, c2 will be large. Reject H0 if c2 > Critical value from Table 3

A university survey reveals that 60% of students get no regular exercise, 25% exercise sporadically and 15% exercise regularly. The university institutes a health promotion campaign and re-evaluates exercise 1 year later.

None Sporadic Regular
Number of students 255 125 90
Example 7.6.

c2 Goodness-of-Fit Test (1 of 4)

1. H0: p1 = 0.60, p2 = 0.25, p3 = 0.15
H1: H0 is false a = 0.05

2. Test statistic:

3. Decision rule: df = k – 1 = 3 – 1 = 2
Reject H0 if c2 ≥ 5.99
Example 7.6.

c2 Goodness-of-Fit Test (2 of 4)

4. Compute test statistic:

None Sporadic Regular Total
No. students (O) 255 125 90 470
Expected (E) 282 117.5 70.5 470

(O – E)2/E 2.59 0.48 5.39

c2 = 8.46
Example 7.6.

c2 Goodness-of-Fit Test (3 of 4)

Example 7.6.

c2 Goodness-of-Fit Test (4 of 4)
5. Conclusion. Reject H0 because 8.46 > 5.99. We have statistically significant evidence at a = 0.05 to show that the distribution of exercise is not 60%, 25%, 15%.

Using Table 3, the p-value is p < 0.005. New Scenario Outcome is continuous. SBP, weight, cholesterol Two independent study samples Data On each participant, identify group and measure outcome. Two Independent Samples (1 of 2) RCT: Set of Subjects Who Meet Study Eligibility Criteria Randomize Treatment 1 Treatment 2 Mean Treatment 1 Mean Treatment 2 Two Independent Samples (2 of 2) Cohort Study: Set of Subjects Who Meet Study Inclusion Criteria Group 1 Group 2 Mean Group 1 Mean Group 2 Hypothesis Testing for (m1 - m2) (1 of 2) Continuous outcome Two independent samples H0: m1 = m2 (m1 - m2 = 0) H1: m1 > m2, m1< m2, m1 ≠ m2 Hypothesis Testing for (m1 - m2) (2 of 2) Continuous outcome Two independent samples H0: m1 = m2 H1: m1 > m2, m1 < m2, m1 ≠ m2 Test statistic: n1 ≥ 30 and (Find critical value n2 ≥ 30 in Table 1C, n1 < 30 or Table 2, df = n1 + n2 – 2) n2 < 30 Pooled Estimate of Common Standard Deviation, Sp Previous formulas assume equal variances (s12 = s22). If 0.5 ≤ s12/s22 ≤ 2, assumption is reasonable. Example 7.9. Hypothesis Testing for (m1 - m2) (1 of 3) A clinical trial is run to assess the effectiveness of a new drug in lowering cholesterol. Patients are randomized to receive the new drug or placebo and total cholesterol is measured after 6 weeks on the assigned treatment. Is there evidence of a statistically significant reduction in cholesterol for patients on the new drug? Example 7.9. Hypothesis Testing for (m1 - m2) (2 of 3) Sample Size Mean Std Dev New drug 15 195.9 28.7 Placebo 15 227.4 30.3 1. H0: m1 = m2 H1: m1 < m2 a = 0.05 2. Test statistic: 3. Decision rule: df = n1 + n2 – 2 = 28 Reject H0 if t ≤ –1.701 Example 7.9. Hypothesis Testing for (m1 - m2) (3 of 3) Assess Equality of Variances Ratio of sample variances: 28.72/30.32 = 0.90 Example 7.9. Hypothesis Testing for (m1 - m2) 4. Compute test statistic: 5. Conclusion. Reject H0 because –2.92 < –1.701. We have statistically significant evidence at a = 0.05 to show that the mean cholesterol level is lower in patients on treatment as compared to placebo. (p < 0.005) New Scenario Outcome is continuous. SBP, weight, cholesterol Two matched study samples Data On each participant, measure outcome under each experimental condition. Compute differences (D = X1 – X2). Two Dependent/Matched Samples Subject ID Measure 1 Measure 2 1 55 70 2 42 60 . . Measures taken serially in time or under different experimental conditions. Crossover Trial Treatment Treatment Eligible R Participants Placebo Placebo Each participant is measured on treatment and placebo. Hypothesis Testing for md Continuous outcome Two matched/paired sample H0: md = 0 H1: md > 0, md < 0, md ≠ 0 Test statistic: n ≥ 30 (Find critical value in Table 1C, n < 30 Table 2, df = n – 1) Example 7.10. Hypothesis Testing for md (1 of 3) Is there a statistically significant difference in mean systolic blood pressures (SBPs) measured at exams 6 and 7 (approximately 4 years apart) in the Framingham Offspring Study? Among n = 15 randomly selected participants, the mean difference was –5.3 units and the standard deviation was 12.8 units. Differences were computed by subtracting the exam 6 value from the exam 7 value. 1. H0: md = 0 H1: md ≠ 0 a = 0.05 2. Test statistic: 3. Decision rule: df = n – 1 = 14 Reject H0 if t ≥ 2.145 or if z ≤ –2.145 Example 7.10. Hypothesis Testing for md (2 of 3) 4. Compute test statistic: 5. Conclusion. Do not reject H0 because –2.145 < –1.60 < 2.145. We do not have statistically significant evidence at a = 0.05 to show that there is a difference in systolic blood pressures over time. Example 7.10. Hypothesis Testing for md (3 of 3) New Scenario Outcome is dichotomous Result of surgery (success, failure) Cancer remission (yes/no) Two independent study samples Data On each participant, identify group and measure outcome (yes/no) Hypothesis Testing for (p1 – p2) Dichotomous outcome Two independent samples H0: p1 = p2 H1: p1 >p2, p1< p2, p1 ≠ p2 Test statistic: (Find critical value in Table 1C) Example 7.12. Hypothesis Testing for (p1 – p2) (1 of 4) Is the prevalence of CVD different in smokers as compared to nonsmokers in the Framingham Offspring Study? Free of CVD History of CVD Total Nonsmoker 2757 298 3055 Current smoker 663 81 744 Total 3420 379 3799 1. H0: p1 = p2 H1: p1 ≠ p2 a = 0.05 2. Test statistic: 3. Decision rule: Reject H0 if Z ≤ –1.96 or if Z ≥ 1.96 Example 7.12. Hypothesis Testing for (p1 – p2) (2 of 4) 4. Compute test statistic: Example 7.12. Hypothesis Testing for (p1 – p2) (3 of 4) 5. Conclusion. Do not reject H0 because –1.96 < 0.927 < 1.96. We do not have statistically significant evidence at a = 0.05 to show that there is a difference in prevalence of CVD between smokers and nonsmokers. Example 7.12. Hypothesis Testing for (p1 – p2) (4 of 4) Hypothesis Testing for More than Two Means* Continuous outcome k independent Samples, k > 2

H0: m1 = m2 = m3 … = mk
H1: Means are not all equal
Test statistic:

(Find critical value in Table 4)
*Analysis of variance

Test Statistic: F Statistic
Comparison of two estimates of variability in data
Between treatment variation, is based on the assumption that H0 is true (i.e., population means are equal).
Within treatment, residual or error variation, is independent of H0 (i.e., we do not assume that the population means are equal and we treat each sample separately).

F Statistic (1 of 2)

Difference between each group mean and overall mean
Difference between each observation and its group mean (within group variation—error)

F = MSB/MSE

MS = Mean Square

What values of F indicate H0 is likely true?

F Statistic (2 of 2)

Decision Rule
Reject H0 if F ≥ critical value of F with df1 = k – 1 and df2 = N – k from Table 4

k = Number of comparison groups
N = Total sample size

ANOVA Table
Source of Sums of Mean
Variation Squares df Squares F

Between
treatments k – 1 SSB/k – 1 MSB/MSE

Error N – k SSE/N – k

Total N – 1

Is there a significant difference in mean weight loss among four different diet programs?

(Data are pounds lost over 8 weeks)

Example 7.14.

ANOVA (1 of 12)

Low-Cal Low-Fat Low-Carb Control

8 2 3 2

9 4 5 2

6 3 4 -1

7 5 2 0

3 1 3 3

1. H0: m1 = m2 = m3 = m4

H1: Means are not all equal a = 0.05

2. Test statistic:

Example 7.14.

ANOVA (2 of 12)

3. Decision rule:
df1 = k – 1 = 4 – 1 = 3
df2 = N – k = 20 – 4 =16

Reject H0 if F ≥ 3.24
Example 7.14.

ANOVA (3 of 12)

Summary on Weight Loss by Treatment

Low-Cal Low-Fat Low-Carb Control
N 5 5 5 5
Mean 6.6 3.0 3.4 1.2

Overall Mean = 3.6
Example 7.14.

ANOVA (4 of 12)

Example 7.14.

ANOVA (5 of 12)
= 5(6.6 – 3.6)2 + 5(3.0 – 3.6)2 + 5(3.4 – 3.6)2 + 5(1.2 – 3.6)2
= 75.8

Example 7.14.

ANOVA (6 of 12)

Example 7.14.

ANOVA (7 of 12)

Example 7.14.

ANOVA (8 of 12)

Example 7.14.

ANOVA (9 of 12)

= 21.4 + 10.0 + 5.4 + 10.6 = 47.4
Example 7.14.

ANOVA (10 of 12)

Source of Sums of Mean
Variation Squares df Squares F

Between 75.8 3 25.3 8.43
Treatments
Error 47.4 16 3.0

Total 123.2 19

Example 7.14.

ANOVA (11 of 12)

4. Compute test statistic:
F = 8.43
5. Conclusion. Reject H0 because 8.43 > 3.24. We have statistically significant evidence at a = 0.05 to show that there is a difference in mean weight loss among four different diet programs.
Example 7.14.

ANOVA (12 of 12)

Two-Factor ANOVA
Compare means of a continuous outcome across two grouping variables or factors
Overall test—is there a difference in cell means?
Factor A—marginal means
Factor B—marginal means
Interaction—difference in means across levels of Factor B for each level of Factor A?

Interaction
Cell Means Factor B
1 2 3
Factor A 1 45 58 70
2 65 55 38

No Interaction
Cell Means Factor B
1 2 3
Factor A 1 45 58 70
2 38 55 65

Example 7.16.

Two-Factor ANOVA (1 of 3)
Clinical trial to compare time to pain relief of three competing drugs for joint pain. Investigators hypothesize that there may be a differential effect in men versus women.
Design: N = 30 participants (15 men and 15 women) are assigned to three treatments (A, B, C)

Example 7.16.

Two-Factor ANOVA (2 of 3)
Mean times to pain relief by treatment and sex

Is there a difference in mean times to pain relief? Are differences due to treatment? Sex? Or both?

Source Sums of Mean
of Variation Squares df Square F p-value

Model 967.0 5 193.4 20.7 0.0001
Treatment 651.5 2 325.7 34.8 0.0001
Sex 313.6 1 313.6 33.5 0.0001
Treatment*Sex 1.9 2 0.9 0.1 0.9054

Error 224.4 24 9.4

Total 1191.4 29
Example 7.16.

Two-Factor ANOVA (3 of 3)

Hypothesis Testing for Categorical

or Ordinal Outcomes*
Categorical or ordinal outcome
Two or more samples

H0: The distribution of the outcome is independent of the groups
H1: H0 is false

Test statistic:

(Find critical value in Table 3: df = (r – 1)(c – 1))
* c2 test of independence

Chi-Square Test of Independence
Outcome is categorical or ordinal (2+ levels) and there are two or more independent comparison groups (e.g., treatments).

H0: Treatment and outcome are independent distributions of outcome are the same across treatments)

Is there a relationship between students’ living arrangement and exercise status?

Exercise Status
None Sporadic Regular Total
Dormitory 32 30 28 90
On-campus apt. 74 64 42 180
Off-campus apt. 110 25 15 150
At home 39 6 5 50
Total 255 125 90 470
Example 7.17.

c2 Test of Independence (1 of 6)

1. H0: Living arrangement and exercise status are
independent
H1: H0 is false a = 0.05

2. Test statistic:

3. Decision rule: df = (r – 1)(c – 1) = 3(2) = 6
Reject H0 if c2 ≥ 12.59
Example 7.17.

c2 Test of Independence (2 of 6)

4. Compute test statistic:

O = Observed frequency
E = Expected frequency

E = (row total)*(column total)/N
Example 7.17.

c2 Test of Independence (3 of 6)

4. Compute test statistic:
Table entries are Observed (Expected) frequencies
Exercise Status
None Sporadic Regular Total
Dormitory 32 30 28 90
(90*255/470 = 48.8) (23.9) (17.2)
On-campus apt. 74 64 42 180
(97.7) (47.9) (34.5)
Off-campus apt. 110 25 15 150
(81.4) (39.9) (28.7)
At home 39 6 5 50
(27.1) (13.3) (9.6)
Total 255 125 90 470
Example 7.17.

c2 Test of Independence (4 of 6)

4. Compute test statistic:

Example 7.17.

c2 Test of Independence (5 of 6)

Example 7.17.

c2 Test of Independence (6 of 6)
5. Conclusion. Reject H0 because 60.5 > 12.59. We have statistically significant evidence at a = 0.05 to show that living arrangement and exercise status are not independent. (P < 0.005) X n s/ μ - X Z 0 = n s/ μ - X t 0 = 22 . 4 3310 / 8 . 36 203 3 . 200 n s/ μ - X Z 0 - = - = = n x p ˆ = 5 )] p n(1 , min[np 0 0 ³ - Z = p̂ – p0 p0 (1 – p0 ) n Z = ˆ p – p 0 p 0 (1 – p 0 ) n Z = p̂ – p0 p0 (1 – p0 ) n Z = ˆ p – p 0 p 0 (1 – p 0 ) n Z = p̂ – p0 p0 (1 – p0 ) n = 0.136 − 0.211 0.211(1− 0.211) 3536 = −10.93 Z = ˆ p – p 0 p 0 (1 – p 0 ) n = 0.136-0.211 0.211(1-0.211) 3536 =-10.93 å = E E) - (O χ 2 2 E ) E - (O Σ = χ 2 2 n1,X1,s1 2(or s1),n2,X2,s2 2(or s2) n 1 ,X 1 ,s 1 2 (ors 1 ),n 2 ,X 2 ,s 2 2 (ors 2 ) 2 1 2 1 n 1 n 1 Sp X - X Z + = 2 1 2 1 n 1 n 1 Sp X - X t + = 2 n n 1)s (n 1)s (n Sp 2 1 2 2 2 2 1 1 - + - + - = 5 . 29 89 . 870 2 15 15 1)30.3 (15 1)28.7 (15 Sp 2 2 = = - + - + - = 92 . 2 15 1 15 1 5 . 29 4 . 227 9 . 195 n 1 n 1 Sp X - X t 2 1 2 1 - = + - = + = d d s , X n, n s μ - X Z d d d = n s μ - X t d d d = 60 . 1 15 / 8 . 12 0 3 . 5 n s μ - X t d d d - = - - = = 2 2 1 1 p ˆ , n , p ˆ , n 5 )] p ˆ (1 n , p ˆ n ), p ˆ (1 n , p ˆ min[n 2 2 2 2 1 1 1 1 ³ - - ÷ ÷ ø ö ç ç è æ + = 2 1 2 1 n 1 n 1 ) p ˆ - (1 p ˆ p ˆ - p ˆ Z ÷ ÷ ø ö ç ç è æ + = 2 1 2 1 n 1 n 1 ) p ˆ - (1 p ˆ p ˆ - p ˆ Z 0.0975 3055 298 p ˆ 0.1089, 744 81 p ˆ 2 1 = = = = 0.0988 3055 744 298 81 p ˆ = + + = 927 . 0 3055 1 744 1 0.0988) - 0.0988(1 0.0975 - 0.1089 Z = ÷ ø ö ç è æ + = k) /(N ) X ΣΣ(X 1) /(k ) X X ( Σn F 2 j 2 j j - - - - = ) X - X ( n Σ = SSB j 2 j ) X - X ( Σ Σ = SSE j 2 ) X - X ( Σ Σ = SST 2 35 40 45 50 55 60 65 70 75 123 A1 A2 35 40 45 50 55 60 65 70 75 123 A1 A2 5 . 60 χ 9.6 9.6) (5 ... 17.2 17.2) (28 23.9 23.9) (30 48.8 48.8) (32 χ 2 2 2 2 2 2 = - + + - + - + - =