The Five Steps of Hypothesis Testing

  1. List the five steps of hypothesis testing and explain the procedure and logic of each.
    a. Restate the question as a research hypothesis and a null hypothesis about the population: Population 1 is the population that represents the general population, or at least the general population concerned with the area that the researcher is studying. Population 2 represents the population that is effected by whatever causal variable the researcher is testing for. The null hypothesis is not rejected if population 1, the general population, scores do not differ significantly from the population 2, test population, scores. However, the alternative hypothesis is found to be statistically significant if population 2 scores are found to differ significantly, usually .05 or .01, from population 1 scores.
    b. Determine the characteristics of the comparison distribution: The comparison distribution is the population that would be true if the null hypothesis were true. Specifically, the characteristics of the comparison distribution are the mean and standard deviation, or with a distribution of means the mean and standard error. The mean is the instrument of central tendency which best describes a distribution that is normally distributed and the standard deviation describes the average deviations that the scores deviate from the mean.
    c. Determine the cutoff sample score on the comparison distribution at which the null hypothesis should be rejected: We must consult the normal curve areas table in order to find out where the p value percentage falls as a z-score. If the test is .05, then we would find out where 5% of the tail falls as a z-score and if .01 where 1% falls as a z-score. If the study is one-tailed, then we use the aforementioned z-scores; but if the study is two-tailed, then have to split the percentages in order to accommodate both tails. So 1% would be .5% for each tale and 5% would be 2.5% for each tale. Once we know where these percentages lie as a z-score we are ready to compare the raw score of the sample to the comparison distribution and find out where it falls in relation to the z-score cut-off of the p value.
    d. Determine your sample’s score on the comparison distribution: Once we know the score, or scores, of the population 2, we can compared them to population 1, or the comparison distribution (distribution of means). All we have to do is convert the score, or mean of scores, from a raw value to a z-score on the comparison distribution. Then we can compare the z-score of the sample to the cut-off z-score of the p value on the comparison distribution.
    e. Decide whether to reject the null hypothesis: If the z-score of the sample raw score reaches or exceeds the selected p value, then the test is said to be statistically significant and the null hypothesis can be rejected. If the score does not reach the z-score cut-off, then the test is said to be statistically insignificant and the null hypothesis cannot be reject. This does not mean that the null hypothesis is true, but only that the alternative hypothesis is not statistically significant.
  2. Based on the information given for each of the following studies, decide whether to reject the null hypothesis. For each, give the Z-score cutoff (or cutoffs) on the comparison distribution at which the null hypothesis should be rejected, the Z score on the comparison distribution for the sample score, and your conclusion. Assume that all populations are normally distributed.
    a. Study A
    i. Reject Null Hypothesis: Yes
    ii. Z-Score Cutoff(s): +1.64
    iii. Z-Score on Comparison Distribution: 2
    iv. Conclusion: Statistically significant at a one-tailed high p-value of .05.
    b. Study B
    i. Reject Null Hypothesis: Yes
    ii. Z-Score Cutoff(s): -1.96, +1.96
    iii. Z-Score on Comparison Distribution: 2
    iv. Conclusion: Statistically significant at a two-tailed split p-value of -.025 and +.025.
    c. Study C
    i. Reject Null Hypothesis: No
    ii. Z-Score Cutoff(s): +2.33
    iii. Z-Score on Comparison Distribution: 2
    iv. Conclusion: Not statistically significant at a one-tailed p-value of .01.
    d. Study D
    i. Reject Null Hypothesis: No
    ii. Z-Score Cutoff(s): -2.57, +2.57
    iii. Z-Score on Comparison Distribution: 2
    iv. Conclusion: Not statistically significant at a two-tailed split p-value of -.005 and +.005.
  3. A researcher predicts that listening to music while solving math problems will make a particular brain area more active. To test this, a research participant has her brain scanned while listening to music and solving math problems, and the brain area of interest has a percent signal change of 58. From many previous studies with this same math-problems procedure (but not listening to music), it is known that the signal change in this brain area is normally distributed with a mean of 35 and a standard deviation of 10.
    a. Using the .01 level, what should the researcher conclude? The researcher should conclude that the score of 58% is not statistically significant at a one-tailed p-value of .01. The Z-score cutoff for the comparison distribution is 2.33 and the sample Z-score is at 2.3, just shy of significant. This is a statistical example of approaching significance. I too am not comfortable with titles such as “approaching significance” because statistical significance is an all or nothing ordeal; however, if I had the standard deviation of the sample I could calculate effect size. I think that regardless of the SD, within reason, the effect size will be medium to large for this particular sample. Also, we cannot forget that we are testing at the .01 level. If we were to test at the .05 level, then the test would have definitely been statistically significant. The bottom line is, not statistically significant, but more studies are required to isolate causal factors of insignificance.
    b. Solve this problem explicitly using all five steps of hypothesis testing, and illustrate your answer with a sketch showing the comparison distribution, the cutoff (or cutoffs), and the score of the sample on this distribution.
    i. Population 1: People who listen to music while studying math problems
    ii. Population 2: People who do not listen to music while studying math problems.
    iii. Null Hypothesis: There is no difference between population 1 and population 2, that listening to music increase brain functioning significantly while studying math.
    iv. Alternative Hypothesis: There is a significant difference (p>.01) in brain functioning between studying math problems with music and without music.
    v. Comparison Distribution: Mean = 35, SD = 10, normal distribution
    vi. Cutoff Sample: At one-tailed .01 level the Z-score cut-off would be +2.33 or a raw score of 58.30%.
    vii. Sample Score on Comparison Distribution: The score of 58% would have a Z-score of +2.30 on the comparison distribution.
    viii. Conclusion: Do not reject the null hypothesis because the score of 58% is not statistically significant.

c. Then explain your answer to someone who has never had a course in statistics (but who is familiar with mean, standard deviation, and Z scores). According to several studies the average levels of brain activity signal change while completing math problems is 35% with a standard deviation of 10. This test uses a single sample to try and find out if listening to music while completing math problems. For the test to be statistically significant, a big enough change to say that the music does increase brain activity while doing math problems, the sample will have to score in the outer 1% of the several studies mentioned before. This is a z-score of +2.33. The sample scored 58% brain activity increase while listening to music and doing math problems; however, this is only a z-score of 2.3. Therefore, the study is not statistically significant and we cannot reject the null hypothesis.

References

Aron, A., Aron, E., & Coups, E. (2006). Statistics for psychology (4th ed.). Upper Saddle River, NJ: Pearson/Allyn Bacon.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Powered by WordPress.com.

Up ↑

%d bloggers like this: