Hypothesis Testing:
Continuous Variables (2 Sample)

1. Introduction
2. Brief Review & Discussion of Logic
3. Independent Groups
   1. Formula
   2. Formal Example - [Minitab] [Spreadsheet]
   1. Research Question
   2. Hypotheses
   3. Assumptions
   4. Decision Rules
   5. Computation
   6. Decision
4. Dependent Groups
1. Discussion
2. Formula
3. Formal Example - [Minitab]
1. Research Question
2. Hypotheses
3. Assumptions
4. Decision Rules
5. Computation
6. Decision

Homework

The use of designs that involve two samples far exceeds that of those previously discussed for two reasons:

1. It is rare that m or s are known. When using two samples, neither of these parameters are required.
2. Since two groups (or measurements) are included, one will serve as a concurrent control. In other words, the two groups (or measurements) occur closely together in time and space. Thus, the treatment and testing circumstances (which introduce lots of potential extraneous variables) can be better controlled. For example, in terms of our IQ/"Bad Kids" example, perhaps the IQ of the population was taken 2 years previously and the IQ in that area was increasing at the rate of 2-4 points per year (for whatever reason; you can be creative here).

Recall our first example of the experimental method at the very beginning of the semester involving the effects of marijuana on memory. The ability to analyze such an experiment has been one of the major goals of this course. In this experiment there were two groups and we need to be able to compare the means and see if the difference is worth paying attention to (i.e., did marijuana have an effect on the memory performance)?

Let's take a step back and review what we have covered thus far about inferential statistics. Actually it goes back a little further than that to where we learned about standard scores and the normal distribution. The key point was that area under the curve implies probability. To determine these probabilities, we computed the standard or z scores. That is:

	for the population
	for samples
	the the general case

We also saw that the sampling distribution of the mean was a normal distribution with:

and

respectively.

So we were able to use Z scores to determine the probability that a particular sample mean was drawn from a given population. That is:

Case I (1 sample, m & s known)

Then we went on to a more realistic situation in which the population standard deviation was not known. We estimated it from the sample standard deviation. This complicated things a bit in that the shape of the sampling distribution, while still normal, differed in its kurtosis as a function of the sample size (or more accurately, the df). This family of distributions was called Student's t and the formula became:

df=N-1

Case II (1 sample, s unknown)

However, as was noted earlier, rarely do we know any of the population parameters, and it is desirable to have a concurrent control. So we need another sampling distribution to help us compute the relevant probabilities.

The sampling distribution involves two means, so it is called the sampling distribution of the difference between means. Note that if the two means are the same (when there is no effect of the IV), the difference between them will be zero. So the value that we are interested here (in terms of the general formula for Z given above) is the difference between the means, that is:

The mean and standard deviation of the sampling distribution of the difference between means are given by:

and

respectively.

The latter is called the standard error of the difference between means. Since the sample standard deviation is again used to estimate the population value, the sampling distribution of the difference between means will also be distributed as t (the family of normal distributions that differ in kurtosis as a function of the df). So the formula becomes:

However, the null hypothesis says that:

H_O: m₁=m₂

That is, the two means come from the same population; there is no difference between them (i.e., m₁-m₂=0). Thus, the formula reduces to:

All we need to do now is determine the formula for the standard error. However, this formula differs depending whether we are dealing with independent or dependent groups. Once we understand this distinction, we can move on to Case III (i.e., independent groups) and Case IV (i.e., dependent groups) of hypothesis testing with continuous variables.

With the independent groups design, the subjects in each of the two groups are different and unrelated in any way. The most common type of dependent groups design is also called a within subjects or repeated measures design, because the same subjects (thus actually only one group) are tested twice.

1. Formula

The defining formula (when the sample sizes are equal) for the standard error of the difference between means is:

And thus the formula for the t value is:

The computational formulas (which will also handle unequal sample sizes) are given by

And:

Since two variances are used in estimating the standard error of the difference between means, the degrees of freedom will equal the sum of the degrees of freedom for each of the variance estimates, that is:

2. Formal Example - [Minitab] [Spreadsheet]

Suppose you are a researcher interested in the factors influencing paper grading by professors. You have a hunch (and/or previous research) might lead you to predict that papers that are typed are rated higher than papers that are handwritten. Research to date though, has only been correlational and thus little can be said in terms of a cause and effect relationship.

So you have 10 freshman students currently taking English as well as an introductory psychology course each write one paper. They should each provide two copies of their paper (one typed and one handwritten). Next, we enlist the aid of 20 English instructors. We randomly assign 10 instructors to each of two groups. Each instructor in one group (the control group) will grade each of the 10 papers that are hand written, while the second group (the experimental group) will grade the same papers that are typed.

1. Research Question
   Does typing a paper influence the grade it receives?
   
   
2. Hypotheses

	In Symbols	In Words
HO	m1=m2	Typing has no effect on the grade a paper receives (as compared to a handwritten paper).
HA	m1¹m2	Typing influences the grade for better or worse.



3. Assumptions
   
1. Our subjects were chosen randomly from the population.
2. The groups are independent.
3. There is homogeneity of variance That is, the amount of variability in the DV is about equal in each of the groups. When the samples sizes are reasonably large and the number of subjects in each group is about equal, we do not have to worry about this too much because the t test is robust. This means that it is strong and can tolerate some violations of its assumptions.
4. Sampling distribution of the difference between means is normal in shape. In other words, the DV should be normally distributed in the population.
5. The null hypothesis.
   
   
4. Decision Rules
   Using alpha of .05 with a two-tailed test and df=N₁+N₂-2=10+9-2=17, we determine from the t table that the critical value is 2.110. Thus:

   If t_obs £ -2.110 or t_obs ³ 2.110, then reject H_O.
   If t_obs > -2.110 and t_obs < 2.110, then do not reject H_O.

5. Computation
   Since we are not interested in the differences between the scores of the 10 papers graded by an instructor, we simply calculate the mean grade given by each instructor. Note that one of the instructors in the Written Group had to be excluded because their dog ate the papers they were supposed to grade. Thus, we then have 19 means . To describe the data, we present the means and standard deviations for each of the two groups, that is:

   	Written (1)	Typed (2)
   Data	81	84
   	81	89
   	79	89
   	80	81
   	84	87
   	87	82
   	75	87
   	83	85
   	88	89
   		83
   	82.0	85.6
   s	4.03	3.03
   N	9	10

Now the inferential question is whether this difference between means is worth paying attention to. Thus, we will use a between groups t test to answer this question.



Substituting the appropriate values gives:



6. Decision
   Since -2.222 (t_obs) < -2.110 (t_crit) we reject H_O and assert the alternative. In other words, we conclude that typing a paper improves the grade it receives. Notice that we have actually gone beyond the alternative hypothesis by specifying that the effect has a direction (typing is good).

1. Discussion
   As noted earlier, the most common type of Dependent Groups Design is also called a Within Subjects or Repeated Measures Design, because the same subjects (thus, actually only one group) are tested twice. There is another situation, though, in which this analysis is sometimes used. It is called the Matched Groups Design. In this case, there are two groups, but they are matched on some variable that is highly and positively correlated with the DV. The procedures involved in matching will be presented more clearly below in the formal example.



2. Formula
   In this case, the standard error of the difference between means is given by:
   

Notice that the formula requires the computation of the correlation between the two sets of scores. It is here that we see the potential advantage to this design. That is, the error term (the standard error of the difference between means) is decreased in direct proportion to the magnitude of this correlation, which results in a potentially more powerful or sensitive test. The disadvantage though is the loss of degrees of freedom. The N here refers to the number of pairs of scores (for an individual or matched pair of individuals). Thus, the degrees of freedom is half what we would have if we had used a between groups approach (i.e., N-1 is 1/2 of N₁+N₂-2). The trick is to make sure the correlation is large enough to offset the loss of df.

The formula above would be very cumbersome to use. Fortunately, there is another technique available for obtaining the t value called the Direct Difference Method. If the difference between the X and Y scores is designated as D (i.e., D=X-Y), then we may then we may restate the null and alternative hypotheses as:

	In Symbols
HO	mD=0
HA	mD¹0

The formula which looked like:

And with:

becomes:

which, given the null (m₁-m₂=0, thus m_D=0) reduces to:

Below is the derivation of the computational formula:

where the df=N-1 and N refers to the number of pairs of scores.

3. Formal Example - [Minitab]

Suppose you are interested in reactions times to different colored lights (especially green and red). We could use either:

• Repeated measures design - test each subject for a number of trials, such as GGRRGRRG, etc. Then compute the average speed to each color light for each subject.
• Matched groups design - test all subjects' reaction times to white light for a given number of trials. Using this data, create two matched groups, that is, take the two quickest subjects and randomly assign one to each of the groups. Then take the next two quickest subjects and randomly assign one of them to each of the groups, etc. Ex:
  

  Ranked Data	Red	Green
  1, 2, 3, 4, 5, 6, 7, 8, . . .	2	1
  	3	4
  	6	5
  	8	7
  	. . .

  
  Note that the number of subjects must be devisable by the number of groups.

1. Research Question
   Does reaction to red and green lights differ?
   
   
2. Hypotheses
   

	In Symbols	In Words
HO	m1=m2	There is no difference in reaction times between red and green lights.
HA	m1¹m2	There is a difference in reaction times between red and green lights.



3. Assumptions
   
1. Our subjects were chosen randomly from the population.
2. The scores of the two conditions are correlated (i.e., the groups are dependent).
3. The sampling distribution of the difference between means is normal in shape. In other words, the DV should be normally distributed in the population.
4. The null hypothesis.
   
   
4. Decision Rules
   We will test 10 (or 20 if matched) subjects. Using alpha of .05 with a two-tailed test and df=N-1=9, we determine from the t table that the critical value is 2.262.
   
   Thus:

   If t_obs £ -2.262 or t_obs ³ 2.262, then reject H_O.
   If t_obs > -2.262 and t_obs < 2.262, then do not reject H_O.

5. Computation
   First we describe the data by computing the means for each condition/group. While we are at it, we might as well compute the difference scores and their squares (since we will need them for the analysis).

Subject (or pair)	X (red)	Y (green)	D	D2
1	18	22	-4	16
2	16	20	-4	16
3	23	29	-6	36
4	30	35	-5	25
5	32	27	5	25
6	30	29	1	1
7	31	33	-2	4
8	25	29	-4	16
9	27	31	-4	16
10	21	24	-3	9


Then, for the inferential test, we will use a within groups t test (the direct difference method) and thus we have the formula:



And substituting the appropriate values gives:



6. Decision
   Since -2.512 (t_obs) < -2.262 (t_crit) we reject H_O and assert the alternative. In other words, we conclude that reaction time is quicker to red as compared to green light.