measured by the program's effectiveness in enhancing participants 

 income and employment). 



The current procedural framework for conducting the CETA 

 Follow-Up Evaluations measures several variables at 2 points in 

 time. However, it would be extremely difficult to be conclusive about 

 the CETA programs' effectiveness on the basis of the measures. The 

 design now being used most resembles a One-Group Pretest-Post- 

 test Design. This design takes the form:, d X O2, where X is the 

 treatment (in this case, CETA program involvement) and Oi is a 

 measure of group O at time 1, and O2 is a measure of group O at time 

 2. This design is subject to severe threats to internal validity. 

 Internal validity provides answers to the question, "has the 

 experimental stimulus made some significant difference in the 

 ongoing experiment?" 



The One-Group Pretest Post-test Design is vulnerable to five 

 major threats to internal validity: (1) history, (2) maturation, (3) 

 testing, (4) instrument decay, and (5) statistical regression. History 

 refers to the unique events which occur between ti and t2 other 

 than the stimulus. Maturation refers to the biological or psycho- 

 logical processes which systematically vary over time independent 

 of the stimulus or other external event. Testing refers to the effects 

 attributable to the pretest itself. Whenever the testing process 

 becomes a stimulus for change rather than a barometer of some 

 trait, a reactive effect occurs. Instrument decay or instrumentation 

 refers to the autonomous changes in the measuring instrument 

 which might account for the changes on Oi - O2. These processes 

 include learning (recall) and fatigue (when recording responses) 

 among respondents. Statistical regression occurs whenever exper- 

 imental groups are selected on the basis of uniqueness or extremity. 

 Extreme scores, in part, are seen to reflect random errors in 

 sampling techniques and thus promote random instability of 

 measurement. Regression effects are often inevitable accompan- 

 iments of imperfect test-retest correlation for groups selected for 

 their extremity (Campbell and Stanley, 1963:11). 



Because of these design problems, internal validity is severely 

 limited. Without internal validity, measures of external validity 

 become less reliable. External validity is concerned with which 

 populations, settings, and variables the effect(s) under study can be 

 generalized. In scientific research we will most commonly sacrifice 

 external validity for internal validity. Unfortunately, the present 

 CETA Follow-Up Evaluation Design is weak in both areas. 



In order to help address some of these problems we would 

 suggest the use of a different and logically superior design 

 commonly referred to as the Solomon Four-Group Design. This 

 design explicitly takes into consideration both external and internal 

 validity factors. The structure of the design is as follows: 



R Oi X O2 



R O3 O4 



R X O5 



R Oe 



