Date of Award:

5-2001

Document Type:

Dissertation

Degree Name:

Doctor of Philosophy (PhD)

Department:

Psychology

Committee Chair(s)

Karl R. White

Committee

Karl R. White

Committee

Jim Dorward

Committee

Byron Burnham

Committee

George Julnes

Committee

Kentaro Hayashi

Abstract

This study collected evidence to address the assumptions underlying the use of the Utah Core Assessment to Pre-Algebra (UCAP) to (a) measure student achievement in pre-algebra, and (b) assist teachers in making adjustments to instruction. An evaluative argument was defined to guide the collection of evidence. Each of the assumptions in the evaluative argument was addressed using data from a suburban northern Utah school district. To collect the evidence, test content was examined including item match to course objectives, reliability, and subtest intercorrelations. Analyses of correlations of the UCAP with convergent and discriminant measures were completed using student test data (N = 1,461), including an examination of both the pattern of correlations and tests of statistical significance. Pre-algebra teachers (N = 12) were interviewed to ascertain the degree to which UCAP results were used to make necessary adjustments to instruction.

It was found that the UCAP was technically sound, but measured only 65% of course objectives. Correlation coefficients were analyzed using pattern comparisons and tests of statistical significance. It was found that the pattern of correlation coefficients and the distinction of convergent and discriminant measures supported the UCAP as a measure of mathematics. Teacher interview data revealed that teachers did not make substantive adjustments to the instruction of pre-algebra based on test scores.

Based on these results it was concluded that the underlying assumptions concerning the use of the UCAP were not fully supported. The lack of complete coverage of the pre-algebra course objectives calls into question the ability of the UCAP scores to be used as measures of student achievement, in spite of the technical quality of the test. There was support for the assumption that the UCAP measures mathematics. There was little evidence that teachers use the UCAP score reports to make meaningful and appropriate adjustments to instruction. More evidence is needed to understand the factors that may have led to this lack of use.

The evaluative argument framework defined in this study provides guidance for future research to collect evidence of the validity of decisions based on UCAP scores.

Checksum

fa8215371724e0597016b38c1ca5e29d

Download

Share

COinS

Copyright for this work is retained by the student. If you have any questions regarding the inclusion of this work in the Digital Commons, please email us at .