Date of Award:

5-2021

Document Type:

Thesis

Degree Name:

Master of Science (MS)

Department:

Communicative Disorders and Deaf Education

Department name when degree awarded

Communication Disorders and Deaf Education

Committee Chair(s)

Sandra Gillam

Committee

Sandra Gillam

Committee

Ron Gillam

Committee

Sarah Schwartz

Abstract

Language sample analysis (LSA) is an important practice for providing a sensitive and accurate measure of a child’s language abilities. Several research-validated progress-monitoring tools are currently available that are designed to measure language quality through narrative samples in school-age children. While these tools provide clinically important information about a child’s language abilities, they can be time-consuming to code, and challenging to code reliably. In recent years there has been a surge in the use of automated essay scoring (AES) systems for the scoring of high-stakes written assessments, but few programs has been designed to automate clinical assessments. Narrative microstructure, an important indicator of language quality, lends itself to computer automation due to its relatively objective and straightforward assessment. The purpose of the current study was to test the feasibility of developing a series of computer-based rulesets, collectively referred to as CAMS, for automatically scoring narrative microstructure in LSA. CAMS was designed to individually score six microstructure elements related to conjunctions, meta-cognitive and linguistic verbs and sentence elaboration. The accuracy and interrater reliability of CAMS with hand-scores (n = 414) was compared against goldstandard expert scores (n = 50) based on percent overlap and quadratic weighted kappa (QWK). Results indicated that QWK between CAMS and expert-scores was higher than that of hand-scorers on four of the six elements. CAMS also met the literature-based threshold for strong interrater reliability (k > 0.60) for all six elements. CAMS shows promise as a means for automating the scoring of microstructure in LSA and narrative samples.

Checksum

ca6ed4f3080464b4671c8e5eedef4b25

Download

Available for download on Friday, May 01, 2026

Share

COinS

Copyright for this work is retained by the student. If you have any questions regarding the inclusion of this work in the Digital Commons, please email us at .