Document Type
Article
Journal/Book Title/Conference
Journal of Learning Analytics
Author ORCID Identifier
Riordan Alfredo https://orcid.org/0000-0001-5440-6143
Vanessa Echeverria https://orcid.org/0000-0002-2022-9588
Linxuan Zhao https://orcid.org/0000-0001-5564-0185
LuEttaMae Lawrence https://orcid.org/0000-0001-6066-5096
Jie Xiang Fan https://orcid.org/0009-0000-8585-2760
Lixiang Yan https://orcid.org/0000-0003-3818-045X
Xinyu Li https://orcid.org/0000-0003-2681-4451
Zachari Swiecki https://orcid.org/0000-0002-7414-5507
Dragan Gašević https://orcid.org/0000-0001-9265-1908
Roberto Martinez-Maldonado https://orcid.org/0000-0002-8375-1816
Volume
11
Issue
3
Publisher
University of Technology Sydney ePRESS
Publication Date
10-23-2024
Journal Article Version
Version of Record
First Page
62
Last Page
87
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Abstract
Despite growing interest in applying human-centred design methods to create learning analytics (LA) systems, most efforts have concentrated on initial design phases, with limited exploration of how LA tools and practices can coevolve during the actual learning and teaching activities. This paper examines how a human-centred LA dashboard can be further refined and adapted by teachers while actively using it in a real-world scenario (i.e., design-in-use), beyond its intended design (i.e., design-for-use). We use instrumental genesis as a theoretical lens to analyze the temporary and permanent instrumentalization of design features and individual and collective instrumentation of the LA dashboard. The analysis of semi-structured individual interviews with five nursing teachers who used an LA dashboard to guide team reflections with 224 students (56 teams) revealed technical and pedagogical changes that occurred in both the system’s features (instrumentalization) and teaching practices (instrumentation). We found that teachers adopted the LA dashboard beyond initially intended ways by (i) providing emotional support with the analytics, (ii) reducing details in AI-powered data visualizations for easier comprehension, (iii) creating data narratives to address data limitations, and (iv) collectively developing new practices to use the LA dashboard for co-teaching. Therefore, teachers’ design-in-use of the LA dashboard highlights the ongoing need for design improvements to address challenges posed by dynamic data and complex algorithms underlying AI and analytics interfaces.
Recommended Citation
Alfredo, R. D., Echeverria, V., Zhao, L., Lawrence, L., Fan, J. X., Yan, L., Li, X., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024). Designing a Human-centred Learning Analytics Dashboard In-use. Journal of Learning Analytics, 11(3), 62-81. https://doi.org/10.18608/jla.2024.8487
Included in
Educational Assessment, Evaluation, and Research Commons, Instructional Media Design Commons, Library and Information Science Commons