Designing a Human-Centred Learning Analytics Dashboard In-Use

Authors

Riordan Alfredo, Monash University, AustraliaFollow
Vanessa Echeverria, Monash University, AustraliaFollow
Linxuan Zhao, Monash University, AustraliaFollow
LuEttaMae Lawrence, Utah State UniversityFollow
Jie Xiang Fan, Monash University, AustraliaFollow
Lixiang Yan, Monash University, AustraliaFollow
Xinyu Li, Monash University, AustraliaFollow
Zachari Swiecki, Monash University, AustraliaFollow
Dragan Gašević, Monash University, AustraliaFollow
Roberto Martinez-Maldonado, Monash University, AustraliaFollow

Document Type

Article

Journal/Book Title/Conference

Journal of Learning Analytics

Author ORCID Identifier

Riordan Alfredo https://orcid.org/0000-0001-5440-6143

Vanessa Echeverria https://orcid.org/0000-0002-2022-9588

Linxuan Zhao https://orcid.org/0000-0001-5564-0185

LuEttaMae Lawrence https://orcid.org/0000-0001-6066-5096

Jie Xiang Fan https://orcid.org/0009-0000-8585-2760

Lixiang Yan https://orcid.org/0000-0003-3818-045X

Xinyu Li https://orcid.org/0000-0003-2681-4451

Zachari Swiecki https://orcid.org/0000-0002-7414-5507

Dragan Gašević https://orcid.org/0000-0001-9265-1908

Roberto Martinez-Maldonado https://orcid.org/0000-0002-8375-1816

Volume

11

Issue

3

Publisher

University of Technology Sydney ePRESS

Publication Date

10-23-2024

Journal Article Version

Version of Record

First Page

62

Last Page

87

Creative Commons License

This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.

Abstract

Despite growing interest in applying human-centred design methods to create learning analytics (LA) systems, most efforts have concentrated on initial design phases, with limited exploration of how LA tools and practices can coevolve during the actual learning and teaching activities. This paper examines how a human-centred LA dashboard can be further refined and adapted by teachers while actively using it in a real-world scenario (i.e., design-in-use), beyond its intended design (i.e., design-for-use). We use instrumental genesis as a theoretical lens to analyze the temporary and permanent instrumentalization of design features and individual and collective instrumentation of the LA dashboard. The analysis of semi-structured individual interviews with five nursing teachers who used an LA dashboard to guide team reflections with 224 students (56 teams) revealed technical and pedagogical changes that occurred in both the system’s features (instrumentalization) and teaching practices (instrumentation). We found that teachers adopted the LA dashboard beyond initially intended ways by (i) providing emotional support with the analytics, (ii) reducing details in AI-powered data visualizations for easier comprehension, (iii) creating data narratives to address data limitations, and (iv) collectively developing new practices to use the LA dashboard for co-teaching. Therefore, teachers’ design-in-use of the LA dashboard highlights the ongoing need for design improvements to address challenges posed by dynamic data and complex algorithms underlying AI and analytics interfaces.

Recommended Citation

Alfredo, R. D., Echeverria, V., Zhao, L., Lawrence, L., Fan, J. X., Yan, L., Li, X., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024). Designing a Human-centred Learning Analytics Dashboard In-use. Journal of Learning Analytics, 11(3), 62-81. https://doi.org/10.18608/jla.2024.8487