Test–retest reliability and reliable change estimates for four mobile cognitive tests administered virtually in community-dwelling adults

Abstract

Objective: Remote mobile cognitive testing (MCT) is an expanding area of research, but psychometric data supporting these measures are limited. We provide preliminary data on test–retest reliability and reliable change estimates in four MCTs from SWAY Medical, Inc.

Methods: Fifty-five adults from the U.S. Midwest completed the MCTs remotely on their personal mobile devices once per week for 3 consecutive weeks, while being supervised with a video-based virtual connection. The cognitive assessment measured simple reaction time (“Reaction Time”), go/no-go response inhibition (“Impulse Control”), timed visual processing (“Inspection Time”), and working memory (“Working Memory”). For each cognitive test except Working Memory, we analyzed both millisecond (ms) responses and an overall SWAY composite score.

Results: The mean age of the sample was 26.69years (SD=9.89; range=18–58). Of the 55 adults, 38 (69.1%) were women and 49 (89.1%) used an iPhone. Friedman’s ANOVAs examining differences across testing sessions were nonsignificant (ps>0.31). Intraclass correlations for Weeks 1–3 were: Reaction Time (ms): 0.83, Reaction Time (SWAY): 0.83, Impulse Control (ms): 0.68, Impulse Control (SWAY): 0.80, Inspection Time (ms): 0.75, Inspection Time (SWAY): 0.75, and Working Memory (SWAY): 0.88. Intraclass correlations for Weeks 1–2 were: Reaction Time (ms): 0.75, Reaction Time (SWAY): 0.74, Impulse Control (ms): 0.60, Impulse Control (SWAY): 0.76, Inspection Time (ms): 0.79, Inspection Time (SWAY): 0.79, and Working Memory (SWAY): 0.83. Natural distributions of difference scores were calculated and reliable change estimates are presented for 70, 80, and 90% CIs.

Conclusion: Test–retest reliability was adequate or better for the MCTs in this virtual remote testing study. Reliable change estimates allow for the determination of whether a particular level of improvement or decline in performance is within the range of probable measurement error. Additional reliability and validity data are needed in other age groups.