Objective This work examined the relationship of the constructs measured by the trust scales developed by Chancey et al. (2017) and Jian et al. (2000) using a multilevel confirmatory factor analysis (CFA).

Background Modern theories of automation trust have been proposed based on data collected using trust scales. Chancey et al. (2017) adapted Madsen and Gregor’s (2000) trust scale to align with Lee and See’s (2004) human–automation trust framework. In contrast, Jian et al. (2000) developed a scale empirically with trust and distrust as factors. However, it remains unclear whether these two scales measure the same construct.

Method We analyzed data collected from previous experiments to investigate the relationship between the two trust scales using a multilevel CFA.

Results Data provided evidence that Jian et al. (2000) and Chancey et al. (2017) automation trust scales are only weakly related. Trust and distrust are found to be distinct factors in Jian et al.’s (2000) scale, whereas performance, process, and purpose are distinct factors in Chancey et al.’s (2017) trust scale.

Conclusion The analysis suggested that the two scales purporting to measure human–automation trust are only weakly related.

Application Trust researchers and automation designers may consider using Chancey et al. (2017) and Jian et al. (2000) scales to capture different characteristics of human–automation trust.