Meta-Analysis of Correlations Among Usability Measures

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Standard

Meta-Analysis of Correlations Among Usability Measures. / Hornbæk, Kasper Anders Søren; Effie Lai Chong, Law.

Conference on Human Factors in Computing Systems: CHI 2007, Reach beyond. Conference Proceedings. ed. / Bo Begole; Stephen Payne; Elizabeth Churchill; Rob St. Amant; David Gilmore; Mary Beth Rosson. Association for Computing Machinery, 2007. p. 617-626.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Harvard

Hornbæk, KAS & Effie Lai Chong, L 2007, Meta-Analysis of Correlations Among Usability Measures. in B Begole, S Payne, E Churchill, R St. Amant, D Gilmore & MB Rosson (eds), Conference on Human Factors in Computing Systems: CHI 2007, Reach beyond. Conference Proceedings. Association for Computing Machinery, pp. 617-626, Conference on Human Factors in Computing Systems, San Jose, Calif., United States, 28/04/2007. https://doi.org/http://doi.acm.org/10.1145/1240624.1240722

APA

Hornbæk, K. A. S., & Effie Lai Chong, L. (2007). Meta-Analysis of Correlations Among Usability Measures. In B. Begole, S. Payne, E. Churchill, R. St. Amant, D. Gilmore, & M. B. Rosson (Eds.), Conference on Human Factors in Computing Systems: CHI 2007, Reach beyond. Conference Proceedings (pp. 617-626). Association for Computing Machinery. https://doi.org/http://doi.acm.org/10.1145/1240624.1240722

Vancouver

Hornbæk KAS, Effie Lai Chong L. Meta-Analysis of Correlations Among Usability Measures. In Begole B, Payne S, Churchill E, St. Amant R, Gilmore D, Rosson MB, editors, Conference on Human Factors in Computing Systems: CHI 2007, Reach beyond. Conference Proceedings. Association for Computing Machinery. 2007. p. 617-626 https://doi.org/http://doi.acm.org/10.1145/1240624.1240722

Author

Hornbæk, Kasper Anders Søren ; Effie Lai Chong, Law. / Meta-Analysis of Correlations Among Usability Measures. Conference on Human Factors in Computing Systems: CHI 2007, Reach beyond. Conference Proceedings. editor / Bo Begole ; Stephen Payne ; Elizabeth Churchill ; Rob St. Amant ; David Gilmore ; Mary Beth Rosson. Association for Computing Machinery, 2007. pp. 617-626

Bibtex

@inproceedings{21b154b0559511dcbee902004c4f4f50,
title = "Meta-Analysis of Correlations Among Usability Measures",
abstract = "Understanding the relation between usability measures seems crucial to deepen our conception of usability and to select the right measures for usability studies. We present a meta-analysis of correlations among usability measures calculated from the raw data of 73 studies. Correlations are generally low: effectiveness measures (e.g., errors) and efficiency measures (e.g., time) has a correlation of .247 ± .059 (Pearson's product-moment correlation with 95% confidence interval), efficiency and satisfaction (e.g., preference) one of .196 ± .064, and effectiveness and satisfaction one of .164 ± .062. Changes in task complexity do not influence these correlations, but use of more complex measures attenuates them. Standard questionnaires for measuring satisfaction appear more reliable than homegrown ones. Measures of users' perceptions of phenomena are generally not correlated with objective measures of the phenomena. Implications for how to measure usability are drawn and common models of usability are criticized.",
author = "Hornb{\ae}k, {Kasper Anders S{\o}ren} and {Effie Lai Chong}, Law",
year = "2007",
doi = "http://doi.acm.org/10.1145/1240624.1240722",
language = "English",
isbn = "9781595935939",
pages = "617--626",
editor = "Bo Begole and Stephen Payne and Elizabeth Churchill and {St. Amant}, Rob and David Gilmore and Rosson, {Mary Beth}",
booktitle = "Conference on Human Factors in Computing Systems",
publisher = "Association for Computing Machinery",
note = "null ; Conference date: 28-04-2007 Through 03-05-2007",

}

RIS

TY - GEN

T1 - Meta-Analysis of Correlations Among Usability Measures

AU - Hornbæk, Kasper Anders Søren

AU - Effie Lai Chong, Law

PY - 2007

Y1 - 2007

N2 - Understanding the relation between usability measures seems crucial to deepen our conception of usability and to select the right measures for usability studies. We present a meta-analysis of correlations among usability measures calculated from the raw data of 73 studies. Correlations are generally low: effectiveness measures (e.g., errors) and efficiency measures (e.g., time) has a correlation of .247 ± .059 (Pearson's product-moment correlation with 95% confidence interval), efficiency and satisfaction (e.g., preference) one of .196 ± .064, and effectiveness and satisfaction one of .164 ± .062. Changes in task complexity do not influence these correlations, but use of more complex measures attenuates them. Standard questionnaires for measuring satisfaction appear more reliable than homegrown ones. Measures of users' perceptions of phenomena are generally not correlated with objective measures of the phenomena. Implications for how to measure usability are drawn and common models of usability are criticized.

AB - Understanding the relation between usability measures seems crucial to deepen our conception of usability and to select the right measures for usability studies. We present a meta-analysis of correlations among usability measures calculated from the raw data of 73 studies. Correlations are generally low: effectiveness measures (e.g., errors) and efficiency measures (e.g., time) has a correlation of .247 ± .059 (Pearson's product-moment correlation with 95% confidence interval), efficiency and satisfaction (e.g., preference) one of .196 ± .064, and effectiveness and satisfaction one of .164 ± .062. Changes in task complexity do not influence these correlations, but use of more complex measures attenuates them. Standard questionnaires for measuring satisfaction appear more reliable than homegrown ones. Measures of users' perceptions of phenomena are generally not correlated with objective measures of the phenomena. Implications for how to measure usability are drawn and common models of usability are criticized.

U2 - http://doi.acm.org/10.1145/1240624.1240722

DO - http://doi.acm.org/10.1145/1240624.1240722

M3 - Article in proceedings

SN - 9781595935939

SP - 617

EP - 626

BT - Conference on Human Factors in Computing Systems

A2 - Begole, Bo

A2 - Payne, Stephen

A2 - Churchill, Elizabeth

A2 - St. Amant, Rob

A2 - Gilmore, David

A2 - Rosson, Mary Beth

PB - Association for Computing Machinery

Y2 - 28 April 2007 through 3 May 2007

ER -

ID: 934900