|
Sign In to gain access to subscriptions and/or personal tools.
|
Percent Agreement, Pearson's Correlation, and Kappa as Measures of Inter-examiner Reliability
R.J. Hunt
Preventive and Community Dentistry, College of Dentistry, University of Iowa, Iowa City, Iowa 52242
Percent agreement and Pearson's correlation coefficient are frequently used to represent inter-examiner reliability, but these measures can be misleading. The use of percent agreement to measure inter-examiner agreement should be discouraged, because it does not take into account the agreement due solely to chance. Caution must be used in the interpretation of Pearson's correlation, because it is unaffected by the presence of any systematic biases. Analyses of data from a reliability study show that even though percent agreement and kappa were consistently high among three examiners, the reliability measured by Pearson's correlation was inconsistent. This study shows that correlation and kappa can be used together to uncover non-random examiner error.
REFERENCES
- Badersten, A.; Nilveus, R.; and Egelberg, J. (1984): Reproducibility of Probing Attachment Level Measurements, J Clin Periodontol 11:475-485.[CrossRef][Medline]
[Order article via Infotrieve]
- Bartko, J.J. (1976): On Various Intraclass Correlation Reliability Coefficients, Psychol Bull 83:762-765.[CrossRef]
- Bell, R.M. and Klein, S.P. (1984): Management and Evaluation of the Effects of Misclassification in a Controlled Clinical Trial, J Dent Res 63 (Spec Iss):731-734.[Medline]
[Order article via Infotrieve]
- Burt, B.A. (1983): Methods for Assessing the Distribution of Oral Diseases. In: Dentistry, Dental Practice, and the Community, 3rd ed., D.F. Striffler, W.W. Young, and B.A. Burt, Eds., Philadelphia: W.B. Saunders Co., pp. 107-108.
- Chilton, N.W. (1982): Reliability Studies. In: Design and Analysis in Dental and Oral Research, 2nd ed., New York: Praeger Publishers, pp. 415-421.
- Clemmer, B.A. and Barbano, J.P. (1974): Reproducibility of Periodontal Scores in Clinical Trials, J Periodont Res 9 (Suppl 14):118-128.[CrossRef]
- Everitt, B.S. (1977): The Analysis of Contingency Tables. London: Chapman and Hall Ltd., pp. 5-6.
- Feldman, R.S.; Douglass, C.W.; Loftus, E.R.; Kapur, K.K.; and Chauncey, H.H. (1982): Inter-examiner Agreement in the Measurement of Periodontal Disease, J Periodont Res 17:80-89.[CrossRef][Medline]
[Order article via Infotrieve]
- Fleiss, J.L. (1981): Statistical Methods for Rates and Proportions, 2nd ed., New York: John Wiley and Sons, pp. 216, 218.
- Fleiss, J.L. and Chilton, N.W. (1983): The Measurement of Interexaminer Agreement on Periodontal Disease, J Periodont Res 18:601-606.[CrossRef][Medline]
[Order article via Infotrieve]
- Fleiss, J.L. and Shrout, P.E. (1977): The Effects of Measurement Errors on Some Multivariate Procedures, Am J Public Health 67:1188-1191.[Abstract/Free Full Text]
- Fleiss, J.L.; Slakter, M.J.; Fischman, S.L.; Park, M.H.; and Chilton, N.W. (1979): Inter-examiner Reliability in Caries Trials, J Dent Res 58:604-609.[Abstract/Free Full Text]
- Fleiss, J.L.; Chilton, N.W.; and Park, M.H. (1980): Inter-and Intra-examiner Variability in Scoring Supragingival Plaque: II. Statistical Analysis, Pharm Therapeut Dent 5:5-9.
- Horowitz, H.S. (1972): Inter- and Intra-examiner Variability. In: Proceedings of the Conference on the Clinical Testing of Cariostatic Agents, Chicago: American Dental Association, pp. 96-97.
- Landis, J.R. and Koch, G.G. (1977): The Measurement of Observer Agreement for Categorical Data, Biometrics 33:159-174.[CrossRef][Medline]
[Order article via Infotrieve]
- Lemasney, J.; O'Mullane, D.; and Coleman, M. (1984): Effect of Fluoridation on Dental Health in 5- and 11-year-old Irish Schoolchildren, Community Dent Oral Epidemiol 12:218-222.[CrossRef][Medline]
[Order article via Infotrieve]
- Leverett, D.H.; Sveen, O.B.; and Jensen, O.E. (1985): Weekly Rinsing with a Fluoride Mouthrinse in an Unfluoridated Community: Results after Seven Years, J Public Health Dent 45:95-100.[CrossRef][Medline]
[Order article via Infotrieve]
- Mann, J.; Goultschin, J.; and Call, R.L. (1985): Assessment of Inter-examiner Agreement in Scoring Periodontal Disease, J Periodont Res 20:86-90.[CrossRef][Medline]
[Order article via Infotrieve]
- Pliskin, J.S.; Shwartz, M.; Grondahl, H.G.; and Boffa, J. (1984): Reliability of Coding Depth of Approximal Carious Lesions from Non-independent Interpretations of Serial Bitewing Radiographs, Community Dent Oral Epidemiol 12:366-370.[CrossRef][Medline]
[Order article via Infotrieve]
- Ran, F. and Anaise, J.Z. (1985): Prevalence of Dental Decay in Jewish and Arab Schoolchildren in the Jerusalem Area, Community Dent Oral Epidemiol 13:97-99.[CrossRef][Medline]
[Order article via Infotrieve]
- Vandervelden, V. (1980): Influence of Probing Force on the Reproducibility of Bleeding Tendency Measurements, J Clin Periodontol 7:421-427.[CrossRef][Medline]
[Order article via Infotrieve]
- Vandervelden, V. and Devries, J.H. (1980): The Influence of Probing Force on the Reproducibility of Pocket Depth Measurements, J Clin Periodontol 7:414-420.[CrossRef][Medline]
[Order article via Infotrieve]
- Vigild, M. (1985): Prevalence of Malocclusion in Mentally Retarded Young Adults, Community Dent Oral Epidemiol 13:183-184.[CrossRef][Medline]
[Order article via Infotrieve]
- World Health Organization (1977): Oral Health Surveys, Basic Methods, 2nd ed. Geneva: WHO, pp. 18-20.
Journal of Dental Research, Vol. 65, No. 2,
128-130 (1986)
DOI: 10.1177/00220345860650020701
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
M. D. Bould, N. A. Crabtree, and V. N. Naik
Assessment of procedural skills in anaesthesia
Br. J. Anaesth.,
October 1, 2009;
103(4):
472 - 483.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
L. Engel, C. Henderson, J. Fergenbaum, and A. Colantonio
Medical Record Review Conduction Model for Improving Interrater Reliability of Abstracting Medical-Related Information
Eval Health Prof,
September 1, 2009;
32(3):
281 - 298.
[Abstract]
[PDF]
|
|
|
|
|
|
|
V S S Pierro, I P R de Souza, R R Luiz, R Barcelos, and R S Moraes
Reliability of two methods for measurement of alveolar bone level in children
Dentomaxillofac. Radiol.,
January 1, 2008;
37(1):
34 - 39.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J.W. Stamm
The Classic Caries Clinical Trial: Constraints and Opportunities
Journal of Dental Research,
July 1, 2004;
83(suppl_1):
C6 - C14.
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. J. WARREN, H. J. COWEN, C. M. WATKINS, and JED. S. HAND
DENTAL CARIES PREVALENCE AND DENTAL CARE UTILIZATION AMONG THE VERY OLD
J Am Dent Assoc,
November 1, 2000;
131(11):
1571 - 1579.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
K. Enskar, M. Carlsson, M. Golsater, E. Hamrin, and A. Kreuger
Life Situation and Problems as Reported by Children With Cancer and Their Parents
Journal of Pediatric Oncology Nursing,
January 1, 1997;
14(1):
18 - 26.
[Abstract]
[PDF]
|
|
|
|
|
|
|
R.J.A.M. De Kanter, G.J. Truin, R.C.W. Burgersdijk, M.A. Van 't Hop, P.G.F.C.M. Battistuzzi, H. Kalsbeek, and A.F. Kayser
Prevalence in the Dutch Adult Population and a Meta-analysis of Signs and Symptoms of Temporomandibular Disorder
Journal of Dental Research,
November 1, 1993;
72(11):
1509 - 1518.
[Abstract]
[PDF]
|
|
|
|
|
|
|
D.W. Banting
Diagnosis and Prediction of Root Caries
Advances in Dental Research,
August 1, 1993;
7(2):
80 - 86.
[Abstract]
[PDF]
|
|
|
|
|
|
|
S.M. Szpunar and B.A. Burt
Dental Caries, Fluorosis, and Fluoride Exposure in Michigan Schoolchildren
Journal of Dental Research,
May 1, 1988;
67(5):
802 - 806.
[Abstract]
[PDF]
|
|
|
|
|
|
|
R.L. Boyd, P.J. Leggott, and P.B. Robertson
Effects on Gingivitis of Two Different 0.4% SnF2 Gels
Journal of Dental Research,
February 1, 1988;
67(2):
503 - 507.
[Abstract]
[PDF]
|
|
|
|
|
|
|
M.J.M. Schaeken, T.J. Creugers, and J.S. Van Der Hoeven
Relationship Between Dental Plaque Indices and Bacteria in Dental Plaque and Those in Saliva
Journal of Dental Research,
September 1, 1987;
66(9):
1499 - 1502.
[Abstract]
[PDF]
|
|
|
|
|
