M. J. Troy, P. E. Miller, N. Price, V. Talwalkar, F. Zaina, S. Donzelli, S. Negrini, M. T. Hresko

March 2019, Volume 28, Issue 3, pp 559 - 566 Original Article Read Full Article 10.1007/s00586-018-5821-8

First Online: 16 November 2018


This study aims to propose and validate a new unified “Risser+” grade that combines the North American (NA) and European (EU) variants of the classic Risser score. The “Risser+ ” grade can effectively combine the North American and European Risser Classifications for skeletal maturity with adequate intra-rater/inter-rater reliability and agreement.


Agreement and reliability were evaluated for 6 raters (3-NA, 3-EU) who assessed 120 pelvic radiographs from the BrAIST trial, all female, average age 13.4 (range 10.1–16.5 years). Blinded raters reviewed x-rays at two time-points. Intra- and inter-rater agreement (RA) were established with Krippendorff’s alpha (k-alpha), while intra- and inter-rater reliability (RR) were established with intraclass correlation coefficients (ICC). Acceptable agreement and reliability were set a priori at 0.80.


Inter-RA for the second reading met study requirements (k-alpha = 0.86 [0.81–0.90]) compared to the first reading (0.72 [0.63–0.79]) while combined readings was close to target agreement (0.79 [0.74–0.84]). Removal of 20 readings demonstrating outlier tendencies increased agreement for the first, second, and combined reads (k-alpha = 0.85, 0.89, 0.87, respectively). Intra-RA was sufficient for 4 out of 6 raters (k-alpha > 0.80) and one rater from EU and NA presented subpar intra-RA (k-alpha = 0.64 and 0.74, respectively). Inter-RR met study requirements overall reads (ICC = 0.96 [0.95–0.97]) including the first (0.94 [0.92–0.95]) and second (0.97 [0.97–0.98]) reads, independently.


The Risser+ system showed excellent reliability across multiple reads and raters and demonstrated 79% agreement overall reads and ratings. Agreement increased to over 85% when raters could distinguish Risser 0 + from Risser 5.

Graphical abstract

These slides can be retrieved from electronic supplementary material.[Figure not available: see fulltext.]

Read Full Article