INTRODUCTION

Orthodontic programs have relied on numerous factors, including high-stakes assessments such as the Graduate Record Examination (GRE), as important screening tools in evaluating applicants. While the specialty of orthodontics graduates the most students of any dental specialty,¹ competing for a position has not been an easy task historically.² One out of five orthodontic residents were reported to have a grade point average (GPA) in the top 2% of their dental class and a staggering 72% in the top 10% overall.³ In addition, applicants must overcome the large quantity of total applicants vying for a limited number of program positions. The American Dental Education Association reported that the orthodontic specialty experienced a 16% increase in application numbers between 2012 and 2017, from 9709 to 11,279.⁴ Conversely, the number of residency positions has remained relatively consistent, with 393 positions across the 68 programs currently. Thus, there are approximately 29 applications submitted for every available orthodontic position, highlighting the extremely competitive nature of the application process.

There is ample literature discussing what screening criteria influence the admissions process for both medical^5–8 and dental^9–11 residency programs. One commonality between these sources is the importance of undergraduate board scores on receiving interview invitations and admittance into a program.

Studies in medicine have shown wide-ranging correlations between undergraduate board scores and performance on postgraduate board examinations. Most studies show positive correlations,^12–16 some a negative correlation,^5,17 and others no correlation at all.^18,19 When studying medical student performance and residency performance, Harfmann and Zirwas²⁰ noted there were inconsistent results for most medical residency performance measures and indicated that there was a need for a more systematic and consistent methodology to determine predictors of success in residency.

In dentistry, the most historically influential standardized test has been the National Board Dental Examination (NBDE). Oral and maxillofacial surgery and pediatric program directors have considered NBDE scores to be the most important criterion when evaluating applicants.^9,10 In 2012, however, the NBDE scores transitioned to a pass/fail format, eliminating this assessment as a tool for programs.²⁰ The Joint Commission on National Dental Examinations noted that the conversion of the NBDE to a pass/fail grading schematic could potentially have an unfavorable impact on candidates from dental institutions that adopted a similar pass/fail assessment strategy.²¹ Seventy-one percent of postgraduate program directors agreed that it is now more difficult to select interview candidates, and 76% would prefer some form of a national, numerically scored exam.²² With the elimination of NBDE scores, many orthodontic programs turned to the GRE as an objective, numerical measure to evaluate their applicants.²³

The GRE is a widely used standardized exam specifically generated to assess “basic developed abilities relevant to performance in graduate studies.”²⁴ Several meta-analyses have been completed to determine the validity of the GRE.^25–29 Kuncel et al.²⁵ observed moderate correlations between GRE scores and imperative criterion measures. In a subsequent study, Kuncel et al.³⁰ showed the GRE to exhibit similar and valuable predictive validity for doctoral and master's level programs. However, the predictive usefulness of the GRE in dental specialties is less clear. Lindauer et al.³¹ found that orthodontic applicants viewed programs more negatively if they emphasized the importance of the GRE as a criterion in evaluating applicants, whereas program directors perceived the opposite. The only study to specifically relate GRE scores with performance in an orthodontic graduate program was a master's thesis in 2014. Menjivar³² analyzed GRE scores as a preresidency factor and found an inverse relationship between GRE scores and student performance.

With orthodontic programs allocating extensive time and resources to evaluate qualified applicants,³² is the GRE truly an appropriate tool to predict orthodontic residency performance? One potential standardized assessment metric within orthodontics is the American Board of Orthodontics (ABO) written exam, which functions as the main exit examination required by many programs. The ABO written exam is a comprehensive exam that evaluates a resident's understanding of basic sciences and clinical concepts central to the practice of orthodontics.³³ Menjivar's thesis³² incorporated postgraduate program GPA and subjective rankings by the faculty but did not investigate objective ABO written board scores. Therefore, the aim of this study was to evaluate the association between resident selection criteria (GRE performance and other selection parameters) and student residency performance outcomes (ABO written exam, graduate orthodontic GPA).

MATERIALS AND METHODS

This retrospective study was evaluated and deemed exempt by the Indiana University Institutional Review Board (1708590483). The sample population included the student academic records of seventy (n = 70) graduates (2008–2017) from the Indiana University School of Dentistry (IUSD), Department of Orthodontics and Oral Facial Genetics. The investigators obtained all pertinent information from the WebAdMIT student record database. The following demographic and scholastic data were extracted from the student database:

Statistical Analysis

Scatterplots and correlation coefficients were used to analyze associations of GRE scores, dental school GPA, and age separately with the ABO written exam quintiles and graduate orthodontic GPAs. Linearity of the relationships was also examined. One-way ANOVA was used to evaluate associations of gender, ethnicity, race, country of origin, and year of dental school graduation separately with ABO written exam quintiles and graduate orthodontic GPAs. Multiple regression analyses were then used to analyze the factors simultaneously as predictors of ABO written exam quintiles and graduate orthodontic GPAs. A 5% statistical significance level was established for all statistical tests.

To help facilitate interpretation of the study's findings, the correlation scheme devised by Evans³⁵ was used to assess obtained associations (Table 1).

Table 1 Correlation Interpretation Scheme

View Fullscreen

RESULTS

Seventy residents (44 men, 26 women) graduated from the IUSD Department of Orthodontics and Oral Facial Genetics from 2008–2017 and were included in the study (Table 2). Most graduates were between 25 and 29 years of age (55.7%) at the time of preresidency application submission. The two largest observed racial groups were whites and Asians, representing 70% and 20%, respectively. Because of the low number of African American, American Indian, Hawaiian/Pacific Islander, and other graduate students, these groups were combined and denoted as “other” for statistical analysis. The reported ethnicity and citizenship also tended to be homogenous, with 89% of the graduates identifying themselves as non-Hispanic/Latino and 78.1% as US citizens.

Table 2 Descriptive Statistics: Categorical Variables

View Fullscreen

Table 3 displays the numerical preresidency selection criteria and the orthodontic residency performance variables. Fifty-one of the students attended dental schools that used a class ranking system, and of those, 38 (75%) were found to be in the top 25% of their class. The mean overall, didactic, clinical, and research orthodontic GPAs of the graduates were 3.83, 3.82, 3.72, and 3.96, respectively. The ABO written examination is scored on a quintile scale, with “1” being the highest achieving quintile and “5” being the lowest. The quintile scores appeared to improve from module 1 to module 4, with means ranging from 3.77 to 3.17, respectively (Table 3).

Table 3 Descriptive Statistics: Continuous Variables

View Fullscreen

The first study objective was to determine if correlations existed between any of the numerical selection criteria and orthodontic residency performance metrics (Table 4). Overall, the GRE failed to demonstrate any significant correlations with numerical residency performance criteria. The one exception was a significant, but weak, negative correlation between the GRE quantitative reasoning section and ABO module 2 (r = −.31, P = .0101). In addition, dental school GPA displayed a moderate positive correlation with overall orthodontic GPA (r = .43, P = .0004) and the didactic orthodontic GPA (r = .48, P < .0001). A significant moderate correlation was also found between the year of dental school graduation with overall orthodontic GPA (r = .41, P = .0005) and didactic orthodontic GPA (r = .41, P = .0005). Meanwhile, there were weak correlations between year of dental graduation and ABO module 1 (r = −.34, P = .0045) and clinical orthodontic GPA (r = .26, P = .0292). There were other significant but weak correlations observed, including dental school GPA and all four ABO modules (module 1: r = −.29, P = .0215; module 2: r = −.28, P = .0243; module 3: r = −.30, P = .0157; module 4: r = −.27, P = .0286), as well as clinical orthodontic GPA (r = .27, P = .0324).

Table 4 Means and Standard Errors for Categorical Variables^a

View Fullscreen

When assessing student demographic characteristics with orthodontic residency performance, no significant difference was noted for gender, and the “ethnicity” variable was removed from the analysis because of a collinearity between age and ethnicity. A significant difference was noted for race, country of origin, and age (Tables 5 and 6). Regarding race, whites had a higher performance on the ABO module 3 than nonwhites (3.54 vs 4.15; P = .0372) as well as a higher didactic orthodontic GPA (3.78 vs 3.68; P = .0110). Compared with international students, US citizens demonstrated higher achievement on ABO module 1 (3.52 vs 4.28; P = .0134), module 2 (3.69 vs 4.12; P = .0394), and module 4 (2.59 vs 3.61; P = .0083). Students with US citizenship also displayed significantly higher overall orthodontic GPA (3.81 vs 3.76; P = .0066) and didactic orthodontic GPA (3.82 vs 3.71; P = .0007). Younger students (25–29 years old) performed significantly better on ABO module 1 and module 4, overall orthodontic GPA, and clinical orthodontic GPA than their older counterparts (Tables 5 and 6).

Table 5 Spearman Correlations of GRE Scores, Dental School GPA, and Age With ABO Exam Quintiles and Graduate Orthodontic GPAs^a

View Fullscreen

Table 6 One-Way Analysis of Variance Evaluating the Association of Demographics With ABO Exam Quintiles and Graduate Orthodontic GPAs^a

View Fullscreen

The multiple linear regression analysis (Table 7) further highlighted the effect of age on performance in the orthodontic residency. Other predictors of high performance on the ABO written examination and overall orthodontic GPA were GRE quantitative score and dental school GPA. The younger the student, the lower the quintile on ABO module 4 (P = .0022) and the higher the overall orthodontic GPA (P = .0362). The GRE quantitative score denoted a significant association with ABO module 2 (P = .0217). Similarly, dental school GPA was associated with didactic orthodontic GPA (P = .256).

Table 7 Multiple Linear Regression^a

View Fullscreen

DISCUSSION

The GRE examination is a widely used numerical assessment employed in higher education and has exhibited valuable predictive validity for doctoral and master's level programs.³⁰ Menjivar's³² work has been the only study to relate GRE scores with orthodontic residency performance. This research found that higher quantitative GRE scores correlated with lower average orthodontic GPA.³² This contradicts the findings of the current study, which found no association between the GRE and orthodontic GPA. In this study, dental school GPA had a moderate positive correlation with both overall orthodontic GPA and didactic orthodontic GPA, while Menjivar found no correlation between these parameters. One limitation of Menjivar's thesis was the small sample size, which consisted of 30 graduates. The current study consisted of 70 graduates, and this sample size difference could explain the discrepancy in observed correlations. Another important finding in the current study was that dental school GPA had a weak correlation with clinical orthodontic GPA and all four sections of the ABO written examination. From the findings, it was concluded that, unlike dental school GPA, the GRE appears to have minimal value when evaluating qualified applicants.

Along with numerical metrics, understanding how demographics can relate to orthodontic residency performance should be considered. This study found no differences in orthodontic residency performance based on student gender, which contradicted previous studies that showed males had higher performances on similar dental standardized examinations.^36,37 The study demonstrated significant performance differences based on age, race, and country of origin. Most of the IUSD orthodontic graduates were between 25 and 29 years old and were white, US citizens. For each of these specific demographic variables, all displayed favorable associations with orthodontic residency performance.

Multiple reasons can be suggested to explain the discrepancies in performance seen among the demographic groups. First, overcoming any written and/or verbal language barriers can be a challenge for non-US citizens. Language challenges can intensify the stress related to academic demands for ethnic minorities.³⁸ Second, there may be significant differences in the dental curriculum, training, and philosophies encountered at US dental schools compared with non-US dental schools. Lastly, the age gap between graduates creates other potential hurdles for nontraditional, older-aged students to overcome.³⁹

It should be acknowledged that this study had limitations. The sample population had a relatively high degree of homogeneity for several variables, including age, race, ethnicity, and country of origin. In addition, the data used in the study came from a single US postdoctoral orthodontic program. Lastly, the study did not consider the impact of other nonscholastic preresidency information, such as research experience, history of community service, and/or documented leadership. While important in the overall assessment of a residency applicant, these parameters are much less objective in nature and were excluded to enhance the ability to assess the hypotheses.

CONCLUSIONS

• The GRE failed to provide any predictive utility for orthodontic residency performance.

• Overall, the GRE failed to demonstrate a predictive relationship with performance on the ABO written examination.

• Dental school GPA and/or class rank appear to be the most predictive numerical parameters for orthodontic residency performance.

• Younger, recent dental graduates from US dental schools appear to exhibit a higher level of performance in an orthodontic residency.