Evaluation of facial attractiveness for patients with malocclusion: A machine-learning technique employing Procrustes
To establish an objective method for evaluating facial attractiveness from a set of orthodontic photographs. One hundred eight malocclusion patients randomly selected from six universities in China were randomly divided into nine groups, with each group containing an equal number of patients with Class I, II, and III malocclusions. Sixty-nine expert Chinese orthodontists ranked photographs of the patients (frontal, lateral, and frontal smiling photos) before and after orthodontic treatment from “most attractive” to “least attractive” in each group. A weighted mean ranking was then calculated for each patient, based on which a three-point scale was created. Procrustes superimposition was conducted on 101 landmarks identified on the photographs. A support vector regression (SVR) function was set up according to the coordinate values of identified landmarks of each photographic set and its corresponding grading. Its predictive ability was tested for each group in turn. The average coincidence rate obtained for comparisons of the subjective ratings with the SVR evaluation was 71.8% according to 18 verification tests. Geometric morphometrics combined with SVR may be a prospective method for objective comprehensive evaluation of facial attractiveness in the near future.ABSTRACT
Objective:
Materials and Methods:
Results:
Conclusions:
INTRODUCTION
The subject of beauty has been the topic of much debate throughout history, and methods for the evaluation of beauty have been the focus of many research projects. According to recent reports, the evaluation of beauty is influenced by factors such as sex, race, and stage of development and involves integration of social knowledge and physical cues.1–3 The concept of beauty also changes with time,4,5 and people who inherit the same culture usually share similar concepts of beauty.6 Our perception of beauty is related to attractiveness, and facial attractiveness in particular is an important physical attribute. An attractive facial appearance invites positive social responses, which have a profound effect on a person's self-esteem and capacity for social adjustment. Orthodontists have the ability to change a patient's facial features and subsequently impact his or her life. Hence, there is a need for orthodontists to understand the esthetic standards for an attractive face.
Clinical evaluation methods for facial attractiveness involve measurement of linear aspects, proportions, and angles. The esthetic line,7 the profile line,8 and the Holdaway line9 are common clinical linear measurements. Although they are quite convenient and popular in clinical practice, they are not very precise, and it is difficult to use them for comparison across patients because of the nonuniformity of photographic dimensions. Proportions and angles, on the other hand, are size-independent variables and are commonly used for photographic evaluation. The golden proportion and the Fibonacci sequence1 have been used for hundreds of years, and many new angles and proportions have been proposed in the last century. Lower face proportion, nasofacial angle, mentocervical angle, and many other proportions and angles are thought be related to facial attractiveness.10 Pallett et al.11 even found a “new golden proportion,” according to which individual attractiveness is optimized when the vertical distance between the eyes and the mouth is approximately 36% of the face's length and the horizontal distance between the eyes and the mouth is approximately 46% of the face's width. However, there are too many correlated variables for an individual to determine facial attractiveness using them. Furthermore, Kiekens et al.12 showed that significantly correlated variables only explained 28.7% of the variance and that this variance was too small to be of clinical importance. Similarly, a study done by Schmid et al.,13 which was based on neoclassical canons, symmetry, and golden ratios, showed a weak predictability of 24.3%.
Orthodontists typically use image analysis methods to examine attractiveness. A standard orthodontic set of photographs includes frontal smiling, lateral, and frontal views; these are the most common records used to establish a treatment plan, compare changes after treatment, and evaluate treatment results. Xu et al.14 and Liu et al.15 used this standard trio of views to analyze the attractiveness preferences of Chinese and US orthodontists. Shafiee et al.16 studied the weight of each type of image in the evaluation of the three views and found that the smiling image played the most important role. The lateral photo is used primarily for angular and linear measurements, while the frontal photo is used mainly to evaluate proportions; smile esthetic analysis often focuses on local manifestations around the mouth and not on the whole face.17,18 However, the evaluation of facial attractiveness should and must consider these three types of images together for a comprehensive judgment. Conventional evaluation methods assess each image using different variables. Only the research of Kiekens et al.12 considered angles and proportions in frontal and lateral photos together, but the result was not satisfactory, as mentioned.
Geometric morphometrics, also known as statistical shape analysis, might be more valid for describing biological shape than angles and proportions and is widely used in other branches of biological science. In statistical terminology, the rotation, translation, and scale parameters that are not of scientific interest are known as nuisance parameters. Morphometrics can help to eliminate the interference of nuisance parameters. Furthermore, morphometrics allows the integration of distinct information present in photographs.19 Procrustes superimposition is a sophisticated morphometric technique that attempts to minimize the apparent effects of translation, rotation, and scale to reveal pure shape discrepancies. Landmarks are used in the superimposition to resize the shape, achieve the best fit, and obtain a uniform coordinate system. In investigations of facial attractiveness, researchers have also used Procrustes superimposition to analyze the effect of averageness and symmetry20; Kagian et al.21 presented a machine-learning model to predict facial attractiveness ratings, but their research was limited to the frontal view.
Another popular method is objective conclusions, in the form of rankings or scores, drawn from observers' subjective judgments. A group of observers is invited to evaluate a series of photos, and the evaluations are statistically analyzed to obtain results. In our study, we used a combination of the aforementioned methods with the aim of obtaining more accurate facial attractiveness results. We attempted to create a more effective objective evaluation system for clinical use based on size adjustment with Procrustes superimposition by using a set of orthodontic photographs. The effect of the variables on the rankings given by a group of orthodontists was studied.
The general linear regression model is the most common method used by orthodontists to create an evaluation system. However, facial attractiveness is so complex that its relationship with photographic variables may not be a simple linear one; this model therefore has low predictability. Instead, in our study, we used support vector regression (SVR), which is based on the computation of a linear regression function in a high-dimensional feature space in which the input data are mapped via a nonlinear function. The accuracy of the function was tested on each of the groups in turn to determine the reliability of the SVR model.
MATERIALS AND METHODS
Patients
The data analyzed in this study were collected from a project involving six universities in China. Ethical approval was obtained from Peking University before the start of the project. The subjects included 2383 patients who underwent orthodontic treatment in the six universities during the years 2003–2008. Written consent for participation was obtained by the universities, and in the case of children under 18 years old, consent from a parent was obtained. Eighteen patients were selected randomly from each of the universities, with an equal number of Class I, Class II, and Class III malocclusion patients included. These 108 patients (30 male and 78 female patients; age range, 10–29 years) were randomly arranged into nine groups containing 12 patients each; in each group, four patients were classified as Class I before treatment, four as Class II before treatment, and four as Class III before treatment. The procedure used for sampling and grouping is shown in Figure 1. The frontal, frontal smiling, and lateral photographs before and after orthodontic treatment for each patient were then printed in a row.



Citation: The Angle Orthodontist 84, 3; 10.2319/071513-516.1
Subjective Attractiveness Evaluation
The judges for the experiments were 69 orthodontic experts (at the level of associate professor or higher) in China, who had more than 10 years of experience in clinical experiments, from 32 universities or hospitals covering 19 provinces of China. They were asked to rank the photographic sets of each group, before and after treatment, in the order of “most attractive” to “least attractive,” so that rankings for 18 groups were obtained (nine groups before treatment and nine groups after treatment). The ranking was done on a scale of 1 to 12, with 1 assigned to the “most attractive” patient and 12 to the “least attractive” patient.
Analysis of the Rankings
A frequency chart (Figure 2) of all the rankings was created for each patient. The four adjacent bars in the central area, which accounted for the largest percentage of rankings, were used for analysis, and the remaining data were excluded. This was done to eliminate inconsistent data that did not agree with the majority. Then, the weighted mean was calculated for each case and was used in the correlation analysis and stepwise linear regression. The weighted mean values were sequenced from the minimum to the maximum. Then, patients with a ranking of 1 to 4 were placed in the “most attractive” subgroup, those with a ranking of 5 to 8 were placed in the “moderately attractive” subgroup, and those with a ranking of 9 to 12 were placed in the “least attractive” subgroup. A three-point grading scale was thus obtained.



Citation: The Angle Orthodontist 84, 3; 10.2319/071513-516.1
Attractiveness Evaluation Using Procrustes Superimposition
Custom-made computer software was used to identify 26 landmarks on the frontal photo, 52 landmarks on the frontal smiling photo, and 23 landmarks on the lateral photo (Table 1; Figures 3–5). The selected landmarks were those frequently used in the orthodontic literature, and all landmarks were easily identifiable, had good repeatability, and were only minimally affected by changes in head position.



Citation: The Angle Orthodontist 84, 3; 10.2319/071513-516.1



Citation: The Angle Orthodontist 84, 3; 10.2319/071513-516.1



Citation: The Angle Orthodontist 84, 3; 10.2319/071513-516.1

For images of each type, Procrustes superimposition was used to eliminate any error introduced by different image sizes and improper head positioning. The configuration of landmarks was first scaled to the same size. The Procrustes superimposition algorithms translate the configurations to superimpose the centroids and iteratively rotate the configurations to minimize the squared differences between the landmarks of the configurations. Landmarks that were not consistently found in all images were eliminated. For example, 26 landmarks were identified on the frontal photo in a few patients, but three of these landmarks were not found in the frontal photos in the other cases; therefore, only the 23 consistently found landmarks were used for Procrustes superimposition of the frontal photos. Through superimposition, coordinate values of all these landmarks were obtained within the same coordinate system of each image type.
For each set of photographs, 24 angles and 81 proportions were calculated according to their coordinate values using a computer-based program. All the variables were used as input data for Pearson correlation analysis with the weighted means of the rankings. Subsequently, stepwise linear regression was performed to ascertain the effect of the variables on the subjective rankings.
SVR Analysis
Rather than angles and proportions, the configuration of landmarks generated from Procrustes superimposition, together with the corresponding three-point scale grade, was used directly as the input data for the SVR model. SVR was used to analyze the input data, and a function was generated according to the landmark configuration corresponding to the different grades. The generated function was used to evaluate the facial attractiveness grading of the new photographic set. Seventeen groups of cases were used to establish the SVR model; then a test was performed on the group of cases that remained. This procedure was repeated for each group.
RESULTS
As shown in Table 2, of the 81 proportions and 24 angles that were investigated, only three proportions and one angle were significantly correlated with subjective rankings. The explained variance with stepwise regression was only 23.5%.

The SVR function was determined for all 18 sets of rankings, with data from any one of the 18 groups kept aside each time for verification of the prediction using SVR. The predictive grading using SVR for this one group (whose data were not entered into the SVR model) was compared with the grading obtained from the rankings of the 69 orthodontists; the rate of coincidence was determined, which reflected the accuracy of the SVR method. As shown in Table 3, the average rate of coincidence was 0.718 and varied from 0.58 to 0.83.

DISCUSSION
Although the concept of beauty varies to a great extent across cultures, there have been many attempts to find universal agreement based on a subjective esthetic view of populations, and several variables, such as linear measurements, proportions, and angles, have been proposed and are in use. Consideration of the average values of the linear measures, angles, and proportions in a given human group or esthetic group is the typical method. Average values are considered ideal, based on the assumption that average faces are attractive and that average facial proportions could provide a basis for quantitative assessment of facial esthetics. However, recent studies have reported that, whereas average faces are attractive, they are not among the most attractive ones22; facial attractiveness goes beyond the merely average and is far more complex. Thus, some controversy remains about the popular attractiveness variables currently in use. In an attempt to address this, in the current study we combined these conventional methods with morphometric methods to produce a prediction model for facial attractiveness.
Objective conclusions drawn from observers' subjective judgments are a recent focus; however, researchers have found differences in judgment based on the observers' race, social role, and occupation, among other variables.6,23,24 Although we used this method, we tried to reduce these influences as much as possible by choosing Chinese specialists from a similar professional background. The purpose of the selection was to reflect the overall esthetic understanding of Chinese orthodontists. Two kinds of subjective judgment techniques have generally been used: scoring and ranking. We chose the ranking method to obtain exact relative priorities across patients. A limitation of this method is that the exact relative priority can be determined only for cases within the same group; no observer can rank hundreds of photos within the same group, and comparability between groups is difficult. This problem was overcome by random sampling and grouping.
According to our results, although some proportions and angles were closely correlated with facial attractiveness, the explained variance in stepwise regression was only 23.5%, similar to the results reported by Kiekens et al.12 and Schmid et al.13 In contrast with the poor predictability of the proportions and angles, a 71.8% coincidence rate was obtained with the prediction model established using morphometric data. Although the final grading was based on only a three-point scale, this is still fairly good for clinical use.
The selection of landmarks strongly affects the results. There are several rules for landmark selection. Reproducibility of the landmarks and stability with changes in the head position are the primary criteria. Kiekens et al.12 found 45 landmarks with acceptable reproducibility from among 61 landmarks on full-face and profile images. In our study, most landmarks were similar to those used by Kiekens et al.12 With regard to the full-face photographs, there were two major differences. The number of landmarks around the eyes was reduced, while the number of landmarks in the lower one-third of the face (around the mouth) was increased. These modifications were made because the lower one-third of the face is the most important area of concern for orthodontists and orthognathic surgeons. The other difference was that the landmarks located on the periphery were abandoned because of their instability with changes in head position. For the smiling image, we introduced some landmarks related to incisor exposure, gingival display, and buccal corridors on the basis of the landmarks used for the full-face image. The smiling images used in our research were regular orthodontic records and were not taken in accordance with any standard conditions for smiling research. This is why we did not choose the same landmarks as other research papers, especially those related to smiling.17,18 Moreover, some other modifications to the landmarks on the lateral images were made so that landmarks more relevant to orthodontists were used: the pupil was replaced with a more stable landmark, the exocanthion; points Go (gonion) and lo (infraorbital), which can only be located precisely by palpation, were eliminated; and point C was added.
CONCLUSIONS
-
Although some ratios and angles were found to have a close correlation with facial attractiveness, they could not be used to comprehensively evaluate facial attractiveness from a set of orthodontic photographs.
-
The SVR function generated was found to be quite reliable for the evaluation of facial attractiveness. Therefore, a combination of geometric morphometrics with SVR may be a prospective method for comprehensive evaluation of facial attractiveness.

Procedure for sampling and grouping.

Frequency chart for case A1.

Landmarks identified in the frontal view.

Landmarks identified in the smiling view.

Landmarks identified in the lateral view.
Contributor Notes