Precision and accuracy assessment of single and multicamera three-dimensional photogrammetry compared with direct anthropometry
ABSTRACT
Objectives
To assess the precision and accuracy of single-camera photogrammetry (SCP) and multicamera photogrammetry (MCP) compared with direct anthropometry (DA).
Materials and Methods
A total of 30 participants were recruited, and 17 soft tissue landmarks were identified and used to complete a total of 16 measurements. Using SCP and MCP, two three-dimensional (3D) images were acquired from each participant. All 3D measurements and direct measurements were measured twice by the same operator to assess intraexaminer repeatability. Intraclass coefficients (ICCs) were used to evaluate intraexaminer repeatability and interexaminer agreement of the methods. Nonparametric bootstrap analyses were used to compare the means of the measurements among the three methods.
Results
All three methods showed excellent intraexaminer repeatability (ICCs > 0.90), except interpupillary distance (ICC = 0.86) measured by SCP. Both SCP and MCP showed excellent interexaminer agreement (ICCs > 0.90), except interpupillary distance (ICC = 0.79), left gonion-pogonion (ICC = 0.74), and columella-subnasale-labrale superior angle (ICC = 0.86) measured by SCP. Overall, there was good agreement between methods, except for columella-subnasale-labrale superior angle (ICC = 0.40) between SCP and MCP.
Conclusions
Both SCP and MCP techniques were found to be reliable and valid options for 3D facial imaging. SCP produced slightly larger mean values for several measurements, but the differences were within a clinically acceptable range. Because of the larger margin of errors, measurements including the gonial area and subnasale should be assessed with caution.
INTRODUCTION
For several years, the dental field has worked to assess and incorporate digital technology into treatment planning.1 As new devices are introduced, it is crucial that sufficient research be performed to evaluate them and ensure they are at least equivalent, if not superior, to technologies currently used within the profession.
Historically, several direct and indirect two-dimensional (2D) methods have been used to collect and measure craniofacial data. However, it is recognized that most of these methods do not fully capture the three-dimensional (3D) identity of a patient's face.1 Direct anthropometry (DA) is advantageous because it is a reliable, repeatable source for facial measurements and has a large, normative database.2 However, it has been shown to take a long time to complete data collection in this manner, which is not ideal in a clinical setting.1 Two-dimensional photogrammetry is a great resource but is often inaccurate.1 Lastly, cephalometry is a practical option for direct measurements. However, it produces a 2D representation of a 3D structure and provides only an analysis of the profile, not the whole face. In addition, cephalometry exposes the patient to radiation, which makes it suboptimal.3
Recently, 3D cameras have been introduced, and they possess various advantages over 2D methods.4,5 They have the ability to quantify linear angles, surface areas and volumes, user-guided landmark localization, and various statistical shape analyses.4,5 A 3D surface analysis allows for statistical analyses directly from the 3D captured image, reduced chair time, and a permanent archive of the patient's facial profile for treatment analysis.
Multicamera photogrammetry (MCP), also called stereophotogrammetry, is a technique that uses two or more stationary cameras configured as a stereo pair to obtain 3D coordinates of facial morphology.6 Previous studies have demonstrated that 3D stereophotogrammetry is as accurate and reliable as DA.7–9 Therefore, MCP systems have been instituted at learning institutions and some offices to collect data. A significant drawback to this system is the substantial cost associated with acquiring such technology. Because of the significant financial investment, many companies and practitioners have sought to find a less costly alternative with equivalent diagnostic and clinical benefits.
With continued advancement in technology, especially considering ease of portability, more companies have developed face scanners and software applications (apps) that can be used on smartphones or tablets to collect 3D patient images in several settings. Compared with MCP systems, single-camera photogrammetry (SCP) systems are less expensive and require less physical equipment. The app collects data using a single, small, and portable camera that rotates around the patient's face, or the patient rotates his or her head in front of a stationary camera. The acquired data can then be exported into several patient management software programs or platforms to be incorporated with other patient information previously obtained.
The precision and accuracy of 3D facial scanning methods have been investigated extensively in the literature.1,3,10–21Table 1 provides a summary of the research methodology used in these studies. Some of these studies used mannequins to simulate in vivo human studies and limit the margin of error from involuntary facial movements.10,13,19,21 Although the data from these in vitro studies were useful to assess the accuracy of the techniques, a question remains regarding the precision of these techniques in real-life clinical circumstances (ie, facial expressions, involuntary movements, facial hairs, lightning). Other in vivo studies either evaluated only the precision of MCP11,12,14,16,17 or compared the precision and accuracy of MCP with DA.3,20 Only one other clinical study has compared the precision and accuracy of SCP and MCP systems with DA.20 Therefore, the aim of this clinical study was to assess the precision and accuracy of SCP and MCP techniques and compare the results with DA. The null hypothesis was that there would be no differences in the precision and accuracy among these three different methods.

MATERIALS AND METHODS
Study Sample
The study sample included 30 individuals. Informed consent was obtained through protocol no. 2011777871 and approved by the institutional review board (IRB) of Indiana University. Participants were recruited using an IRB-approved flyer displayed throughout the Indiana University School of Dentistry. Inclusion criteria were adults aged 18 years or older who were students and/or personnel at the School of Dentistry willing to participate and sign an informed consent. Exclusion criteria included individuals with facial hair, surgical scars, or significant facial defects and individuals who could not show their hair or skin from the neck up.
Data Collection
A total of 17 soft tissue landmarks were identified and used to make 12 linear and 4 angular measurements (Figure 1). Before data collection, operator 1 (Dr Staller) was trained to mark anatomical landmarks, use calipers for direct facial measurements, and collect data via 3D imaging. Then, operator 2 (Dr Anigbo) was trained and calibrated with operator 1 for landmark identification and analysis of the 3D images. Landmarks were marked directly on the face of the participants with a black, temporary tattoo marker (BIC, Clichy, France). Pittsburgh 6-inch digital calipers (Harbor Freights Tools, Calabasas, Calif) were used to measure the distance between landmarks directly. Before each use, the calipers were calibrated to ensure accuracy. The digital caliper measured to 0.01 mm precision. All 3D facial scans were acquired in the same room under dark conditions. The scans were performed with the participants in a natural head position, with teeth in occlusion and lips at rest. To determine the natural head position, all participants were asked to position their heads at the greatest comfort (self-balanced position) and look toward a distant spot at the wall.22 An SCP device, Bellus Face Camera Pro (Bellus 3D, Inc., Campbell, Calif), was positioned on top of the computer monitor, and the participants were prompted to turn their heads to capture facial features and compile a 3D image. The final set of 3D images was obtained with the MCP device 3dMD Trio (3dMD LLC, Atlanta, Ga). All measurements were repeated the same day by the same operator (Dr Staller) to calculate the intraexaminer repeatability of the methods.



Citation: The Angle Orthodontist 92, 5; 10.2319/101321-770.1
Statistical Analyses
All statistical analyses were performed separately for each calculated linear distance. Intraclass coefficients (ICCs) were used to evaluate the agreement between SCP, MCP, and DA measurements. Nonparametric bootstrap analyses were used to compare the means of the measurements from the three methods. This allowed for the assessment of bias between SCP and MCP measurements compared with DA measurements and mean differences between SCP and MCP measurements. A 5% statistical significance level was used for all tests. A minimal clinically important difference (MCID) of 2 mm was used as a threshold for comparisons of the three methods.23,24 Analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, N.C.).
Before conducting the study, the sample size was determined based on the precision for estimating the ICCs. With a sample size of 30 participants, the width of the 95% confidence intervals for the ICCs would range between 0.08 and 0.28 from the estimated ICCs, assuming ICCs between 0.8 and 0.95.
RESULTS
Table 2 provides the descriptive statistics of the measurements and includes the mean, standard deviation (SD), minimum, and maximum values. The ICCs for intraexaminer repeatability of the methods are shown in Table 3. SCP displayed excellent intraexaminer repeatability (ICCs > 0.90) for all measurements, except interpupillary distance (ICC = 0.86). Both MCP and DA showed excellent intraexaminer repeatability (ICCs > 0.90) for all measurements.


The interexaminer agreement analysis was performed only between SCP and MCP, and the results are shown in Table 4. Of 16 measurements, 13 showed excellent interexaminer agreement (ICCs > 0.90) for SCP. Interpupillary distance and the columella-subnasale-labrale superior angle showed good agreement (0.90 > ICCs > 0.75), whereas the left gonion to pogonion distance showed moderate agreement (ICC = 0.74). All measurements performed by MCP showed excellent interexaminer agreement (ICCs > 0.90).

The results of the agreements between methods are shown in Table 5. Between SCP and MCP, 10 measurements showed excellent agreement (ICCs > 0.90), and four measurements showed good agreement (0.90 > ICCs > 0.75). The left gonion-pogonion distance showed moderate agreement (0.75 > ICCs > 0.50), and the columella-subnasale-labrale superior angle showed poor agreement (ICC < 0.50). Between SCP and DA, five measurements showed excellent agreement (ICCs > 0.90), and four measurements showed good agreement (0.90 > ICCs > 0.75). Only the pronasale-menton, interpupillary, and left gonion-pogonion distances showed moderate agreement (0.75 > ICCs > 0.50). Between MCP and DA, eight measurements showed excellent agreement (ICCs > 0.90), and two measurements showed good agreement (0.90 > ICCs > 0.75). Only the pronasale-menton and interpupillary distances showed moderate agreement (0.75 > ICCs > 0.50).

The final analysis involved the statistical comparison of the mean measurements obtained by the three methods, and the results are shown in Table 6. Of 16 measurements, 10 differed significantly between SCP and MCP (P < .05), and only three (right gonion-pogonion, left gonion-pogonion, and columella-subnasale-labrale superior) reached an MCID level (>2 mm). Between SCP and DA, the mean values of nine of 12 measurements were significantly different (P < .05), and only five of them (subnasale-menton, interpupillary distance, right tragion-pogonion, right gonion-pogonion, and left gonion-pogonion) reached an MCID level (>2 mm). Five of 12 measurements differed significantly between MCP and DA (P < .05), with only one of them (interpupillary distance) reaching an MCID level (>2 mm).

DISCUSSION
The results showed that both SCP and MCP were highly precise for making face height measurements. In addition to being precise, SCP and MCP were also accurate. The mean differences between the methods did not reach the MCID level, except for the 2.05-mm difference between SCP and DA observed for the subnasale-menton distance. When considering the relatively lower accuracy with the subnasale landmark, it is recommended that the more accurate pronasale be used for face height measurements.
Precision assessment of the cheek measurements revealed excellent intraexaminer repeatability and interexaminer agreement, except for the left gonion-pogonion measurements, which showed moderate agreement with SCP. Gonion was by far the most difficult landmark to locate among these landmarks. Two factors that could explain the difficulty in identifying gonion include it being positioned so lateral to the midline and that it is an imaginary landmark on a curvature at the angle of the mandible. Both methods showed high accuracy in measurements involving the tragus, pronasale, and pogonion. However, both right and left gonion to pogonion distances displayed clinically significant differences between both SCP and MCP and SCP and DA. SCP measured these distances approximately 3.5- to 5-mm longer, which could have been attributed to either incorrect landmark identification or enlargement of this area with this technique. There were a few instances in which there was shadowing on the image in this region and the marking was small or surrounded by freckles on the participants' skin. Although most measurements in the study displayed highly acceptable levels of precision and accuracy, this factor could have influenced the results for some of the parameters. This result showed low accuracy of SCP while acquiring areas with more curvature, especially the gonial angle. Similarly, Weinberg et al. found that estimates of error magnitude of 3D images tended to be higher in variables of greater size, landmarks that were harder to see, and variables crossing the labial fissure.1 Hong et al. found that areas with more curvature showed the greatest error, and longer measurements showed larger variations compared with short distances.19 Overall, both SCP and MCP were precise and accurate in profile measurements of the cheeks. However, MCP was more accurate in measuring the soft tissue mandibular corpus length. The findings suggested that additional improvement is needed with SCP when capturing the gonial areas.
When measuring the interpupillary distance, SCP showed relatively lower intraexaminer repeatability and interexaminer agreement compared with MCP. This is likely attributed to the difficulty in locating the middle of the pupil in each measurement. Both 3D methods also displayed clinically meaningful differences with DA, which also highlights difficulty in measuring the distance between the middle of the pupils directly compared with indirectly on an image.
With both 3D imaging techniques, one of the most easily imaged and measured anatomic structures was the nose. Being in the midline, it was not affected by rotation of the head during image capture with SCP. Also, anatomic landmarks such as nasion and pronasale were easily located and had a smaller margin of error. In this study, measurements including these two landmarks had excellent intraexaminer repeatability and interexaminer agreement. However, this was not the case for subnasale, which was difficult to locate and had a wider margin of error. As with the current study, Kim et al. observed lower interexaminer reliability for one measurement, nasion-subnasale (0.795).20 Masoud et al. also reported difficulties in locating the subnasale because it varied tremendously with lip posture, as well as nose and lip morphology.25 These results underscore the need for being cautious when evaluating measurements with 3D imaging methods that involve the subnasale.
Overall, both SCP and MCP methods displayed excellent intraexaminer repeatability and interexaminer agreement for facial convexity measurements, except for upper lip convexity, which showed good interexaminer agreement with SCP. This measurement was found to be the least accurate because there was a 7.15° difference between SCP and MCP. However, without a DA-derived gold standard for angular measurements, it could not be ascertained clearly if one method was more or less accurate than the other.
There were a few limitations with the current study. One limitation was that the participants included were healthy adults without any significant craniofacial anomalies. Further studies are needed to evaluate the accuracy of these systems in individuals at different growth stages and with craniofacial anomalies. Likewise, the inclusion of individuals with facial hair could have resulted in slightly different results but could help expand the applicability of the findings in a broader segment of the population.
CONCLUSIONS
-
Both SCP and MCP techniques were found to be reliable and valid options for 3D facial imaging.
-
SCP produced slightly larger mean values for several measurements, but the differences were within the clinically acceptable range.
-
With 3D facial imaging techniques, the gonial area and subnasale need to be assessed cautiously because of the larger margins of error observed.

Anatomic soft tissue landmarks used in the study. (A) Frontal view. (B) Profile view.
Contributor Notes