Editorial Type:
Article Category: Research Article
 | 
Online Publication Date: 12 Aug 2016

Accuracy and reliability of 3D stereophotogrammetry: A comparison to direct anthropometry and 2D photogrammetry

,
,
,
, and
Page Range: 487 – 494
DOI: 10.2319/041415-244.1
Save
Download PDF

ABSTRACT

Objective: 

To evaluate the accuracy of three-dimensional (3D) stereophotogrammetry by comparing it with the direct anthropometry and digital photogrammetry methods. The reliability of 3D stereophotogrammetry was also examined.

Materials and Methods: 

Six profile and four frontal parameters were directly measured on the faces of 80 participants. The same measurements were repeated using two-dimensional (2D) photogrammetry and 3D stereophotogrammetry (3dMDflex System, 3dMD, Atlanta, Ga) to obtain images of the subjects. Another observer made the same measurements for images obtained with 3D stereophotogrammetry, and interobserver reproducibility was evaluated for 3D images. Both observers remeasured the 3D images 1 month later, and intraobserver reproducibility was evaluated. Statistical analysis was conducted using the paired samples t-test, intraclass correlation coefficient, and Bland-Altman limits of agreement.

Results: 

The highest mean difference was 0.30 mm between direct measurement and photogrammetry, 0.21 mm between direct measurement and 3D stereophotogrammetry, and 0.5 mm between photogrammetry and 3D stereophotogrammetry. The lowest agreement value was 0.965 in the Sn-Pro parameter between the photogrammetry and 3D stereophotogrammetry methods. Agreement between the two observers varied from 0.90 (Ch-Ch) to 0.99 (Sn-Me) in linear measurements. For intraobserver agreement, the highest difference between means was 0.33 for observer 1 and 1.42 mm for observer 2.

Conclusions: 

Measurements obtained using 3D stereophotogrammetry indicate that it may be an accurate and reliable imaging method for use in orthodontics.

INTRODUCTION

After the emergence of the soft tissue paradigm in orthodontics, orthodontic diagnosis and treatment planning based on dental and skeletal structures was replaced by orthodontic approaches built on the positive and negative characteristics of the facial soft tissues.1 With an objective evaluation of soft tissues; an efficient treatment planning can be made, and the patient can be accurately assessed at the end of the treatment.2

Various methods have been used to measure facial soft tissues. These methods were direct anthropometry,3 two-dimensional (2D) photogrammetry,4 lateral cephalometry,5 cone beam computed tomography (CBCT), and surface scanning methods (laser scanning, moiré topography, and the three-dimensional [3D] stereophotogrammetric method).68

Direct anthropometric measurement is a reliable and affordable method. Farkas et al.3 conducted important studies to create a large database of direct anthropometric measurements that can be used for facial measurements. However, direct anthropometry, which is considered as gold standard for facial measurements, has some disadvantages; for example, it is time consuming and requires patient compliance.9 Frontal and profile photographs are generally used for photogrammetric measurement. Two-dimensional images (photograph, lateral cephalometry) are a snapshot of a dynamic object and, therefore, require cooperation only during acquisition and are easier to obtain than direct measurement.10 On the other hand, with 2D imaging methods; magnification and distortion problems may occur and many variables can affect measuring standardization, such as illumination variations and object-camera distance. Furthermore, evaluation of the 2D cross-sectional images of a 3D object has significant inadequacies.11

The limitations of previous methods were managed using methods such as CBCT and laser scanners.6,12 Computed tomography is an expensive and an invasive produce because of the radiation used.13 In the laser surface scanning method, motion artifacts may occur if the scanning time is long.14 To overcome these limitations, 3D stereophotogrammetry was developed. With this method, 3D images are acquired by combining photographs captured from various angles with synchronous digital cameras. The advantages of this method are the lack of motion artifacts because of the short imaging time, high color resolution, and the opportunity for administration without harming patients for repeated analyses, quick configuration, imaging via advanced software, ease of archiving, and 3D storage of patient images.15,16

When 3D stereophotogrammetry devices become affordable and easily available, the method could become a routine procedure in orthodontic practice. Therefore, the present study aimed to compare three measurement methods and evaluate the accuracy of 3D stereophotogrammetry (3dMDflex System, 3dMD, Atlanta, Ga) by comparing it with direct anthropometry. Furthermore, the study also aimed to evaluate the intraobserver and interobserver reliability of 3D stereophotogrammetric measurements in 10 linear and six angular measurements.

MATERIALS AND METHODS

After approval was obtained from Ethical Committee of Gülhane Military Medical Academy in Turkey, the participants were informed verbally and in writing before the study, and volunteer consent forms were obtained from all participants.

Sample

The present study was conducted with 80 white participants between 25 and 45 years old. For an effect size of 0.4 at a .05 significance level, there could be more than 90% power with a sample size of 80. The mean age of the participants was 37.2 years, and there were 42 women (mean age  =  36.6 years; range  =  26.4–44.1 years) and 38 men (mean age  =  39.7 years, range  =  25.2–44.8 years).

The study included subjects who had no previous surgery on the face, no craniofacial defects, and no specific scar tissue on the face. The 2D and 3D images were acquired within the same day at the Gülhane Military Medical Academy, Medical Design and Manufacturing Center. Direct measurements were made on the day of image acquisition. Measurements of 2D and 3D images were made in a randomized manner and on a different day from direct measurements to avoid interference. First, 2D and 3D images were acquired, and then direct measurements were made through markings on the face.

Direct Measurements

Morphologic points required for linear measurements were determined by inspection and palpation and were then marked on the face. During the determination and measurement of the points, it was ensured that the patients were relaxed and seated with a natural head position and relaxed lips.

A digital millimeter caliper (sliding) was used to directly measure the distance between the four different points in the frontal plane (exocanthion, endocanthion, cheilion, and alare base), and the distance between seven different points in the sagittal plane (tragus, exocanthion, nasion, pronasale, subnasale, stomion, and menton) (Figure 1B). Measurements were made under the same room conditions with the same illumination (Table 1).

Figure 1. (A) Two-dimensional photogrammetry with millimeter calibration ruler. (B) Direct anthropometry with a digital millimeter sliding caliper.Figure 1. (A) Two-dimensional photogrammetry with millimeter calibration ruler. (B) Direct anthropometry with a digital millimeter sliding caliper.Figure 1. (A) Two-dimensional photogrammetry with millimeter calibration ruler. (B) Direct anthropometry with a digital millimeter sliding caliper.
Figure 1. (A) Two-dimensional photogrammetry with millimeter calibration ruler. (B) Direct anthropometry with a digital millimeter sliding caliper.

Citation: The Angle Orthodontist 86, 3; 10.2319/041415-244.1

Table 1. Anthropometric Landmarks and Definition of Linear Distances
Table 1.

Two-dimensional photogrammetric acquisition and measurements

All images were acquired by positioning the participants in the same position as in the direct measurement method. Frontal and profile photographs of the participants were captured under the same illumination using a professional camera (Canon EOS 40D, Tokyo, Japan, Tamron 17-50 mm f/2.8), which was placed on a tripod 60 cm away from the participant. A millimeter ruler was used in to avoid magnification errors, and care was taken to hold the ruler in the same plane with the frontal and profile measurement points (Figure 1A). All photographs were transferred to a computer, and calibration procedures were performed using View Box 4.0 software (dHAL Software, Kifissia, Greece). Ten linear and six angular measurements were made on the images.

Three-dimensional stereophotogrammetry acquisition and measurements

All 3D stereophotogrammetric images were acquired with the 3dMDflex system (3dMD, Atlanta, Ga). The device was calibrated before each acquisition. Participants were positioned in a natural head position on an adjustable chair. The images were acquired within 1.5 msec. After acquisition, the images were opened and checked in the 3dMD patient software; acquisition was repeated as required (if images were missing, blurred, or too bright). Measurements were made using the 3dMD patient software (Figure 2). The distance between the two points designated in this software can be obtained as surface distance and caliper distance. In the present study, measurements of caliper distance (the shortest distance between two points) were taken into account.

Figure 2. Three-dimensional stereophotogrammetric image with anatomic landmarks. Landmark abbreviations: Tr indicates tragus; Ex, exocanthion; En, endocanthion; Gb, glabella; N, nasion; Prn, pronasal; Cl, columella; Al, alare; Sn, subnasal; Ls, labiu m superior; St, stomion; Li, labium inferior; Sm, supramental; Me, menton; Ch, chellion.Figure 2. Three-dimensional stereophotogrammetric image with anatomic landmarks. Landmark abbreviations: Tr indicates tragus; Ex, exocanthion; En, endocanthion; Gb, glabella; N, nasion; Prn, pronasal; Cl, columella; Al, alare; Sn, subnasal; Ls, labiu m superior; St, stomion; Li, labium inferior; Sm, supramental; Me, menton; Ch, chellion.Figure 2. Three-dimensional stereophotogrammetric image with anatomic landmarks. Landmark abbreviations: Tr indicates tragus; Ex, exocanthion; En, endocanthion; Gb, glabella; N, nasion; Prn, pronasal; Cl, columella; Al, alare; Sn, subnasal; Ls, labiu m superior; St, stomion; Li, labium inferior; Sm, supramental; Me, menton; Ch, chellion.
Figure 2. Three-dimensional stereophotogrammetric image with anatomic landmarks. Landmark abbreviations: Tr indicates tragus; Ex, exocanthion; En, endocanthion; Gb, glabella; N, nasion; Prn, pronasal; Cl, columella; Al, alare; Sn, subnasal; Ls, labiu m superior; St, stomion; Li, labium inferior; Sm, supramental; Me, menton; Ch, chellion.

Citation: The Angle Orthodontist 86, 3; 10.2319/041415-244.1

Statistical Analysis

All statistical analyses were performed using SPSS version 20 software (SPSS Inc, Chicago, Ill). Normal distribution of the data was evaluated using the Shapiro-Wilk test and none of the variables violated the normality assumption. The paired samples t-test was used to determine the significance of the difference between the direct-measurement method, photogrammetry, and 3D stereophotogrammetry. Then, the intraclass correlation coefficient (ICC) was used to evaluate the agreement between measurements Finally, the Bland-Altman limits of agreement was used to determine the limits of agreement of the different methods. The results were determined to be clinically acceptable at an arbitrary value of 2 mm between two different measurements.

In the 3D stereophotogrammetric images, in order to evaluate the interobserver agreement, a different observer made the measurements in the same way as the first observer, to determine the intraobserver agreement, both observers made the same measurements 1 month after the initial measurements. Intraobserver and interobserver agreement for the 3D stereophotogrammetry method were evaluated similarly using the paired samples t-test, ICC, and the Bland-Altman limits of agreement. The results were determined to be clinically acceptable at an arbitrary value of 2 mm between observers and within an observer. The statistical significance level was .05 in all statistical analyses.

RESULTS

Comparison of the Methods

There was no statistically significant difference between the methods at a significance level of P < .05. All differences between means were observed to be <1 mm for all parameters. The highest mean difference was 0.30 mm between direct measurement and photogrammetry, 0.21 mm between direct measurement and 3D stereophotogrammetry, and 0.5 mm between photogrammetry and 3D stereophotogrammetry. Table 2 presents the Bland-Altman limits of agreement between mean differences. Table 3 presents the very high level of agreement of all three methods together and with each other in measurements; the ICC values were between 0.96 (Sn-Pro) and 0.99.

Table 2. Mean Difference, Bland-Altman Upper and Lower Limits of Agreements Estimated by Mean Difference ± 1.96 Standard Deviation of the Differences, and CIs of Differences Between Measurementsa
Table 2.
Table 3. Reliability Coefficient (ICC) Between Methodsa
Table 3.

Intraobserver and Interobserver Agreement in 3D Stereophotogrammetry

In linear measurements, the intraobserver agreement varied from 0.97 (Sn-St) to 0.99 for observer 1, and from 0.91 (N-Sn) to 0.98 for observer 2. A significant difference was found in N-Prn and N-Sn measurements for observer 2. The highest difference between means was 0.33 mm for observer 1 compared with 1.42 mm for observer 2. When the means were compared between the two observers, a significant difference was found in the four parameters (En-En, Sn-St, N-Sn, and N-Prn). The highest difference between the means was 1.42 mm in N-Prn. However, all of the differences between the means of the two observers were <2 mm (Table 4). Table 4 presents the Bland Altman limits of the intraobserver and interobserver agreement. The agreement between the two observers varied from (ICC) 0.90 (Ch-Ch) to 0.99 (Sn-Me) (Table 5).

Table 4. Mean Difference, Bland-Altman Upper and Lower Limits of Agreement Estimated by Mean Difference ± 1.96 Standard Deviation of the Differences and CIs of Differences Between Measurements Within and Between Observersa
Table 4.
Table 5. Reliability Coefficient (ICC) Within and Between Observersa
Table 5.

In angular measurements, the highest and lowest mean difference of the intraobserver measurements was 0.87° (NFA) and 0.09° (NLA), respectively. There was a significant difference between intraobserver measurements in the lower third for observer 1 and in the NFA for observer 2 (P < .05). Mean difference between measurements of different observers and within the same observer was <1.5 mm for all parameters (Table 4). Table 4 presents the Bland-Altman limits of agreement between the two observers and within the same observer. Intraobserver and interobserver agreement (ICC) varied from 0.85 to 0.99 in angular measurements (Table 5). Despite the significant difference in NFA between observers, the difference was 1.13°.

DISCUSSION

The present study evaluated the accuracy and reliability of 3D stereophotogrammetry. A very high level of agreement was found between direct anthropometry, photogrammetry, and 3D stereophotogrammetry; the highest mean difference was 0.5 mm. For intraobserver and interobserver reliability, the mean difference was <2 mm in linear measurements and <2° for angular measurements in 3D stereophotogrammetric images. These values were considered to be clinically insignificant.

Quantitative evaluations that are performed in facial soft tissues, in addition to clinical evaluations, are important for assessing treatment aims and treatment outcomes. CBCT can be used for soft tissue analysis three dimensionally, but it has several disadvantages.12 Besides, the measurement difference between CBCT and 3D stereophotogrammetry was clinically insignificant.17 So, the use of stereophotogrammetry is considered adequate in imaging soft tissues and measuring facial soft tissues. The direct anthropometry method is not routinely used in clinical practice; however, it is important to provide actual measurement results when accurate and careful measurements are made. In this study the mean difference was found to be <0.5 mm between these two methods. The validity of 3D stereophotogrammetry was evaluated methodologically using different types and brands of devices. Such studies have found <2 mm mean difference between different methods, and between intraobserver and interobserver measurements.9,1720

One of the most important factors was the reproducibility of the points determined for measurements. In fact, the low-range variation in the present study is believed to have resulted from minor deviations that occurred during the placement of the morphologic points. In the literature, deviations are reported to be mostly caused by observer errors during placement of anthropometric points.21,22 Plooij et al.23 found intraobserver reliability varying between 0.90 and 0.99. They found interobserver agreement above 0.8 in most points. Reproducibility was reported to be substantially lower than 0.5 mm. Aldridge et al.19 reported that the grand mean of the precision calculated across subjects along all axes for all landmarks was 0.827 mm. Khambay et al.24 found a reproducibility error of <0.5 mm. Lübbers et al.25 examined the precision and accuracy of 3D stereophotogrammetry and reported a mean global error between 0.1 and 0.5 mm.

The greatest and most significant difference was 1.42 mm in the N-Prn measurement for interobserver agreement. However, the difference was < 2 mm for any parameters. The highest interobserver agreement was in the Al-Al measurement (95% CI, –0.21 0.56; mean  =  0.17 mm). Agreement between measurements was above 0.9 in all measurements based on ICC results. These findings are consistent with the studies by Aldridge et al.,19 Wong et al.,9 and Schaaf et al.26 According to Heike et al.,27 the intrarater reliability correlation coefficients for the 3D stereophotogrammetric images were greater than or equal to 0.95 for 26 of the 30 measurements and mean absolute differences were <1 mm.

The present study found agreement between 2D photogrammetry and other methods, which suggests that 2D measurements can be used safely in images acquired with accurate technique and attention. Farkas et al.28 reported that the difference between direct anthropometry and 2D photogrammetry may be caused by the distortions in photogrammetry. It seems very important to make the calibration in the same plane as the measuring points in 2D measurements. Furthermore, it is possible to reuse such data when required and to use such data by comparing it with other methods.

For reliability, it is important to have a clear acquisition of the facial regions to be measured. There may be image errors in the ear region (e.g., tragus) in systems that can capture image up to 180°. The 3dMDflex device, which was used in the present study, can be converted into different modules according to the desired imaging area. In the present study, 360° images were acquired using five modular units (front: two, rear: two and top: one) in order to avoid image loss all over the face, including the ear region. Acquisition was repeated in case of image distortions, especially in the ear region, depending on the facial morphology.

Evaluating the properties of facial soft tissue clinically and quantitatively is the priority of contemporary orthodontic diagnosis and treatment planning. Many imaging techniques have been used to evaluate facial soft tissues, but 3D stereophotogrammetry attracted more attention because of the many advantages mentioned earlier. This method was found to be accurate and reliable to be integrated into orthodontic clinical practice. Furthermore, it is possible to process and analyze 3D stereophotogrammetric images in different software in accordance with different clinical or research purposes.

CONCLUSIONS

  • Measurements using 3D stereophotogrammetry were consistent with both direct anthropometric and 2D photogrammetric measurements.

  • The high intraobserver and interobserver reproducibility suggest that this method can be used reliably.

  • Although some parameters had statistically significant differences, they were not clinically significant.

REFERENCES

  • 1.
    Ackerman JL,
    Proffit WR,
    Sarver DM.
    The emerging soft tissue paradigm in orthodontic diagnosis and treatment planning. Clin Orthod Res. 1999;2:4952.
  • 2.
    Primozic J,
    Perinetti G,
    Richmond S,
    Ovsenik M.
    Three-dimensional evaluation of facial asymmetry in association with unilateral functional crossbite in the primary, early, and late mixed dentition phases. Angle Orthod. 2013;83253258.
  • 3.
    Farkas LG,
    Posnick JC,
    Hreczko TM.
    Anthropometric growth study of the head. Cleft Palate Craniofac J. 1992;29:303308.
  • 4.
    Dimaggio FR,
    Ciusa V,
    Sforza C,
    Ferrario VF.
    Photographic soft-tissue profile analysis in children at 6 years of age. Am J Orthod Dentofacial Orthop. 2007;132:475480.
  • 5.
    Bavbek NC,
    Tuncer BB,
    Tortop T.
    Soft tissue alterations following protraction approaches with and without rapid maxillary expansion. J Clin Pediatr Dent. 2014;38:277283.
  • 6.
    Baik H-S,
    Kim S-Y.
    Facial soft-tissue changes in skeletal Class III orthognathic surgery patients analyzed with 3-dimensional laser scanning. Am J Orthod Dentofacial Orthop. 2010;138:167178.
  • 7.
    Zhao H,
    Du H,
    Li J,
    Qin Y.
    Shadow moiré technology based fast method for the measurement of surface topography. Appl Opt. 2013;52:78747881.
  • 8.
    Ayoub AF,
    Wray D,
    Moos KF,
    et al
    . Three-dimensional modeling for modern diagnosis and planning in maxillofacial surgery. Int J Adult Orthodon Orthognath Surg. 1996;11:225233.
  • 9.
    Wong JY,
    Oh AK,
    Ohta E,
    et al
    . Validity and reliability of craniofacial anthropometric measurement of 3D digital photogrammetric images. Cleft Palate Craniofac J. 2008;45:232239.
  • 10.
    Edler R,
    Wertheim D,
    Greenhill D.
    Comparison of radiographic and photographic measurement of mandibular asymmetry. Am J Orthod Dentofacial Orthop. 2003;123:167174.
  • 11.
    Cutting CB,
    McCarthy JG,
    Karron DB.
    Three-dimensional input of body surface data using a laser light scanner. Ann Plast Surg. 1988;21:3845.
  • 12.
    Kuijpers MAR,
    Chiu Y-T,
    Nada RM,
    Carels CEL,
    Fudalej PS.
    Three-dimensional imaging methods for quantitative analysis of facial soft tissues and skeletal morphology in patients with orofacial clefts: a systematic review. PLoS One. 2014;9:e93442.
  • 13.
    Littlefield TR,
    Kelly KM,
    Cherney JC,
    Beals SP,
    Pomatto JK.
    Development of a new three-dimensional cranial imaging system. J Craniofac Surg. 2004;15:175181.
  • 14.
    Hajeer MY,
    Millett DT,
    Ayoub AF,
    Siebert JP.
    Applications of 3D imaging in orthodontics: part I. J Orthod.. 2004;31:6270.
  • 15.
    Brons S,
    van Beusichem ME,
    Bronkhorst EM,
    et al
    . Methods to quantify soft-tissue based facial growth and treatment outcomes in children: a systematic review. PLoS One. 2012;7:e41898.
  • 16.
    Kochel J,
    Meyer-Marcotty P,
    Strnad F,
    Kochel M,
    Stellzig-Eisenhauer A.
    3D soft tissue analysis—part 1: sagittal parameters. J Orofac Orthop. 2010;71:4052.
  • 17.
    Metzger TE,
    Kula KS,
    Eckert GJ,
    Ghoneima AA.
    Orthodontic soft-tissue parameters: a comparison of cone-beam computed tomography and the 3dMD imaging system. Am J Orthod Dentofacial Orthop. 2013;144:672681.
  • 18.
    Weinberg SM,
    Scott NM,
    Neiswanger K,
    Brandon CA,
    Marazita ML.
    Digital three-dimensional photogrammetry: evaluation of anthropometric precision and accuracy using a Genex 3D camera system. Cleft Palate Craniofac J. 2004;41:507518.
  • 19.
    Aldridge K,
    Boyadjiev SA,
    Capone GT,
    DeLeon VB,
    Richtsmeier JT.
    Precision and error of three-dimensional phenotypic measures acquired from 3dMD photogrammetric images. Am J Med Genet A. 2005;138A:247253.
  • 20.
    Winder RJ,
    Darvann TA,
    McKnight W,
    Magee JDM,
    Ramsay-Baggs P.
    Technical validation of the Di3D stereophotogrammetry surface imaging system. Br J Oral Maxillofac Surg. 2008;46:3337.
  • 21.
    Kohn LA,
    Cheverud JM,
    Bhatia G,
    Commean P,
    Smith K,
    Vannier MW.
    Anthropometric optical surface imaging system repeatability, precision, and validation. Ann Plast Surg. 1995;34:362371.
  • 22.
    Tzou C-H. J,
    Artner NM,
    Pona I,
    et al
    . Comparison of three-dimensional surface-imaging systems. J Plast Reconstr Aesthet Surg. 2014;67:489497.
  • 23.
    Plooij JM,
    Swennen GRJ,
    Rangel FA,
    et al
    . Evaluation of reproducibility and reliability of 3D soft tissue analysis using 3D stereophotogrammetry. Int J Oral Maxillofac Surg. 2009;38:267273.
  • 24.
    Khambay B,
    Nairn N,
    Bell A,
    Miller J,
    Bowman A,
    Ayoub AF.
    Validation and reproducibility of a high-resolution three-dimensional facial imaging system. Br J Oral Maxillofac Surg. 2008;46:2732.
  • 25.
    Lübbers H-T,
    Medinger L,
    Kruse A,
    Grätz KW,
    Matthews F.
    Precision and accuracy of the 3dMD photogrammetric system in craniomaxillofacial application. J Craniofac Surg. 2010;21:763767.
  • 26.
    Schaaf H,
    Pons-Kuehnemann J,
    Malik CY,
    et al
    . Accuracy of three-dimensional photogrammetric images in non-synostotic cranial deformities. Neuropediatrics. 2010;41:2429.
  • 27.
    Heike CL,
    Cunningham ML,
    Hing AV,
    Stuhaug E,
    Starr JR.
    Picture perfect? Reliability of craniofacial anthropometry using three-dimensional digital stereophotogrammetry. Plast Reconstr Surg. 2009;124:12611272.
  • 28.
    Farkas LG,
    Bryson W,
    Klotz J.
    Is photogrammetry of the face reliable? Plast Reconstr Surg. 1980;66:346355.
Copyright: © 2016 by The EH Angle Education and Research Foundation, Inc.
Figure 1.
Figure 1.

(A) Two-dimensional photogrammetry with millimeter calibration ruler. (B) Direct anthropometry with a digital millimeter sliding caliper.


Figure 2.
Figure 2.

Three-dimensional stereophotogrammetric image with anatomic landmarks. Landmark abbreviations: Tr indicates tragus; Ex, exocanthion; En, endocanthion; Gb, glabella; N, nasion; Prn, pronasal; Cl, columella; Al, alare; Sn, subnasal; Ls, labiu m superior; St, stomion; Li, labium inferior; Sm, supramental; Me, menton; Ch, chellion.


Contributor Notes

Corresponding author: Dr Furkan Dindaroğlu, Ege Üniversitesi Diş Hekimliği Fakültesi Ortodonti, Anabilim Dall, Bornova, lzmir, Turkey (e-mail: furkandindaroglu@yahoo.com.tr)
Received: 01 Apr 2015
Accepted: 01 Jun 2015
  • Download PDF