INTRODUCTION

Cephalometric analysis is an indispensable diagnostic tool in clinical orthodontics. It enables assessment of dentofacial proportions, anatomical basis of malocclusion, growth pattern, and post-treatment changes.¹

Traditionally, cephalometric analysis has been conducted using a manual drawing technique involving acetate tracing paper, rulers, and protractors. However, this approach is inherently time-consuming and susceptible to inter- and intra-operator variability.² Over recent decades, the advent of computer-aided cephalometric analysis software allowed for a more efficient measurement process and reduced error in landmark identification and linear/angular measurements compared to the manual technique.³ Nonetheless, variability in landmark identification remains a significant source of random error, even with computer-aided systems.^4–6 Thus, manual and computer-aided cephalometric analyses are open to considerable subjectivity and remain time-intensive, although to a different extent.

Given these limitations, the integration of artificial intelligence (AI) has been tested to automate cephalometric analysis. Briefly, these models can be broadly categorized into semi-automatic and fully automatic systems. Semi-automatic AI models assist operators by providing tools for landmark detection, measurement estimation, and image segmentation. In contrast, fully automated systems perform all aspects of cephalometric analysis autonomously, including landmark identification, measurement computation, and reporting.^7,8

Despite the obvious advantages of the fully automated method,⁹ a general consensus for its clinical application has not been reached due to accuracy and reliability concerns.^10,11 Indeed, the literature has shown conflicting results since some studies^12–15 reported good-to-excellent agreement with manual landmarking, whereas other studies^16–18 reported a lower level of accuracy, especially for linear measurements. Additionally, a recent meta-analysis¹⁹ highlighted biases in patient selection, insufficient randomization processes, and absence of standardized protocols.

A further limitation of existing studies is that none of them considered the influence of head positional error on automated landmark identification. In such cases, when the head is slightly rotated and/or inclined relative to the ideal position, in which the midsagittal plane of the patient’s head should align parallel to the detector’s plane, it is suggested to average the positions of bilateral structures for landmark identification.²⁰ This method has been shown to ensure reliable cephalometric analysis while avoiding the need for repeated acquisition of cephalograms. However, no studies in the literature have tested AI models to address this task under the conditions of imprecise patient posture.

The present study aimed to compare the accuracy and the reliability of different fully automated AI-based cephalometric software applications in deriving linear and angular measurements performed on cephalograms featuring correct and incorrect head posture. The null hypothesis was that fully automatic software would achieve excellent agreement in all cephalometric measurements compared to the manual method, under ideal and challenging conditions.

MATERIALS AND METHODS

This retrospective study was approved by the Institutional Review Committee of the University of Catania (IRC n° A.Q.A.M.DI. 119/2020/PO) and was conducted in accordance with the principles of the Declaration of Helsinki. Informed consent was obtained from all the patients and/or their legal guardians.

Cephalometric Analysis

Cephalometric measurements considered in the present study were derived from the American Board of Orthodontics (ABO) cephalometric guidelines.²² Calibration of measurements was performed using a known distance (20 mm) between two ruler points.

Nonautomated computer-aided landmarking was performed in CHP and IHP groups by an experienced orthodontist with more than 10 years of experience, using Dolphin 11.8 software (Patterson Dental Supply) (Figure 1). Measurements of 10 randomly selected cephalograms were repeated in three sessions with an interval of 1 week between each repetition. The midpoint was constructed to make a single landmark for bilateral structures and double images (IHP group). The experienced orthodontist was not aware of the AI-driven cephalometric analysis results before finishing manual landmarking.

View Figure

• Download as Powerpoint
• Download Figure

Fully automatic AI-driven landmarking was performed in CHP and IHP groups without any human correction and before nonautomated cephalometric analysis. Data were collected by an expert operator author who was not involved in the nonautomated landmarking process. Three software programs were tested: (1) WebCeph (AssembleCircle Corp., Gyeonggi-do, Republic of Korea, https://webceph.com) (Figure 2), (2) Ceph Assistant (Budapest, Hungary, https://www.cephassistant.com/) (Figure 3), and (3) AudaxCeph (Audax, d.o.o., Ljubljana, Slovenia, https://www.audaxceph.com/) (Figure 4). Digital lateral cephalograms were uploaded in the systems that automatically identified the landmarks and performed the cephalometric analysis.

View Figure

• Download as Powerpoint
• Download Figure

View Figure

• Download as Powerpoint
• Download Figure

View Figure

• Download as Powerpoint
• Download Figure

In addition, the performance of AI cephalometric analyses was assessed with the success classification rate (SCR) compared to nonautomated cephalometric analysis. This is a metric for the classification of anatomical types established by the Symposium on Biomedical Imaging conferences in 2015.²³ The SCR was applied to the ABO parameters,²² which were included within this classification: ANB(^) (type 1: 3.2°–5.7° Class I; type 2: >5.7° Class II; type 3: <3.2° Class III), SNA(^) (type 1: 79.4°–83.2° normal maxilla; type 2: >83.2° prognathic maxilla; type 3: <79.4° retrognathic maxilla), SNB(^) (type 1: 76.4°–78.7° normal mandible; type 2: <74.6° retrognathic mandible; type 3: >78.7° prognathic mandible), SN-MP(^) (type 1: 26.8°–31.4°; type 2: >31.4° mandible high angle; type 3: <26.8° mandible low angle).

RESULTS

Measurement Reliability

Intra-examiner ICC for repeated measurements was >0.90 indicating high reliability. Table 1 reports the results of manual tracing with Dolphin and the comparisons with AI software.

Table 1.Comparison of Manual Measurements and AI Fully Automated Measurements Based on WebCeph, Ceph Assistant, and AudaxCeph^a

View Fullscreen

Intragroup Assessments

For WebCeph software, none of the cephalometric parameters showed excellent agreement in the CHP group. Indeed, good agreement was achieved only for SNB (^), U1 to SN, L1 to NB (^), and LL to E-line, whereas the other values showed moderate-to-poor agreement. Go-Me (mm), U1 to NA (mm), and L1 to NB (mm) showed the worst agreement (ICC < 0.50). In the IHP group, SN-MP (^) demonstrated excellent agreement (ICC = 0.94), whereas good agreement was achieved for SNB (^), U1 to SN (^), and LL to E-line (mm). All the other measurements showed moderate-to-poor agreement. Go-Me (mm), U1 to NA (mm), L1 to NB (mm), and L1 to NB (^) showed the worst agreement (ICC < 0.50) (Table 2).

Table 2.Comparisons of AI Fully Automated Cephalometric Analysis Based on WebCeph, Ceph Assistant, and AudaxCeph

View Fullscreen

For Ceph Assistant software, none of the cephalometric parameters showed excellent agreement in the CHP group. Indeed, good agreement was achieved for SNA (^), SNB (^), SN-MP (^), U1 to NA (mm), U1 to NA (^), L1 to NB (mm), and LL to E-line (mm), whereas the other values showed moderate-to-poor agreement. Go-Me (mm) demonstrated the worst ICC agreement (ICC = 0.12). In the IHP group, SN-MP (^) showed excellent agreement (ICC = 0.96), whereas good agreement (ICC: 0.75–0.90) was achieved for SNB (^), FMA (^), U1 to SN (^), U1 to NA (^), and UL to E-line (mm). All the other measurements had moderate-to-poor agreement. Go-Me (mm) showed the worst result (ICC = −0.21).

Regarding AudaxCeph software, SNA (^), SNB (^), and L1 to NB (^) showed excellent agreement in the CHP group. Good agreement was achieved for SN-MP (^), U1 to SN (^), U1 to NA (^), U1 to NA (mm), L1 to MP (^), L1 to NB (^), and L1 to NB (mm), whereas the other values showed moderate-to-poor agreement, with FMA (^) having the worst ICC agreement (ICC = 0.44). In the IHP group, excellent agreement was found for SNB (^), SN-MP (^), U1 to NA (^) and U1 to NA (mm), whereas good agreement was achieved for SNA (^), U1 to SN (^), L1 to NB (^) and L1 to NB (mm). All the other measurements showed moderate-to-poor agreement, with FMA (^), Go-Me (mm), L1 to MP (^), UL to E-line (mm), and LL to E-line (mm) having ICC values <0.50.

DISCUSSION

Gradual refinement of AI-driven automatic cephalometric analysis accuracy has been achieved over the years thanks to the development of increasingly high-performance algorithms.^1,24,25 However, the key question is whether an orthodontist could simply load a lateral cephalogram into a fully automated system and expect reliable and predictable cephalometric analysis without having to intervene. In this context, it is important to assess AI-based system accuracy in common scenarios in which cephalograms are taken with the head slightly rotated or inclined. In this regard, this study tested the accuracy of three fully automated AI-based cephalometric analysis software applications compared to the computer-aided tracing with Dolphin Imaging software, performed using cephalograms with correct and incorrect head posture.

Based on the current findings, the fully automated systems tested exhibited only partial consistency in the CHP group. Linear parameters, such as Go-Me (mm), U1 to NA (mm), and L1 to NB (mm) showed greater variability. In contrast, angular measurements such as SN-MP (^) and SNB (^) showed higher accuracy across all software platforms, likely due to their dependence on the relative positioning of landmarks rather than absolute distances. This finding was consistent with prior research, which also identified linear measurements as more error-prone in fully automated systems.^17,18 Indeed, to bridge the gap between manual and fully automated approaches, software for semi-automated cephalometric analysis^14,16 has been developed. Semi-automatic AI-based software allows human correction of AI-generated landmark identification, with the potential to improve accuracy by leveraging the strengths of human expertise and AI-driven precision.²⁶ Obviously, the trade-off is reduced efficiency compared to a fully automated system.

An intriguing factor influencing the performance of fully automated methods was the position of the cephalometric landmarks. Among the three AI-based systems evaluated, AudaxCeph exhibited the most consistent performance for measurements derived from landmarks located within bony structures, such as SNA (^) and SNB (^) angles, as well as U1 to NA (mm) and U1 to NA (^). This indicated that AudaxCeph’s algorithm excels in identifying internal landmarks with distinct radiographic boundaries. However, its reliability diminished when applied to soft tissue measurements, such as UL to E-line (mm) and LL to E-line (mm), likely due to the challenges of locating landmarks along the external contours of soft tissues. WebCeph and Ceph Assistant exhibited more variable performance. WebCeph achieved good agreement for some parameters like SNB (^), and U1 to SN (^), but struggled with linear measurements, particularly Go-Me (mm), U1 to NA (mm), and L1 to NB (mm) measurements. Ceph Assistant exhibited similar variability, with notable weaknesses in linear measurements like Go-Me (mm). Interestingly, Ceph Assistant performed better for soft tissue measurements, suggesting that its algorithms may prioritize external landmark identification differently. These findings highlight the importance of selecting AI cephalometric software based on the clinician’s specific needs. For applications in which skeletal landmarks are the primary focus, AudaxCeph may provide more consistent results, whereas Ceph Assistant appeared to be more reliable for soft tissue measurements. The variability observed in WebCeph and Ceph Assistant for linear parameters further reinforces the need for manual verification, particularly when assessing critical dimensions such as Go-Me (mm). These insights may help orthodontists integrate AI-assisted cephalometry more effectively into clinical workflows, balancing efficiency with accuracy.

A novel contribution of this study was the evaluation of AI performance in cephalograms with incorrect head posture. Positional errors reduced the landmarking accuracy across most parameters, in particular for linear parameters. The impact of positional errors was most pronounced for Go-Me (mm), which relies on the precise identification of mandibular landmarks. All three systems exhibited poor agreement for this parameter in the IHP group. These results were in agreement with prior literature, which consistently highlighted the challenges associated with the localization of the Go landmark and ruler calibration.¹⁹ In contrast, angular measurements such as SNB (^) and SN-MP (^) obtained good-to-excellent agreement in the IHP, suggesting that angular measurements are more robust to positional variability. These findings are of clinical relevance, as they highlight the limitations of actual, fully automated systems in analyzing cephalograms that may not meet ideal imaging conditions.

Beyond the accuracy factor, one other primary concern related to AI applications in orthodontics is data protection.²⁷ Although online AI-based cephalometric tools offer the possibility of faster and less time-consuming cephalometry, they raise significant concerns regarding patient data privacy and security.²⁸ To comply with regulations such as the GDPR (General Data Protection Regulation)²⁷ in the European Union and HIPAA (Health Insurance Portability and Accountability Act),²⁷ patient data should be anonymized before uploading, by removing both direct and indirect identifiers.²⁸ Additionally, encryption protocols should secure data both in transit and at rest, using advanced standards like the Advanced Encryption Standard with a key length of at least 256 bits (AES-256).²⁹ While waiting for a centralized protocol for data encryption and sharing for AI-based technology, informed consent is crucial to notify patients of risk of data breaches and loss of privacy and their rights regarding data processing and storage.²⁷ Implementing these safeguards can help balance AI’s clinical utility with ethical and legal responsibilities.

CONCLUSIONS

• Fully automated AI-based cephalometric software demonstrated variable accuracy depending on the parameters concerned, with angular measurements showing higher reliability compared to linear parameters.

• This pattern was exacerbated under conditions involving head positional errors in cephalograms.

• Therefore, the observed discrepancies for key measurements indicate that clinician oversight remains essential and that semi-automated systems that allow manual corrections, rather than fully automated systems, should be preferred.