Semi or fully automatic tooth segmentation in CBCT images: a review

Classification and yearly distribution of included articles

The articles included in our review primarily arose from the rapidly evolving landscape of fully automatic and semiautomatic image segmentation algorithms developed in recent years. The methods employed for tooth segmentation can be broadly categorized into two groups: knowledge-based methods and deep learning-based methods.

As illustrated in Fig. 2, studies conducted before 2019 predominantly focus on knowledge-based methodologies, whereas in the post-2019 era, most studies highlight deep learning-based approaches. Notably, the year 2022 had the highest number of relevant articles, underscoring the ongoing exploration on tooth segmentation.

Evaluation metrics

Segmentation outcomes in various studies are evaluated by diverse assessment metrics, broadly categorized into three groups: overlap-based metrics, distance-based metrics, and volume-based metrics. The calculation methods for these metrics are detailed in Table 1.

Overlap-based metrics gauge the dissimilarity in the overlap between automated segmentation outcomes and manual segmentation outcomes, leveraging the confusion matrix. This matrix, presented in tabular format, provides a comprehensive overview of a model's predictions compared to the actual outcomes. The model is divided into four quadrants, defining four distinct outcomes: true positive (TP), false positive (FP), false negative (FN), and true negative (TN). TP denotes the accurate segmentation of foreground pixels, FP indicates the count of pixels erroneously classified as foreground, FN represents the total count of pixels incorrectly classified as background, and TN signifies the correct classification of background pixels (Qiu et al., 2021). These elements facilitate the computation of various secondary indicators, such as accuracy (Ac), precision (Pre), recall (Rec), specificity (Spe), Dice similarity coefficient (Dice), and Jaccard similarity coefficient (Jac), or the intersection over union (IoU).

Distance-based metrics assess the geometric disparities between the segmentation result and the ground truth. In the context of tooth segmentation, commonly used distance-based metrics include the average symmetric surface distance (ASSD), maximum symmetric surface distance (MSSD), and 95% Hausdorff distance (95 HD).

Volume-based metrics concentrate on the volume and shape attributes of the segmented region, encompassing the volume overlap error (VOE) and relative volume difference (RVD). These three metric categories collectively provide an exhaustive evaluation, quantifying the performance of segmentation algorithms from diverse perspectives. These algorithms empower researchers to gain a comprehensive understanding of their accuracy and reliability in medical image analysis.

Result analysis

In the realm of results analysis, Dice, ASSD, and MSSD emerged as the most frequently employed metrics in knowledge-based methodological research. Conversely, in deep learning-based methods, Dice coefficient and IoU/Jac are the most commonly utilized metrics. Nevertheless, due to the diverse range of evaluation metrics employed across various studies for assessing segmentation outcomes, it is challenging to quantitatively evaluate the results of all publications under a unified standard. This article conducts a brief statistical analysis of publications that adopt Dice coefficient as one of the metrics for assessing result accuracy. As depicted in Fig. 3, there was an upward trend in the Dice scores of publications over time, indicating the ongoing advancement in tooth segmentation methods.

The subsequent sections of this article provide a distinct summary and discussion of the included literature on various image segmentation methods. We believe that by elucidating the principles of different methods and presenting their application instances in the specific task of tooth segmentation, readers will gain a thorough understanding of the current status and developmental trajectories of this field.

Knowledge-based methods

Primarily, knowledge-based approaches leverage existing image processing methods, mathematical modeling, and domain-specific expertise to recognize grayscale features in CBCT images, thereby achieving the automatic extraction of tooth structures from the background. These methods can be divided into distinct classes: threshold-based segmentation methods, edge-based segmentation methods, region-based segmentation methods, contour-based segmentation methods and other methods based on specific theories. Table 2 provides a summary of these knowledge-based approaches.

Threshold-based segmentation methods

Threshold-based methods execute image segmentation by categorizing each pixel based on its gray value compared to a predefined threshold. These encompass fixed, histogram, iterative, and adaptive threshold models.

Marin et al. (2015) employ an adaptive threshold model to filter images and identify potential tooth edges. Indraswari et al. (2018) applied a histogram threshold model for grayscale binarization. However, in Indraswari et al. (2018), due to the concentrated gray levels between teeth and alveolar bones, the histogram threshold model was employed in conjunction with other segmentation approaches to achieve a high accuracy of 0.9775.

Edge-based segmentation methods

Edge-based segmentation methods identify disjointed regions in an image by discerning variations in grayscale, color, and texture among different objects. Detecting target image edges is central to accomplishing segmentation tasks. Prominent differential operators for edge detection include the Sobel, Roberts, Prewitt, Canny, and Laplacian operators.

The approach advocated by Pavaloiu et al. (2015a, 2015b) employs Canny operators for edge detection, leveraging anatomical knowledge of teeth to alleviate computational burdens and enhance outcomes. Therefore, as indicated by Pavaloiu et al. (2015b), this approach has achieved an approximately sixfold increase in speed and an approximately 10% improvement in accuracy compared to their previous work.

Region-based segmentation methods

Region-based methods are applied to segment teeth in CBCT images by identifying regions of homogeneity or similarity. These algorithms include seeded region growing, region split and merge and watershed algorithms. Compared to edge-based methods, these approaches place greater emphasis on the overall contextual information within the segmented regions, enabling them to adapt to gradual variations or blurred tooth boundaries. To validate the performance of these methods in different patients, several scholars (Kang et al., 2015; Jiang et al., 2019; Kakehbaraei, Seyedarabi & Zenouz, 2018) started collecting clinical CBCT images (7–10 patients) and conducted quantitative statistical analyses on the segmentation results.

Contour-based segmentation methods

Contour-based methods for tooth segmentation refer to image processing techniques that primarily rely on identifying and delineating the boundaries or contours of teeth in dental images. These methods utilize mathematical curves, such as B-spline curves or active contours, to capture the intricacies and variable shapes of teeth.

Other methods based on specific theories

In addition to the aforementioned methodologies, certain studies explore the application of specific machine learning or image segmentation algorithms to automate the tooth segmentation process.

Summary

Typically, the knowledge-based methodologies explored in this study leverage prior knowledge and heuristic principles, demonstrating a notable level of interpretability in addressing distinctive attributes of dental structures and segmentation outcomes. Nevertheless, since the effectiveness of these methods is heavily contingent upon the accuracy and comprehensiveness of prior knowledge, they are potentially less robust when confronted with complex variations in tooth structures or specific scenarios. Additionally, due to the significant degree of manual intervention, these methods may be susceptible to subjective influences.

Therefore, many researchers have introduced artificial intelligence (AI)-based techniques to achieve fully automatic tooth segmentation. Compared to manual operation and semiautomatic segmentation, these methods intelligently learn features of teeth in CBCT images through end-to-end training on large-scale data. Therefore, they can adapt to more diverse datasets and varied tooth morphologies.

Deep learning-based methods

Machine learning (ML) is a subset of AI which addresses the question of how to build computers that improve automatically through experience (Jordan & Mitchell, 2015). As an advanced type of ML, deep learning (DL) employs artificial neural networks with multiple layers (deep neural networks) to model and process complex data representations. It discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer (LeCun, Bengio & Hinton, 2015). Medical image segmentation primarily involves deep neural network architectures, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and encoder-decoder structures.

CNNs are among the most successful and widely utilized architectures in DL, particularly in computer vision tasks (Minaee et al., 2022). They employ convolutional, nonlinear, and pooling layers to extract features efficiently. Through locally connected units and shared weights within receptive fields, CNNs reduce parameter count while enabling hierarchical feature learning across multiple resolutions (Aloysius & Geetha, 2017).

RNNs are specifically designed for processing sequential data. However, they sometimes struggle with long sequences due to their inability to capture long-term dependencies and susceptibility to gradient vanishing or exploding problems (Minaee et al., 2022). LSTM, a specific type of RNN, addresses these issues by incorporating gates (input gate, output gate, and forget gate) to regulate information flow into and out of memory cells, allowing storage of values over arbitrary time intervals (Byeon et al., 2015).

Encoder-decoder structures are popular in sequence-to-sequence modeling for image-to-image translation, where the output can be an enhanced version of an image or a segmentation map (Cho et al., 2014).

These architectures constitute the fundamental elements of numerous state-of-the-art automatic image segmentation models based on DL. Notably, these models have gained extensive traction in the domain of dentistry, effectively mitigating limitations in knowledge-based tooth segmentation methodologies. Categorically, tooth segmentation models based on DL can be broadly classified into semantic segmentation models and instance segmentation models.

Semantic segmentation models

Semantic segmentation entails partitioning an image into distinct regions based on features such as grayscale, color, spatial texture, and geometric shape (Mo et al., 2022). Table 3 provides a summary of these semantic segmentation models. Consequently, these models specifically achieve pixel-level segmentation, extracting all teeth from both the upper and lower jaws as a unified entity from the background.

Instance segmentation models

Instance segmentation entails discriminating and outlining individual teeth in CBCT images as a distinct instance (Tian et al., 2022). This approach surpasses semantic segmentation by offering a nuanced comprehension of the spatial boundaries and type of each tooth. Table 4 shows a summary of these instance segmentation models. These models can be broadly classified into two groups: two-stage models and one-stage models.

Summary

The outlined deep learning-based approaches overcome the limitations of traditional methods and achieve fully automatic and robust tooth segmentation. The significant advantages of DL in tooth segmentation include precise segmentation, flexibility in handling various imaging challenges, and continuous advancements in model architectures. Collectively, these aspects establish DL as a pivotal tool in the evolution of dental image segmentation methodologies.

Finally, when different DL models are used for automated tooth segmentation tasks, dentists should take into consideration patient needs, possible side effects during dental practice, and the characteristics of the radiographic image dataset.

Challenges and advances

Automating tooth segmentation encounters numerous challenges, such as the intricacy of dental and bone tissues, issues arising from low-quality images, and difficulties in procuring sufficient training datasets.

Research indicates that segmentation inaccuracies predominantly manifest as fine anatomical structures such as cervical margins, pits and fissures (Galibourg et al., 2018) or apical lesions in the context of multiple-rooted teeth (Qian et al., 2021). The presence of surrounding periodontal tissues often obscures the true boundaries of teeth (Mortaheb, Rezaeian & Soltanian-Zadeh, 2013; Ji, Ong & Foong, 2014; Li et al., 2020; Xie, Yang & Chen, 2023), thereby intensifying segmentation complexities. Segmenting extremely small teeth (Ma & Yang, 2019), irregularly arranged teeth (Gan et al., 2017), and impacted third molars (Qian et al., 2021) with diverse eruption positions and shapes (Wu et al., 2020) is challenging. Additionally, the proximity of adjacent teeth (Mortaheb, Rezaeian & Soltanian-Zadeh, 2013; Kakehbaraei, Seyedarabi & Zenouz, 2018; Kakehbaraei, Seyedarabi & Zenouz, 2018; Ma & Yang, 2019) or closely positioned teeth during occlusion (Gan et al., 2017; Kakehbaraei, Seyedarabi & Zenouz, 2018; Qian et al., 2021; Wu et al., 2020) may induce mutual interference, leading to segmentation failure in delineating joint contours.

Several quality parameters of CBCT images, such as low resolution, noise, contrast deficiency (Galibourg et al., 2018; Mortaheb, Rezaeian & Soltanian-Zadeh, 2013), and the presence of metal artifacts (Kang et al., 2015; Gan et al., 2017; Kakehbaraei, Seyedarabi & Zenouz, 2018), negatively impact segmentation outcomes. In response to these constraints, segmentation pipelines undergo continual refinement. During the preprocessing phase, images typically require homogenization and enhancement before automated software-driven segmentation to address artifacts and enhance data consistency (Kakehbaraei, Seyedarabi & Zenouz, 2018; Galibourg et al., 2018; Naumovich, Naumovich & Goncharenko, 2015). Several studies project 3D images into 2D (Galibourg et al., 2018) or regress tooth poses and realign them (Chung et al., 2020; Lee et al., 2022) to mitigate the spatial complexities of CBCT images, thereby contributing to improved segmentation model performance.

In DL-based tools, the efficacy of automatic segmentation models is occasionally constrained by the scale and annotation accuracy of the training dataset. Cui et al. (2022) emphasized the significance of large-scale, multicenter, and real-clinical data over synthesized small-sized datasets. Ayidh Alqahtani et al. (2023) also noted that a large labeled training dataset is essential to avoid overfitting a model, enhance its learning and optimization, and effectively capture the inherent data distributions. Recently, specific data augmentation techniques, such as flipping, rotation, random deformation, and conditional generative models, have been implemented to augment the training dataset (Cui et al., 2022).

Clinical application and ethical responsibilities of AI

The ongoing progress in automatic tooth segmentation technology has significantly streamlined the workflow in clinical digital dentistry, providing substantial benefits for patients and dental professionals. In comparison to traditional manual methods, automatic segmentation technology takes approximately 20–30 s to segment a CBCT scan, whereas manual segmentation requires approximately 3–5 h (Wang et al., 2021). Moreover, even in the presence of metal or other artifacts in CBCT images, the models of Fontenele et al. (2022) and Ayidh Alqahtani et al. (2023) can accurately perform segmentation without relying on additional image processing steps.

However, both dentists and computer programmers engaged in AI should recognize the ethical responsibilities associated with the application of AI in dentistry. This approach is particularly pertinent to issues of patient data privacy, compliance, and underlying errors in AI-driven methodologies. Silva & Soto (2022) emphasize that in the realm of AI-related research and clinical experiments, direct and candid communication with patients to obtain their consent is established prior to data collection. Roy (2022) and Harvey & Gowda (2021) stressed the importance of encryption and access restriction protocols to ensure the confidentiality of imaging data from clinical patients. They believe that this relies heavily on stringent cybersecurity measures, comprehensive regulatory frameworks, and regular auditing mechanisms to mitigate the risk of data breaches. Finally, acknowledging the possible errors in AI-driven approaches, Pesapane et al. (2018) suggested that dental practitioners must actively identify discrepancies while applying AI technology. Researchers, in turn, should conduct regular assessments and update AI systems based on clinical feedback. In summary, these considerations collectively contribute to ethical responsibility in the application and development of AI in dental practices.

Future directions

First, future research endeavors should focus on the continuous optimization of tooth segmentation algorithms to adapt to increasingly diverse datasets and mitigate the impact of factors such as image artifacts and anatomical complexities. During the algorithm optimization process, researchers may contemplate the incorporation of techniques such as adaptive learning (Kerr, 2016) and transfer learning (Weiss, Khoshgoftaar & Wang, 2016) to enhance the algorithm’s performance across various data distributions. Furthermore, for distinct types of dental structures, the use of advanced network architectures such as transformers can be considered (Strudel et al., 2021; Dosovitskiy et al., 2020). Exploring methods that leverage multiscale and contextual information is essential for capturing both detailed anatomical structures and their holistic relationships (Strudel et al., 2021). This integration may entail models with a larger number of parameters and more complex structures, necessitating increased computational resources and training time (Dosovitskiy et al., 2020).

Second, future research endeavors should focus on achieving precise segmentation of each internal structure while segmenting the entire tooth. This includes detailed delineation of the tooth crown and roots and accurate segmentation of structures such as enamel, dentin, dental bone, and root canals. By integrating advanced computer vision and DL technologies, researchers can strive to develop more refined and accurate segmentation models, thereby providing clinicians with more detailed information about dental structures.

Third, for future research on tooth segmentation techniques, segmentation techniques should be integrated with other advanced image processing technologies for a more comprehensive understanding and presentation of dental anatomical structures. For instance, combining 3D imaging (Karatas & Toy, 2014) and virtual reality (Anthes et al., 2016) technologies can offer dental professionals a more innovative 3D display of dental structures, leading to more precise surgical planning and preoperative predictions.

Generally, future research on tooth segmentation should further advance algorithm optimization, detailed structure segmentation, and the integration of multiple technologies to meet increasingly complex clinical demands. This approach will enhance the practicality and accuracy of segmentation techniques in dental practice.