SEPARATION OF OVERLAPPING OBJECT SEGMENTATION USING LEVEL SET WITH AUTOMATIC INITALIZATION ON DENTAL PANORAMIC RADIOGRAPH

To extract features on dental objects, it is necessary to segment the teeth. Segmentation is separating between the teeth (objects) with another part than teeth (background). The process of segmenting individual teeth has done a lot of the recently research and obtained good results. However, when faced with overlapping teeth, this is quite challenging. Overlapping tooth segmentation using the latest algorithm produces an object that should be segmented into two objects, instantly becoming one object. This is due to the overlapping between two teeth. To separate overlapping teeth, it is necessary to extract the overlapping object first. Level set method is widely used to segment overlap objects, but it has a limitation that needs to define the initial level set method manually by the user. In this study, an automatic initialization strategy is proposed for the level set method to segment overlapping teeth using hierarchical cluster analysis on dental panoramic radiographs images. The proposed strategy was able to initialize overlapping objects properly with accuracy of 73%. Evaluation to measure quality of segmentation result are using misscassification error (ME) and relative foreground area error (RAE). ME and RAE were calculated based on the average results of individual tooth segmentation and obtain 16.41% and 52.14%, respectively. This proposed strategy are expected to be able to help separate the overlapping teeth for human age estimation through dental images in forensic odontology.


Introduction
Dental panoramic radiograph is used widely in the field of odontology to analyse and diagnose the tooth parts that are not visible directly to the eye. A dental panoramic radiograph can transfer useful information with extracting the features from an image. One of step to extract the information from the image is image segmentation [1]. Image segmentation is done to distinguish between foreground in this case is a dental object, with the background in this case is a part other than teeth [2]. The segmentation process of dental panoramic radiograph has been carried out and get good 26 Jurnal Ilmu Komputer dan Informasi (Journal of Computer Science and Information), volume 13, issue 1, February 2020 results with various algorithms when applied to a single tooth image [3]- [5]. But this becomes a problem when faced with overlapping teeth [6].
In the dental panoramic radiograph segmentation, both overlapping teeth are segmented into one object. This is a problem when the result of the segmentation process itself is expected to be a single tooth. The problem caused by overlapping teeth during segmentation is when it faced to age assessment through teeth on dental panoramic radiograph, one of the difficulties in assessment is the presence of overlapping teeth [7].
Overlapping objects become a problem in image segmentation. The Chan-Vese segmentation method results only consist of the general outline area of the target object, but there are some false contours produced by overlapped object [8].
Overlapping objects also cause the process of interpreting an object to be more difficult [9]. To separate overlapping objects, it is necessary to extract the overlap area first [10]. Overlap objects can be extracted based on the intensity and geometric features they have. Success in segmentation of overlapping objects is very dependent on the type of image data used. Because the features that can be utilized is different from each various image. Research [8] utilizes the green channel intensity feature in RGB images to segment overlapping leaves because the leaves have green intensity features. Lu's research [11] segment the overlapping cervical cells using level sets. Segmentation using the level set method requires initials as a zero-level set to trace the edge of the desired object. Geometric features of cervical cell images have a nucleus that is always in the center of the cell object. Assuming each cell has its own nucleus even in overlapping position. Then by using the detected nucleus can be used as initials to do segment using level sets. However, in dental panoramic radiographs, it does not have special feature marks for initial level set segmentation. This can be solved by marking manually overlapping objects. But this is quite exhausting if it applied in a lot of data. The observer's objectivity is also crucial. So, we need a strategy to determine initialization of overlap objects automatically This research proposes an automatic initialization strategy for the level set method to segment overlapping teeth using Hierarchical Cluster Analysis on dental panoramic radiographs images. To obtain an initial zero level set, it can be facilitated by the intensity and geometry features of the dental panoramic radiograph. Overlapping objects in dental images have features of higher grayscale intensity than another object. This feature can be extracted by image binarization using HCA (hierarchical cluster analysis) multi-thresholding [12]. Geometric features of overlap objects in the dental image have a vertical orientation obtained by selecting the region and orientation of the overlap. The data used in this research is ROI (region of interest) obtained by manual cropping from a dental panoramic radiograph. Test evaluation uses relative foreground area error (RAE) and Misclassification Error (ME) to find out how well the results of segmentation compare to ground truth. The results of this study are expected to be able to determine the initial zero level set on overlapping objects appropriately so that they can segment using the level set properly.

Related Work
Some research with panoramic radiographic image data, the presence of overlapping teeth is a major challenge in getting a single tooth for the classification process [13]. There are several research of image segmentations with overlapping objects but not in dental panoramic radiograph. The Yadollahi's research separates overlapping dental objects using the normal vector principle to detect the intersection of adjacent teeth edges in the gypsum image of the dental arch component [14].But in their research, overlapping is meant by teeth that are attached to each other on the tooth gypsum image, not overlapping objects in dental panoramic radiograph. So their method cannot applied in our research.
Mahyari's research [15] segment the object containing translucent overlapped regions in synthetic images using random walker algorithm. They segment the translucent overlapping object based on random walker method. The Laplacian matrix are generated from multi-layer graph from a translucent overlapped input image in 2 dimensions. A small number of user-defined labels (or seeds) are provided by the new Laplacian matrix. But overlapping objects in this study have irregular shapes. Different from the overlap object characteristics in dental panoramic radiograph which has a vertically oriented shape. So, this method cannot be applied to our research.
Arifin's research conducted the detection of overlapping teeth as a first step to separating the two overlapping teeth [6]. Detection is done utilizing the intensity features of overlap objects by performing adapted automatic thresholding to get markers on the location of overlapping objects. However, the adapted thresholding method used is not robust to some images. To overcome this problem, the geometry features of the overlap object are used to get the candidate object overlap. selection is carried out on several candidate objects namely orientation and neighbourhood selection.
This idea motivated us to use the intensity and geometry features of overlapping objects to get overlapping objects in dental panoramic radiograph.
A joint level set optimization is used to segment all detected nuclei and cytoplasm pairs [16]. The cells as input image are overlapping at clump region. the area and length of each cell, previous cell shape, several overlapping cell and grayscale value within the overlapping region are constraining the optimisation of level set. The initial zero level set is obtained from detected nuclei. This idea motivated us to use the level set method to segment overlapping objects on dental panoramic radiographs. However, the determination of the initial zero level set in our study needs to be adjusted to the characteristics of the overlap objects of the image we use.

Proposed Method
The proposed strategy consists of several steps, namely: (1) defining the region of interest (ROI) (2) preprocessing (3) overlap teeth segmentation (4) overlap object segmentation (5) separation of overlapping teeth. All stages can be seen in Figure  1 and will be discussed each step.

Defining Region of Interest (ROI)
The data used in this research is dental panoramic radiographs provided by Rumah Sakit Gigi dan Mulut, Universitas Airlangga (RSGM UNAIR) Hospital. All images in individual patients aged 16 to 70 years with male and female genders. Panoramic images of the teeth that have been collected will be observed and selected ROI based on the requirements according to figure 2. ROI criteria used are: (1) There are 2 teeth that overlap partially with the overlapping area on the enamel, (2) The overlap teeth indicate all parts of the teeth from the crown to the root of the tooth, (3) overlap area objects have a vertical orientation, (4) No ghost image, (5) No teeth are conserved or restored.

Preprocessing
At the preprocessing stage there are 2 steps, namely image enhancement and image rotation. The input from pre-processing is ROI of the dental panoramic image and the expected output is ROI'. ROI' is ROI that has pass through the process of image enhancement and rotation that can be seen in Figure 3. Image enhancement is important because the image on ROI that is cropped directly from the original image has low contrast, nonuniform lighting, presence of noise, and unclear edge. To overcome noisy image, the image was improved using Median filtering. Median filtering operations are known to be often used for the purpose of denoising and smoothing the image [17]. To solve the non-uniform lighting on ROI, improvements are made using CLAHE (Contrast Limited Adequate Histogram Equalization).
ROI rotation is important because it make overlap object in a perfect vertical position. Rotating image requires rotating angle input. To determine the angle of rotation can automatically  utilize the gap between the upper jaw and the lower jaw. The gap between the upper jaw and the lower jaw has the lowest intensity so that by performing image binarization using Otsu thresholding [18]. This gap can be extracted by taking pixels that are 0 value (black in color) and have the highest number of pixels or the widest area. To get the orientation of the dental cavity object, orientation checks are performed with creating an elliptical object that surrounds all the pixels in the object illustrated in Figure 4a. Figure 4a shows the corresponding object and ellipse area. Figure 4b shows the same ellipse then formed a solid blue line representing the axis. The blue dot at the end of the blue line shows focus. The orientation is the angle between the horizontal dashed line and the main axis. The angle obtained will be used as input for ROI rotation. Output image from preprocessing stage are illustrated in figure 5. Image that has been rotated produces a triangles shaped region with 0 pixels value (black) in each corner of the image. Then the unnecessary pixels are cut. Determination of windows cropping is obtained from the pixel coordinates in the corner of the image on each axis.

Overlap Object Segmentation
The overlap object segmentation is done using the level set segmentation method. To do this, initialization is required to trace the contour of the desired object. In this case the desired object is the overlap area. To identify overlap parts, it is necessary to know the special characteristics of overlap objects on the dental panoramic radiograph. Overlap object characteristics are having high grayscale intensity (brightest) from other parts. The part of normal teeth (without overlapping) that has the highest brightness of all parts of the tooth is enamel [19]. The overlapping object is formed by overlapping parts of the enamel. Then the overlap part has a higher brightness or equal to enamel. So, to get the overlap object we need a thresholding method that can form the Threshold at the top of the second peak on the histogram.
Hierarchical Cluster Analysis (HCA) method has the advantage able to do multithresholding. HCA method uses hierarchical clustering algorithm to clustering the pixels based on grayscale dendrogram in image histogram. To get overlapping objects, the histogram is divided into 4 clusters using the HCA algorithm, which means it has 3 thresholds. We divided a histogram into 4 clusters or 3 thresholds because based on observations by plotting histograms on an ROI sample of overlap teeth, obtained 3 modals, each modal representing an object in an image sample as shown in figure 6a while plotting histograms can be seen in figure 6b. In the figure 6b, the first modal represents the background, the second modal represents the dental object, and the third modal represents the overlapping object. So to get the overlap object, it needs to get the 3rd threshold using thresholding method that can find the threshold in the third modal. To find the threshold as desired, this research uses a multi thresholding Hierarchical Cluster Analysis (HCA) method. By selecting the third threshold, the overlapping object can be extracted as illustrated in Figure 6. After extracting the overlap object, morphological operations are performed to ensure that the overlap object is separated from the other parts. Morphological operations used are erosion operations that make objects eroded. Erosion operations are carried out using structuring element (strel) 3×3 rectangular shaped which is the most commonly used type. The output from erosion operations is not enough to find the location of overlapping objects. To make it easier to find overlapping objects, an area selection and orientation are needed. Area selection is done to get several large objects and eliminate small objects. While the orientation selection is done to ensure that overlap objects are in accordance with its geometric features that is vertically oriented. Area selection is done by counting the number of pixels in an object and sorted descending and then taking the 5 biggest objects. Orientation selection is done by measuring the slope of the overlap object using the same method by getting the angle of rotation described earlier. But the slope angle output determines the orientation of the object. The object is selected as vertical with the condition that it has a degree of slope < −° or > ° [6]. Objects that out of conditions will be eliminated. So that the result of orientation selection leaves only one object, the overlap object. The remaining objects will be taken centroids coordinates as initial zerolevel sets. The output of each selection can be seen in figure 7.
From the initial coordinates obtained, then trace the edge using a level set function. Input from this process are initial coordinates and ROI'. The level set function method in this study adapted research from Li namely DRLSE (Distance Regularized Level Set Evolution) [20]. The DRLSE method requires initialization manually, so automation is carried out in our research. The initialisation formed from the previous process is used as the initial to do curve evolution which can be expressed as: where F is the velocity function that controls the motion of the contour, and is a normal vector into the curve . The evolution of the curve in Equation 1 in the context of parameters can be converted to a set level formulation by embedding a dynamic contour ( , ) as zero level set depends on the time of LSF ( , , ). Assuming that the insertion of LSF takes a negative value inside the zero-level contour and a positive value outside, the normal vector inside can be expressed as = − | |, where is the gradient operator. Evolution will stop when the curve is in an area with a high gradient value which means the curve is already on the edge of the object. Initial coordinates and final curve evolution can be seen at figure 8.

Overlap Teeth Segmentation
The target objects focused in this study are two overlapping teeth and the overlapping object itself. To shorten and simplify the segmentation of the two objects, a background removal is carried out. Based on previous research on overlapping object segmentation, the first step is background removal [8], [10], [11]. The segmentation process is not interrupted with other object if background removal is done. All overlap teeth segmentation process can be seen in Figure 9. Background removal is using image segmentation method. Because both background removal and image segmentation are having same goal. In this research using thresholding method that carryout by Otsu [18]. This method can find the threshold based on image histogram to separate object and background automatically. Output of the Otsu thresholding segmentation result is binary image. The binary image result still contains small object that are considered as noise. Post-processing is done to erase small object in the binary image using morphology opening operation to eliminate small objects. Next post processing is selection by area where only the largest objects are taken. The largest area is the highest number of pixels among several objects. This is done assuming that the largest object as teeth object. The segmentation result can be seen at Figure 10.  After both overlap object and teeth object are segmented, next step is separating two overlapping teeth. To separate both, need to be extracting the edge line of the object. Edge line are the pixels that constantly arranged around the object. The distance of pixel in the both object edge line (overlap object and teeth) are calculated to find the adjacent pixel. The nearest pixel in showed by the smallest distance. The distance of the pixel is calculated using Euclidean distance measurement. Both pixels that have smallest distance are set as intersection point as Figure 11. There is two intersection point in one image, that is upper and bottom and then both intersection point are connected by line equation through two points − / − = − / − . This line separating the overlapping teeth become two individual teeth as illustrated in Figure 11a. After obtaining two individual teeth, then the individual teeth object is merged with overlapping objects using OR operation as Figure 11d. So we get the results of overlapping tooth segmentation individually as shown in the Figure 12b and 12c.

Experiment and Analysis
This research using dental panoramic radiograph images provided by Rumah Sakit Gigi dan Mulut, Universitas Airlangga (RSGM UNAIR) Hospital [2]. All proposed strategy in this research all coded on mathematical computing software, Matlab. 30 images are used, all of which are the ROI defined above as seen at Figure 13. To measure the performance of the proposed strategy, an evaluation is needed. Evaluation is carried out in 2 scenarios, the first of which is to measure how well the proposed method strategy is to initialize. In other words, how well the proposed strategy for finding overlapping object locations. Accuracy is used to measure the first scenario. Accuracy in this case is calculated from the total number of correct prediction classes divided by the number of image data according to equation (2).
The second scenario is to measure how well the proposed method to separate the overlapping teeth as individual. Misclassification errors (ME) and Relative foreground area errors (RAE) are used in this scenario. ME is calculates the pixel ratio of the wrong object classified as background, and vice versa according to equation (3).
ME value is obtained from a reduction of 1 to the number of object pixels that intersect between ground truth pixels and segmentation results plus the number of background pixels that are slices between ground truth pixels and segmentation results. The sum of them are divided by the absolute sum of and . ME values are in the range of 0 to 1 with evaluation indicators the smaller the ME value the closer the ground truth, the better the performance of the proposed segmentation method.
The RAE evaluation method measures the ratio of the difference between object areas in ground truth and segmentation results. In evaluating the results of segmentation, the RAE method only calculates the ratio of differences in RAE value is calculated from the reduction of the number of pixels on the ground truth object and the results of segmentation divided by the number of pixels ground truth if the value of is smaller than . However, if the number of pixels is more than equal to , then vice versa. The range of RAE values is between 0 and 1.Where the evaluation indicator is getting smaller the RAE value, the area of the object segmentation results is increasingly similar to the object area in the ground truth image so that the performance of the proposed segmentation method is getting better and vice versa.

Evaluation of automatic initialization
Accuracy evaluates the correct amount of data in initiating. The data is classified as TRUE when the automatic initialization done by the system is precisely in the overlap object. The amount of data used in this research were 30 images. The sample data used can be seen in Figure 12. The experiment carried out successfully initialized 22 data exactly. So that an accuracy obtained is 73%. This achievement can be said successful in initializing the overlap object.
The main challenge when obtaining initial point is that some overlapping objects are not fully disconnected with other objects. This causes overlap objects to be detected as horizontal objects during orientation selection. This is caused by the HCA algorithm cannot optimally get the 3rd threshold. This problem can be solved by erosion morphological operation. But for some images still cannot disconnecting overlapping objects that are connected with other objects. Another object that is most often connected with overlapping objects is the enamel teeth component. The example of disconnected overlap object can be seen in figure  14. The teeth component enamel has a grayscale intensity that is similar to the overlap object. So, the HCA algorithm cannot handle this yet. Another problem that is quite challenging is that the overlap object is excluded when selecting areas. Some nonoverlapping objects have the same characteristics as overlap objects. But does not represent the location of the overlap object. When area selection, the overlap object does not become the largest object because it loses from the other object. So that the system is wrong in determining the initial point This research using HCA in determining threshold to get overlapping objects. Incorrect threshold result in overlapping objects are not segmented properly as previously explained. Therefore, the thresholding method greatly influences the final determination of the initial point. In this study we compare the method we propose with 2 comparative thresholding methods, namely the Gaussian Mixture Model (GMM) [21] and Modified Otsu's thresholding [6]. GMM method was chosen as a comparison because of its ability to do multi-thresholding the same as the HCA method. while the modified Otsu's method was chosen because the algorithm has been modified so that it can segmenting overlap objects. GMM models the histogram of the grayscale intensity of an image as a combination of several gaussian distributions. each gaussian distribution will be updated on mean and standard deviations to maximize the similar grayscale set for each gaussian distribution [21]. the result of GMM is a gaussian distribution of k which represents the distribution of grayscale intensities in the histogram, with (k-1) threshold being able to separate each gaussian distribution. While the modified Otsu performs a thresholding by modifying the Otsu algorithm. Modification is done by dividing the x-axis on the histogram into 4 divisions. Based on observations, overlapping objects will be segmented when the threshold is on the second peak slope on the histogram [6]. Therefore, the calculation of class variance and total variance is only calculated from division 4, which is grey level 192-255.
The result of the threshold comparison formed from each method can be seen in Figure 15. From the sample image in Figure 15a, then the histogram in Figure 15b. The GMM method produces a threshold of 147 which is indicated by the blue line. The modified Otsu method produces a threshold of 194 with yellow line. While the proposed strategy produces a threshold of 168 with Figure. 14. Disconnected overlap object sample

Evaluation of teeth segmentation result
The second scenario is the evaluation of overlap separation segmentation results. The extracted individual teeth are important for future research in odontology forensic for human age estimation through dental images. Then the evaluation of the system is needed to find out how well the proposed strategy works. The evaluation method used is misclassification error (ME) and Relative foreground area error (RAE) as explained above.
The experiment is carried out only on data that has been successfully initialized correctly in the overlap object. So, the object being evaluated is really an overlap object and unsuccessful data will be ignored. Based on the test results, the proposed system was successful in extracting 22 of 30. So, the evaluation of segmentation results was only carried out on 22 data. The average ME value obtained is 16.41% while RAE value 52.14%. ME values obtained indicate the results of segmentation are almost similar to ground truth because ME evaluation calculates the pixel difference ratio of objects that are incorrectly classified as background and vice versa. The sample of input image and overlapping teeth separation result shown in Figure 16.
The challenge when segmenting overlap objects are that overlap objects are often oversegmentation. This is because the segmentation method used is the level set methods that are very sensitive to gradation changes. The level set method performs segmentation by tracking the object's edge. So, for images that have bad edge, will also worsen the results of segmentation. Overlap object also affect to determination of intersection point because the nearest pixel are obtained from the pixel of overlap object edge line. Another challenge that affect the segmentation result is the process of overlapping teeth segmentation. Overlapping teeth segmentation in this research are using Otsu's thresholding method. This method is not work well with any medical images. Because medical image especially x-ray based, have a low contrast that difficult to find the accurate threshold to separate object and background using Otsu's method. These challenges contribute greatly to the evaluation results of the overlapping teeth segmentation.

Conclusion
Separation of overlapping teeth segmentation on dental panoramic radiograph has been presented. Thresholding using HCA in overlap object segmentation can determine the centroid coordinates that will be used as zero-level set in the level set method. Erosion morphology operations can be used to separate overlapping objects with other objects connected with overlapping objects. Area and orientation selection can be relied upon when selecting candidates to ensure overlapping objects and eliminating other objects except overlapping objects that can resulting wrong initiation. The level set method combined with the automatic initiation strategy can segment overlap objects properly. Proper initiation largely determines the success of the level set method in curve evolution. Conversely, improper initialization will be resulting wrong determination of the target object that causes the level set method to segment target object wrongly.
Based on experiments result, it can be concluded that the initiation strategy using HCA is better in determining initialization than using the modified Otsu and Gaussian Mixture Model methods. The combined strategy which consists in determining the intersection point and algebraic operation of the equation of a line passing through two points is successful to separating the individual teeth segmentation on dental panoramic radiograph. For future work, the method for background removal or segmentation of overlapping teeth can be improved considering medical image have a specific problem to be segmented.