MULTI-CLASS REGION MERGING FOR INTERACTIVE IMAGE SEGMENTATION USING HIERARCHICAL CLUSTERING ANALYSIS

In interactive image segmentation, distance calculation between regions and sequence of region merging is being an important thing that needs to be considered to obtain accurate segmentation results. Region merging without regard to label in Hierarchical Clustering Analysis causes the possibility of two different labels merged into a cluster and resulting errors in segmentation. This study proposes a new multi-class region merging strategy for interactive image segmentation using the Hierarchical Clustering Analysis. Marking is given to regions that are considered as objects and background, which are then referred as classes. A different label for each class is given to prevent any classes with different label merged into a cluster. Based on experiment, the mean value of ME and RAE for the results of segmentation using the proposed method are 0.035 and 0.083, respectively. Experimental results show that giving the label on each class is effectively used in multi-class region merging.


Introduction
Image segmentation can be defined as the process of separating digital images into several parts or series of pixels [1].Segmentation is the first step in the image analysis process that aims to represent the image into something that is easy to analyze and more meaningful.The segmentation process itself is done by separating objects from the background.In general, there are three ways in the segmentation process, including automatic, semi-automatic, and manual.In some low-contrast medical images such as images of Dental Cone Beam CT and Dental Panoramic Radiographs, automatic segmentation is quite a complicated thing to do.The use of semi-automatic segmentation by providing additional information from users can be a solution to improve the quality of segmentation results.Semi-automatic segmentation is commonly referred to as interactive segmentation.
On the separation of an image into an object and background, the calculation of the distance between regions and the sequence of regions of merging is an important thing that needs to be considered to obtain accurate segmentation results.In cluster analysis, an object is arranged in such a way that objects in a cluster have a high 114 Jurnal Ilmu Komputer dan Informasi (Journal of Computer Science and Information), volume 12, issue 2, June 2019 degree of similarity, and they are different from objects, which are located outside the cluster [2].In general, the techniques which are used in the clustering process can be classified into several categories: Partitioning Clustering, Hierarchical Clustering, Density-Based Clustering, Grid-Based Clustering, Model-Based Clustering, etc. [3].The main method which is discussed in this paper is Hierarchical Clustering where a number of formed clusters are mapped in the form of trees [2].The process of Hierarchical Clustering is done by calculating the distance between sub clusters.Sub-clusters with the closest distance will be combined into one cluster or group.The distance between sub-clusters can be done with various approaches such as Single Linkage, Complete Linkage, or Average Linkage [2].Single Linkage focuses on the smallest distance or the closest neighbour between sub-clusters.Complete Linkage takes the largest distance or the farthest neighbour between sub-clusters [4], while Average Linkage takes the average distance between sub-clusters.
Previous research [5] conducted a hierarchical tree-based image segmentation.This study uses a tree structure to describe hierarchies of region merging in supervised image segmentation.Segmentation is done by combining regions that are considered to have proximity to the desired number of final classes.This method can separate objects based on the desired number of classes but does not pay attention to the labels in each class.
The Hierarchical Clustering Analysis method is only able to combine data without regard to labels.In some data with low contrast levels, it is possible to combine two different labels into one cluster, so that it can have a wrong result in class selection.
This study proposes a new multi-class region merging strategy for interactive image segmentation using the Hierarchical Clustering Analysis method.Some approaches will be compared such as Single Linkage, Complete Linkage, and Average Linkage.The data used in testing include the image of Dental Cone Beam CT, Dental Panoramic Radiographs, and natural imagery.Testing the results of segmentation on each data is done by using Misclassification Error (ME) and Relative Foreground Area Error (RAE).This research is expected to be able to guarantee the accuracy of segmentation results by separating each class with different labels.

Related Work
Interactive image segmentation is still an active research topic in image processing.In the last decade, interactive image segmentation received a lot of attention, and several methods have been proposed to interactively segment images.The most widely used interactive image segmentation method is to assume that the initial label has been correctly placed before segmenting the regions.Labels that are not accurate even with small regions will result in inappropriate segmentation.
In research [6] and [7] applied Markov to segment the seeded image where research [6] combines the subMarkov method and random walk by changing the auxiliary node to facilitate the transformation and searching process between bridges that are built.Whereas research [7] combines pixels and super pixels with multi-label to overcome the sensitivity of image quantity and quality.The Markov random field framework is used to combine pixel layers, super pixels, and labels in order to increase the accuracy of segmentation.In addition, parallel partial optimization strategies are used to improve efficiency.However, the parameter value is very influential, and it has the disadvantage of adjusting these values manually.
Graph-based optimization techniques are also used in image segmentation [8].Structureaware labelling is done by using probability occurrence and co-occurrence (OCP) of the value of each colour on each label.OCP can enable robust segmentation of images for labels that are not given accurately and address small-cut issues.However, this approach cannot overcome cases that have an initial label with almost the same colour distribution.Therefore, this method can assume a small number of labels that are correctly placed in different regions, and they become outliers and eliminate them.
The techniques for image segmentation carried out in previous studies can be categorized as two types: region-based models and edge-based models.The region-based model aims to identify each region by using a specific region explanatory method as a contour guide, while the edge-based model uses edge information for image segmentation [9].Region-based segmentation methods are categorized into three main categories, namely region growing, region splitting, and region merging [10].Research [5] proposed a hierarchical approach supervised for image segmentation.The approach begins with over segmentation on super pixel, using a tree structure to represent hierarchies from the region merging so that it can reduce the problem of finding labels on nodes in trees.Calculation of boundary classifier is used to combine regions.The advantage of this method is that the desired number of clusters can be determined and does not require adjusting the parameter values for classification, but the division of regions resulting from segmentation without any information about the cluster is divided into less helpful for analysis of segmentation results.In addition, the training process is also needed repeatedly so that regions are properly combined.
Research [11] is a development of research [5] by proposing a semi-supervised approach to reduce ground truth data requirements in the region merging process.The term unsupervised loss was developed to enforce consistent predictions of the learning function.Then, supervised and unsupervised information in probabilistic learning is combined with the Bayesian model.The experimental results show that by using a subset of 3-7% of all ground truth data, this approach works almost the same as the supervised method with a complete labelled data set.

Proposed Method
Image segmentation is a process for separating images into a number of objects and backgrounds.Interactive segmentation is a segmentation method that combines two other segmentation methods, namely: automatic segmentation (automatic segmentation by the system) and manual segmentation (manual segmentation done by the user).This study proposes semi-automatic segmentation in multi-class regions merging using Hierarchical Clustering Analysis.Semi-automatic segmentation used for this study consists of three main processes, namely: initial image segmentation, image marking, and region merging using Hierarchical Clustering.The main stages in this study are shown in Figure 1.

Initial Image Segmentation
The initial image segmentation aims to divide the image into a number of small regions [12].This initial segmentation uses a mean-shift algorithm which in the process uses the meanshift segmentation software which is provided by the Edison System [13].

Image Marking
Image marking is done to give markings to regions that represent each class that will be segmented.The results of marking will divide the image into several classes, namely non-marked classes of objects, background, and regions.Giving marking to object 1 will be entered into the  1 class, giving marking to object 2 will be entered into the  2 class, and so on until the desired number of -classes of objects.Regions marked as the background will be included in class , while regions that do not enter any class (non-marked) will be included in class .So that in one image, it will be formed several classes as in Equation 1, Equation 2, and Equation 3.  is a class that represents n-objects with each object storing  regions. represents the background class with the number of regions as many as , while regions with a number of  that are not classified in any class will be stored in class .So that in one image it will be divided into -marked objects, backgrounds, and regions as in Equation 4.

Multi-Class Region Merging with Hierarchical Clustering
The region merging stage aims to combine non-marked regions into classes that have been defined.Multi-Class region merging in this study is done by labeling each class to prevent the merging of two different classes into one cluster.Overall, the multi-class region merging process flow is illustrated in Figure 2.
The initial step in the multi-class region merging process labels each class based on the marking results of the user.Labels will be given to each non-marked region, object classes, and background classes.non-marked region with other non-marked regions, non-marked regions with each class, and the distance among classes will be calculated using inter and intra-class.The class or region with the smallest distance (i, j) will be combined into one cluster with the terms i and j are not belonged to the object class, or i and j are not the object class and background.If the conditions are not fulfilled, then the merger will be carried out in the region or class with the closest distance that meets the requirements.

Figure. 2. Flowchart Region Merging
Calculation of distances between nonmarked regions with other non-marked regions, non-marked regions with each class, and the distance between classes will be recalculated according to the region or class that has been combined.The process will be repeated until all non-marked regions join the class and only remain n-class objects and one background class.
Distance measurement in this study uses inter-class and intra-class variance [1].Inter-class variance is used to calculate variances between regions in the image.Inter-class variance measurement begins with the division of the ratio results from cluster  (  ) and the ratio of cluster  (  ) to the sum of ratios of cluster  (  ) and the ratio of cluster  (  ).Furthermore, the result of the division is multiplied by the result of the square of the reduction of mean in cluster  (  ) and mean in cluster  (  ).The calculation is repeated as long as  is less than the number of classes  up to the number of classes to add the results.The inter-class variance for a number of  cluster can be formulated in Equation 5. (5) In addition to the inter-class variance, the merging region stage also uses intra-class variance.Intra-class variance is a variant calculation among fellow regions in a cluster.Intra-class variance calculations are carried out by dividing the square of the subtraction  and mean of cluster  (  ) by the ratio of the cluster ( 1 ).The variants of each cluster are calculated using Equation 6 with  being the index of the cluster.
Based on Equation 6, the intra-class variance value for  clusters is obtained by summing the multiplication of variants (  2 ) and ratio (  ) of each cluster (), as shown in Equation 7.

Analysis and Result
This study uses a data set of grayscale images in the form of Cone Beam CT dental images [1], ROI of dental panoramic images [14], and natural images [15].
Based on Equation 8,   and   are pixels of objects and background from segmentation image results.  and   are pixels of objects and background of ground truth images.Then Equation 9shows the RAE calculation where   is the object area of the ground truth image.Whereas,   is the object area of the segmented image.The comparison of segmentation results among single linkage, complete linkage, and average linkage in the image of Dental Cone Beam CT are shown in Table 1, the results of dental Panoramic image segmentation are shown in Table 2, while for natural images are shown in Table 3.
The evaluation results of the Dental Cone Beam CT images in Table 1 show that four of the five images have the lowest ME and RAE values on the Average Linkage.The results of segmentation using the Complete Linkage in dental panoramic images give the best results for the entire image.Whereas in natural images, three of the five images give the best results using the Complete Linkage.From these results, it can be concluded that using the segmentation in the image of the Dental Cone Beam CT, Average Linkage can be more effectively separate between classes in the image, while the Complete Linkage is suitable for use in dental panoramic images and natural images.The comparison of segmentation results between the proposed method and the Hierarchical Clustering method without multiclass was conducted to determine the effectiveness of the proposed region merging strategy in segmenting.The results of ME and RAE value between previous research [1] and proposed method are shown in Table 4.
The ME and RAE mean values which are obtained in the overall image segmentation using the proposed method are 0.035 and 0.083, respectively.Whereas ME and RAE only use the Hierarchical Clustering Analysis method, respectively 0.06 and 0.177.Based on the test results, the average ME and RAE values using the proposed method have a lower value.The lower ME and RAE values of image segmentation, it shows that is more closer the image with the ground truth.Those values show that the results of segmentation using the proposed method have better quality than only using the Hierarchical Clustering Analysis.The results of segmentation in each image can be seen in Table 5, Table 6, and Table 7.
The evaluation results show that the proposed method has better quality segmentation results compared to the previous research.The addition of labels in combining regions can improve the results of segmentation in quantitative.This is due to the prevention of two classes with different labels joining into one cluster.
However, the proposed method still relies heavily on image marking.Giving marking to each class greatly impacts the results of segmentation.The better of image marking, the results of segmentation will be better.Meanwhile, giving inappropriate marking will reduce the results of segmentation quality.Marking stage is a mandatory step to do in segmentation process.If only mark the image with a pixel, there will be a difference in the segmentation results.The difference in ME and RAE from the use of marking is shown in  Based on Table 8, the table shows the comparison of ME and RAE results between images with marking in proposed method and images that use one pixel marking.ME and RAE results from three types of images show that images with marking in proposed method have higher values than those images that only use one pixel in marking.

Conclusion
This study proposes a multi-class region merging in interactive image segmentation using the Hierarchical Clustering Analysis method.Evaluations were carried out on three types of images, namely Dental Cone Beam CT, Dental Panoramic Radiographs, and natural images.
Based on the evaluation results, the ME and RAE mean values of the entire images using the proposed method are 0.035 and 0.083, respectively.The experimental results show that the proposed method has better quality segmentation results.The labeling of each class to prevent the existence of two classes with different labels joining into one cluster proved to be effective in increasing the results of segmentation in the multi-class region merging.However, the proposed method still relies heavily on image marking.Therefore accurate automatic marking needs to be done in future studies to improve the results of segmentation quality.

116 Jurnal Ilmu Komputer dan Informasi (Journal of Computer Science and Information), volume
The distance between one 12, issue 2, June 2019 the difference in area between objects, which are contained in the ground truth image and segmented image.Equations 8 and 9 below show the ME and RAE formulas respectively. measure

TABLE 4 COMPARISON
OF ME AND RAE BETWEEN PROPOSED METHOD AND HIERARCHICAL CLUSTERING METHOD WITHOUT MULTI-CLASS

TABLE 8 COMPARISON
OF ME AND RAE BETWEEN PROPOSED METHOD AND HIERARCHICAL CLUSTERING METHOD WITH A PIXEL MARKING