Robust cell nuclei segmentation using statistical modelling

The objective analysis of cytological and histological images has been the subject of research for many years. One of the most difficult fields in histological image analysis is the automated segmentation of tissue-section images. We propose a multistage segmentation method for the isolation of cell nuclei in such images. In the first stage the compact Hough transform (CHT) is used to determine possible locations of the nuclei. We then define a likelihood function which enables us to perform an optimization procedure based on the maximization of this function. Global grey-level histogram information is used thoughout the algorithm to reduce the amount of computation and to reject unwanted artefacts. The algorithm is tested on connective tissue images with very encouraging results. Apart from detecting well-separated nuclei in the images, it performs well in separating dividing nuclei into likely substructures. At the same time the algorithm provides us with a measure of uncertainty along the detected boundary, in the form of the value of the likelihood function.