Publications by authors named "Ashnil Kumar"

29 Publications

  • Page 1 of 1

Predicting distant metastases in soft-tissue sarcomas from PET-CT scans using constrained hierarchical multi-modality feature learning.

Phys Med Biol 2021 Nov 24. Epub 2021 Nov 24.

The University of Sydney, Sydney, 2006, AUSTRALIA.

Objective: Positron emission tomography-computed tomography (PET-CT) is regarded as the imaging modality of choice for the management of soft-tissue sarcomas (STSs). Distant metastases (DM) are the leading cause of death in STS patients and early detection is important to effectively manage tumors with surgery, radiotherapy and chemotherapy. In this study, we aim to early detect DM in patients with STS using their PET-CT data.

Approach: We derive a new convolutional neural network (CNN) method for early DM detection. The novelty of our method is the introduction of a constrained hierarchical multi-modality feature learning approach to integrate functional imaging (PET) features with anatomical imaging (CT) features. In addition, we removed the reliance on manual input, e.g., tumor delineation, for extracting imaging features.

Main Results: Our experimental results on a well-established benchmark PET-CT dataset show that our method achieved the highest accuracy (0.896) and AUC (0.903) scores when compared to the state-of-the-art methods (unpaired student's t-test p-value < 0.05).

Significance: Our method could be an effective and supportive tool to aid physicians in tumor quantification and in identifying image biomarkers for cancer treatment.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1088/1361-6560/ac3d17DOI Listing
November 2021

Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation.

IEEE J Biomed Health Inform 2021 09 3;25(9):3507-3516. Epub 2021 Sep 3.

Multimodal positron emission tomography-computed tomography (PET-CT) is used routinely in the assessment of cancer. PET-CT combines the high sensitivity for tumor detection of PET and anatomical information from CT. Tumor segmentation is a critical element of PET-CT but at present, the performance of existing automated methods for this challenging task is low. Segmentation tends to be done manually by different imaging experts, which is labor-intensive and prone to errors and inconsistency. Previous automated segmentation methods largely focused on fusing information that is extracted separately from the PET and CT modalities, with the underlying assumption that each modality contains complementary information. However, these methods do not fully exploit the high PET tumor sensitivity that can guide the segmentation. We introduce a deep learning-based framework in multimodal PET-CT segmentation with a multimodal spatial attention module (MSAM). The MSAM automatically learns to emphasize regions (spatial areas) related to tumors and suppress normal regions with physiologic high-uptake from the PET input. The resulting spatial attention maps are subsequently employed to target a convolutional neural network (CNN) backbone for segmentation of areas with higher tumor likelihood from the CT image. Our experimental results on two clinical PET-CT datasets of non-small cell lung cancer (NSCLC) and soft tissue sarcoma (STS) validate the effectiveness of our framework in these different cancer types. We show that our MSAM, with a conventional U-Net backbone, surpasses the state-of-the-art lung tumor segmentation approach by a margin of 7.6% in Dice similarity coefficient (DSC).
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/JBHI.2021.3059453DOI Listing
September 2021

Unsupervised Domain Adaptation to Classify Medical Images Using Zero-Bias Convolutional Auto-Encoders and Context-Based Feature Augmentation.

IEEE Trans Med Imaging 2020 07 3;39(7):2385-2394. Epub 2020 Feb 3.

The accuracy and robustness of image classification with supervised deep learning are dependent on the availability of large-scale labelled training data. In medical imaging, these large labelled datasets are sparse, mainly related to the complexity in manual annotation. Deep convolutional neural networks (CNNs), with transferable knowledge, have been employed as a solution to limited annotated data through: 1) fine-tuning generic knowledge with a relatively smaller amount of labelled medical imaging data, and 2) learning image representation that is invariant to different domains. These approaches, however, are still reliant on labelled medical image data. Our aim is to use a new hierarchical unsupervised feature extractor to reduce reliance on annotated training data. Our unsupervised approach uses a multi-layer zero-bias convolutional auto-encoder that constrains the transformation of generic features from a pre-trained CNN (for natural images) to non-redundant and locally relevant features for the medical image data. We also propose a context-based feature augmentation scheme to improve the discriminative power of the feature representation. We evaluated our approach on 3 public medical image datasets and compared it to other state-of-the-art supervised CNNs. Our unsupervised approach achieved better accuracy when compared to other conventional unsupervised methods and baseline fine-tuned CNNs.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/TMI.2020.2971258DOI Listing
July 2020

A deep learning technique for imputing missing healthcare data.

Annu Int Conf IEEE Eng Med Biol Soc 2019 Jul;2019:6513-6516

Missing data is a frequent occurrence in medical and health datasets. The analysis of datasets with missing data can lead to loss in statistical power or biased results. We address this issue with a novel deep learning technique to impute missing values in health data. Our method extends upon an autoencoder to derive a deep learning architecture that can learn the hidden representations of data even when data is perturbed by missing values (noise). Our model is constructed with overcomplete representation and trained with denoising regularization. This allows the latent/hidden layers of our model to effectively extract the relationships between different variables; these relationships are then used to reconstruct missing values. Our contributions include a new loss function designed to avoid local optima, and this helps the model to learn the real distribution of variables in the dataset. We evaluate our method in comparison with other well-established imputation strategies (mean, median imputation, SVD, KNN, matrix factorization and soft impute) on 48,350 Linked Birth/Infant Death Cohort Data records. Our experiments demonstrate that our method achieved lower imputation mean squared error (MSE=0.00988) compared with other imputation methods (with MSE ranging from 0.02 to 0.08). When assessing the imputation quality using the imputed data for prediction tasks, our experiments show that the data imputed by our method yielded better results (F1=70.37%) compared with other imputation methods (ranging from 66 to 69%).
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/EMBC.2019.8856760DOI Listing
July 2019

An Automated Framework for Large Scale Retrospective Analysis of Ultrasound Images.

IEEE J Transl Eng Health Med 2019 19;7:1800909. Epub 2019 Nov 19.

1School of Computer ScienceThe University of SydneySydneyNSW2006Australia.

Objective: Large scale retrospective analysis of fetal ultrasound (US) data is important in the understanding of the cumulative impact of antenatal factors on offspring's health outcomes. Although the benefits are evident, there is a paucity of research into such large scale studies as it requires tedious and expensive effort in manual processing of large scale data repositories. This study presents an automated framework to facilitate retrospective analysis of large scale US data repositories.

Method: Our framework consists of four modules: (1) an image classifier to distinguish the Brightness (B) -mode images; (2) a fetal image structure identifier to select US images containing user-defined fetal structures of interest (fSOI); (3) a biometry measurement algorithm to measure the fSOIs in the images and, (4) a visual evaluation module to allow clinicians to validate the outcomes.

Results: We demonstrated our framework using thalamus as the fSOI from a hospital repository of more than 80,000 patients, consisting of 3,816,967 antenatal US files (DICOM objects). Our framework classified 1,869,105 B-mode images and from which 38,786 thalamus images were identified. We selected a random subset of 1290 US files with 558 B-mode (containing 19 thalamus images and the rest being other US data) and evaluated our framework performance. With the evaluation set, B-mode image classification resulted in accuracy, precision, and recall (APR) of 98.67%, 99.75% and 98.57% respectively. For fSOI identification, APR was 93.12%, 97.76% and 80.78% respectively.

Conclusion: We introduced a completely automated approach designed to analyze a large scale data repository to enable retrospective clinical research.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/JTEHM.2019.2952379DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6908460PMC
November 2019

Convolutional sparse kernel network for unsupervised medical image analysis.

Med Image Anal 2019 08 12;56:140-151. Epub 2019 Jun 12.

School of Computer Science, University of Sydney, NSW, Australia. Electronic address:

The availability of large-scale annotated image datasets and recent advances in supervised deep learning methods enable the end-to-end derivation of representative image features that can impact a variety of image analysis problems. Such supervised approaches, however, are difficult to implement in the medical domain where large volumes of labelled data are difficult to obtain due to the complexity of manual annotation and inter- and intra-observer variability in label assignment. We propose a new convolutional sparse kernel network (CSKN), which is a hierarchical unsupervised feature learning framework that addresses the challenge of learning representative visual features in medical image analysis domains where there is a lack of annotated training data. Our framework has three contributions: (i) we extend kernel learning to identify and represent invariant features across image sub-patches in an unsupervised manner. (ii) We initialise our kernel learning with a layer-wise pre-training scheme that leverages the sparsity inherent in medical images to extract initial discriminative features. (iii) We adapt a multi-scale spatial pyramid pooling (SPP) framework to capture subtle geometric differences between learned visual features. We evaluated our framework in medical image retrieval and classification on three public datasets. Our results show that our CSKN had better accuracy when compared to other conventional unsupervised methods and comparable accuracy to methods that used state-of-the-art supervised convolutional neural networks (CNNs). Our findings indicate that our unsupervised CSKN provides an opportunity to leverage unannotated big data in medical imaging repositories.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.media.2019.06.005DOI Listing
August 2019

Co-Learning Feature Fusion Maps from PET-CT Images of Lung Cancer.

IEEE Trans Med Imaging 2019 Jun 17. Epub 2019 Jun 17.

The analysis of multi-modality positron emission tomography and computed tomography (PET-CT) images for computer aided diagnosis applications (e.g., detection and segmentation) requires combining the sensitivity of PET to detect abnormal regions with anatomical localization from CT. Current methods for PET-CT image analysis either process the modalities separately or fuse information from each modality based on knowledge about the image analysis task. These methods generally do not consider the spatially varying visual characteristics that encode different information across the different modalities, which have different priorities at different locations. For example, a high abnormal PET uptake in the lungs is more meaningful for tumor detection than physiological PET uptake in the heart. Our aim is to improve fusion of the complementary information in multi-modality PET-CT with a new supervised convolutional neural network (CNN) that learns to fuse complementary information for multi-modality medical image analysis. Our CNN first encodes modality-specific features and then uses them to derive a spatially varying fusion map that quantifies the relative importance of each modality's features across different spatial locations. These fusion maps are then multiplied with the modality-specific feature maps to obtain a representation of the complementary multi-modality information at different locations, which can then be used for image analysis. We evaluated the ability of our CNN to detect and segment multiple regions (lungs, mediastinum, tumors) with different fusion requirements using a dataset of PET-CT images of lung cancer. We compared our method to baseline techniques for multi-modality image fusion (fused inputs (FS), multi-branch (MB) techniques, and multichannel (MC) techniques) and segmentation. Our findings show that our CNN had a significantly higher foreground detection accuracy (99.29%, p < 0:05) than the fusion baselines (FS: 99.00%, MB: 99.08%, TC: 98.92%) and a significantly higher Dice score (63.85%) than recent PET-CT tumor segmentation methods.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/TMI.2019.2923601DOI Listing
June 2019

Decision Fusion-Based Fetal Ultrasound Image Plane Classification Using Convolutional Neural Networks.

Ultrasound Med Biol 2019 05 27;45(5):1259-1273. Epub 2019 Feb 27.

Department of Engineering Design, Indian Institute of Technology Madras, India. Electronic address:

Machine learning for ultrasound image analysis and interpretation can be helpful in automated image classification in large-scale retrospective analyses to objectively derive new indicators of abnormal fetal development that are embedded in ultrasound images. Current approaches to automatic classification are limited to the use of either image patches (cropped images) or the global (whole) image. As many fetal organs have similar visual features, cropped images can misclassify certain structures such as the kidneys and abdomen. Also, the whole image does not encode sufficient local information about structures to identify different structures in different locations. Here we propose a method to automatically classify 14 different fetal structures in 2-D fetal ultrasound images by fusing information from both cropped regions of fetal structures and the whole image. Our method trains two feature extractors by fine-tuning pre-trained convolutional neural networks with the whole ultrasound fetal images and the discriminant regions of the fetal structures found in the whole image. The novelty of our method is in integrating the classification decisions made from the global and local features without relying on priors. In addition, our method can use the classification outcome to localize the fetal structures in the image. Our experiments on a data set of 4074 2-D ultrasound images (training: 3109, test: 965) achieved a mean accuracy of 97.05%, mean precision of 76.47% and mean recall of 75.41%. The Cohen κ of 0.72 revealed the highest agreement between the ground truth and the proposed method. The superiority of the proposed method over the other non-fusion-based methods is statistically significant (p < 0.05). We found that our method is capable of predicting images without ultrasound scanner overlays with a mean accuracy of 92%. The proposed method can be leveraged to retrospectively classify any ultrasound images in clinical research.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.ultrasmedbio.2018.11.016DOI Listing
May 2019

A direct volume rendering visualization approach for serial PET-CT scans that preserves anatomical consistency.

Int J Comput Assist Radiol Surg 2019 May 19;14(5):733-744. Epub 2019 Jan 19.

Sydney Medical School, The University of Sydney, Sydney, Australia.

Purpose: Our aim was to develop an interactive 3D direct volume rendering (DVR) visualization solution to interpret and analyze complex, serial multi-modality imaging datasets from positron emission tomography-computed tomography (PET-CT).

Methods: Our approach uses: (i) a serial transfer function (TF) optimization to automatically depict particular regions of interest (ROIs) over serial datasets with consistent anatomical structures; (ii) integration of a serial segmentation algorithm to interactively identify and track ROIs on PET; and (iii) parallel graphics processing unit (GPU) implementation for interactive visualization.

Results: Our DVR visualization more easily identifies changes in ROIs in serial scans in an automated fashion and parallel GPU computation which enables interactive visualization.

Conclusions: Our approach provides a rapid 3D visualization of relevant ROIs over multiple scans, and we suggest that it can be used as an adjunct to conventional 2D viewing software from scanner vendors.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1007/s11548-019-01916-2DOI Listing
May 2019

Unsupervised Two-Path Neural Network for Cell Event Detection and Classification Using Spatiotemporal Patterns.

IEEE Trans Med Imaging 2019 06 7;38(6):1477-1487. Epub 2018 Dec 7.

Automatic event detection in cell videos is essential for monitoring cell populations in biomedicine. Deep learning methods have advantages over traditional approaches for cell event detection due to their ability to capture more discriminative features of cellular processes. Supervised deep learning methods, however, are inherently limited due to the scarcity of annotated data. Unsupervised deep learning methods have shown promise in general (non-cell) videos because they can learn the visual appearance and motion of regularly occurring events. Cell videos, however, can have rapid, irregular changes in cell appearance and motion, such as during cell division and death, which are often the events of most interest. We propose a novel unsupervised two-path input neural network architecture to capture these irregular events with three key elements: 1) a visual encoding path to capture regular spatiotemporal patterns of observed objects with convolutional long short-term memory units; 2) an event detection path to extract information related to irregular events with max-pooling layers; and 3) integration of the hidden states of the two paths to provide a comprehensive representation of the video that is used to simultaneously locate and classify cell events. We evaluated our network in detecting cell division in densely packed stem cells in phase-contrast microscopy videos. Our unsupervised method achieved higher or comparable accuracy to standard and state-of-the-art supervised methods.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/TMI.2018.2885572DOI Listing
June 2019

Controlled Synthesis of Dermoscopic Images via a New Color Labeled Generative Style Transfer Network to Enhance Melanoma Segmentation.

Annu Int Conf IEEE Eng Med Biol Soc 2018 Jul;2018:2591-2594

Dermoscopic imaging is an established technique to detect, track, and diagnose malignant melanoma, and one of the ways to improve this technique is via computer-aided image segmentation. Image segmentation is an important step towards building computerized detection and classification systems by delineating the area of interest, in our case, the skin lesion, from the background. However, current segmentation techniques are hard pressed to account for color artifacts within dermoscopic images that are often incorrectly detected as part of the lesion. Often there are few annotated examples of these artifacts, which limits training segmentation methods like the fully convolutional network (FCN) due to the skewed dataset. We propose to improve FCN training by augmenting the dataset with synthetic images created in a controlled manner using a generative adversarial network (GAN). Our novelty lies in the use of a color label (CL) to specify the different characteristics (approximate size, location, and shape) of the different regions (skin, lesion, artifacts) in the synthetic images. Our GAN is trained to perform style transfer of real melanoma image characteristics (e.g. texture) onto these color labels, allowing us to generate specific types of images containing artifacts. Our experimental results demonstrate that the synthetic images generated by our technique have a lower mean average error when compared to synthetic images generated using traditional binary labels. As a consequence, we demonstrated improvements in melanoma image segmentation when using synthetic images generated by our technique.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/EMBC.2018.8512842DOI Listing
July 2018

Patch-level Tumor Classification in Digital Histopathology Images with Domain Adapted Deep Learning.

Annu Int Conf IEEE Eng Med Biol Soc 2018 Jul;2018:644-647

Tumor histopathology is a crucial step in cancer diagnosis which involves visual inspection of imaging data to detect the presence of tumor cells among healthy tissues. This manual process can be time-consuming, error-prone, and influenced by the expertise of the pathologist. Recent deep learning methods for image classification and detection using convolutional neural networks (CNNs) have demonstrated marked improvements in the accuracy of a variety of medical imaging analysis tasks. However, most well-established deep learning methods require large annotated training datasets that are specific to the particular problem domain; such datasets are difficult to acquire for histopathology data where visual characteristics differ between different tissue types, in addition to the need for precise annotations. In this study, we overcome the lack of annotated training dataset in histopathology images of a particular domain by adapting annotated histopathology images from different domains (tissue types). The data from other tissue types are used to pre-train CNNs into a shared histopathology domain (e.g., stains, cellular structures) such that it can be further tuned/optimized for a specific tissue type. We evaluated our classification method on publically available datasets of histopathology images; the accuracy and area under the receiver operating characteristic curve (AUC) of our method was higher than CNNs trained from scratch on limited data (accuracy: 84.3% vs. 78.3%; AUC: 0.918 vs. 0.867), suggesting that domain adaptation can be a valuable approach to histopathological images classification.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/EMBC.2018.8512353DOI Listing
July 2018

Dermoscopic Image Segmentation via Multistage Fully Convolutional Networks.

IEEE Trans Biomed Eng 2017 09 7;64(9):2065-2074. Epub 2017 Jun 7.

Objective: Segmentation of skin lesions is an important step in the automated computer aided diagnosis of melanoma. However, existing segmentation methods have a tendency to over- or under-segment the lesions and perform poorly when the lesions have fuzzy boundaries, low contrast with the background, inhomogeneous textures, or contain artifacts. Furthermore, the performance of these methods are heavily reliant on the appropriate tuning of a large number of parameters as well as the use of effective preprocessing techniques, such as illumination correction and hair removal.

Methods: We propose to leverage fully convolutional networks (FCNs) to automatically segment the skin lesions. FCNs are a neural network architecture that achieves object detection by hierarchically combining low-level appearance information with high-level semantic information. We address the issue of FCN producing coarse segmentation boundaries for challenging skin lesions (e.g., those with fuzzy boundaries and/or low difference in the textures between the foreground and the background) through a multistage segmentation approach in which multiple FCNs learn complementary visual characteristics of different skin lesions; early stage FCNs learn coarse appearance and localization information while late-stage FCNs learn the subtle characteristics of the lesion boundaries. We also introduce a new parallel integration method to combine the complementary information derived from individual segmentation stages to achieve a final segmentation result that has accurate localization and well-defined lesion boundaries, even for the most challenging skin lesions.

Results: We achieved an average Dice coefficient of 91.18% on the ISBI 2016 Skin Lesion Challenge dataset and 90.66% on the PH2 dataset.

Conclusion And Significance: Our extensive experimental results on two well-established public benchmark datasets demonstrate that our method is more effective than other state-of-the-art methods for skin lesion segmentation.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/TBME.2017.2712771DOI Listing
September 2017

An Ensemble of Fine-Tuned Convolutional Neural Networks for Medical Image Classification.

IEEE J Biomed Health Inform 2017 01 5;21(1):31-40. Epub 2016 Dec 5.

The availability of medical imaging data from clinical archives, research literature, and clinical manuals, coupled with recent advances in computer vision offer the opportunity for image-based diagnosis, teaching, and biomedical research. However, the content and semantics of an image can vary depending on its modality and as such the identification of image modality is an important preliminary step. The key challenge for automatically classifying the modality of a medical image is due to the visual characteristics of different modalities: some are visually distinct while others may have only subtle differences. This challenge is compounded by variations in the appearance of images based on the diseases depicted and a lack of sufficient training data for some modalities. In this paper, we introduce a new method for classifying medical images that uses an ensemble of different convolutional neural network (CNN) architectures. CNNs are a state-of-the-art image classification technique that learns the optimal image features for a given classification task. We hypothesise that different CNN architectures learn different levels of semantic image representation and thus an ensemble of CNNs will enable higher quality features to be extracted. Our method develops a new feature extractor by fine-tuning CNNs that have been initialized on a large dataset of natural images. The fine-tuning process leverages the generic image features from natural images that are fundamental for all images and optimizes them for the variety of medical imaging modalities. These features are used to train numerous multiclass classifiers whose posterior probabilities are fused to predict the modalities of unseen images. Our experiments on the ImageCLEF 2016 medical image public dataset (30 modalities; 6776 training images, and 4166 test images) show that our ensemble of fine-tuned CNNs achieves a higher accuracy than established CNNs. Our ensemble also achieves a higher accuracy than methods in the literature evaluated on the same benchmark dataset and is only overtaken by those methods that source additional training data.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/JBHI.2016.2635663DOI Listing
January 2017

Saliency-Based Lesion Segmentation Via Background Detection in Dermoscopic Images.

IEEE J Biomed Health Inform 2017 11 16;21(6):1685-1693. Epub 2017 Jan 16.

The segmentation of skin lesions in dermoscopic images is a fundamental step in automated computer-aided diagnosis of melanoma. Conventional segmentation methods, however, have difficulties when the lesion borders are indistinct and when contrast between the lesion and the surrounding skin is low. They also perform poorly when there is a heterogeneous background or a lesion that touches the image boundaries; this then results in under- and oversegmentation of the skin lesion. We suggest that saliency detection using the reconstruction errors derived from a sparse representation model coupled with a novel background detection can more accurately discriminate the lesion from surrounding regions. We further propose a Bayesian framework that better delineates the shape and boundaries of the lesion. We also evaluated our approach on two public datasets comprising 1100 dermoscopic images and compared it to other conventional and state-of-the-art unsupervised (i.e., no training required) lesion segmentation methods, as well as the state-of-the-art unsupervised saliency detection methods. Our results show that our approach is more accurate and robust in segmenting lesions compared to other methods. We also discuss the general extension of our framework as a saliency optimization algorithm for lesion segmentation.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/JBHI.2017.2653179DOI Listing
November 2017

Automatic detection and classification of regions of FDG uptake in whole-body PET-CT lymphoma studies.

Comput Med Imaging Graph 2017 Sep 2;60:3-10. Epub 2016 Dec 2.

School of Information Technologies, University of Sydney, NSW, Australia; Department of Molecular Imaging, Royal Prince Alfred Hospital, NSW, Australia; Sydney Medical School, University of Sydney, NSW, Australia.

[F]-Fluorodeoxyglucose (FDG) positron emission tomography-computed tomography (PET-CT) scans of lymphoma patients usually show disease involvement as foci of increased radiotracer uptake. Existing methods for detecting abnormalities, model the characteristics of these foci; this is challenging due to the inconsistent shape and localization information about the lesions. Thresholding the degree of FDG uptake is the standard method to separate different sites of involvement. But may fragment sites into smaller regions, and may also incorrectly identify sites of normal physiological FDG uptake and normal FDG excretion (sFEPU) such as the kidneys, bladder, brain and heart. These sFEPU can obscure sites of abnormal uptake, which can make image interpretation problematic. Identifying sFEPU is therefore important for improving the sensitivity of lesion detection and image interpretation. Existing methods to identify sFEPU are inaccurate because they fail to account for the low inter-class differences between sFEPU fragments and their inconsistent localization information. In this study, we address this issue by using a multi-scale superpixel-based encoding (MSE) to group the individual sFEPU fragments into larger regions, thereby, enabling the extraction of highly discriminative image features via domain transferred convolutional neural networks. We then classify there regions into one of the sFEPU classes using a class-driven feature selection and classification model (CFSC) method that avoids overfitting to the most frequently occurring classes. Our experiments on 40 whole-body lymphoma PET-CT studies show that our method achieved better accuracy (an average F-score of 91.73%) compared to existing methods in the classification of sFEPU.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.compmedimag.2016.11.008DOI Listing
September 2017

Automatic Measurement of Thalamic Diameter in 2-D Fetal Ultrasound Brain Images Using Shape Prior Constrained Regularized Level Sets.

IEEE J Biomed Health Inform 2017 07 20;21(4):1069-1078. Epub 2016 Jun 20.

We derived an automated algorithm for accurately measuring the thalamic diameter from 2-D fetal ultrasound (US) brain images. The algorithm overcomes the inherent limitations of the US image modality: nonuniform density; missing boundaries; and strong speckle noise. We introduced a "guitar" structure that represents the negative space surrounding the thalamic regions. The guitar acts as a landmark for deriving the widest points of the thalamus even when its boundaries are not identifiable. We augmented a generalized level-set framework with a shape prior and constraints derived from statistical shape models of the guitars; this framework was used to segment US images and measure the thalamic diameter. Our segmentation method achieved a higher mean Dice similarity coefficient, Hausdorff distance, specificity, and reduced contour leakage when compared to other well-established methods. The automatic thalamic diameter measurement had an interobserver variability of -0.56 ± 2.29 mm compared to manual measurement by an expert sonographer. Our method was capable of automatically estimating the thalamic diameter, with the measurement accuracy on par with clinical assessment. Our method can be used as part of computer-assisted screening tools that automatically measure the biometrics of the fetal thalamus; these biometrics are linked to neurodevelopmental outcomes.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/JBHI.2016.2582175DOI Listing
July 2017

Efficient visibility-driven medical image visualisation via adaptive binned visibility histogram.

Comput Med Imaging Graph 2016 07 20;51:40-9. Epub 2016 Apr 20.

Sydney Medical School, University of Sydney, Australia; Department of Molecular Imaging, Royal Prince Alfred Hospital, Australia.

'Visibility' is a fundamental optical property that represents the observable, by users, proportion of the voxels in a volume during interactive volume rendering. The manipulation of this 'visibility' improves the volume rendering processes; for instance by ensuring the visibility of regions of interest (ROIs) or by guiding the identification of an optimal rendering view-point. The construction of visibility histograms (VHs), which represent the distribution of all the visibility of all voxels in the rendered volume, enables users to explore the volume with real-time feedback about occlusion patterns among spatially related structures during volume rendering manipulations. Volume rendered medical images have been a primary beneficiary of VH given the need to ensure that specific ROIs are visible relative to the surrounding structures, e.g. the visualisation of tumours that may otherwise be occluded by neighbouring structures. VH construction and its subsequent manipulations, however, are computationally expensive due to the histogram binning of the visibilities. This limits the real-time application of VH to medical images that have large intensity ranges and volume dimensions and require a large number of histogram bins. In this study, we introduce an efficient adaptive binned visibility histogram (AB-VH) in which a smaller number of histogram bins are used to represent the visibility distribution of the full VH. We adaptively bin medical images by using a cluster analysis algorithm that groups the voxels according to their intensity similarities into a smaller subset of bins while preserving the distribution of the intensity range of the original images. We increase efficiency by exploiting the parallel computation and multiple render targets (MRT) extension of the modern graphical processing units (GPUs) and this enables efficient computation of the histogram. We show the application of our method to single-modality computed tomography (CT), magnetic resonance (MR) imaging and multi-modality positron emission tomography-CT (PET-CT). In our experiments, the AB-VH markedly improved the computational efficiency for the VH construction and thus improved the subsequent VH-driven volume manipulations. This efficiency was achieved without major degradation in the VH visually and numerical differences between the AB-VH and its full-bin counterpart. We applied several variants of the K-means clustering algorithm with varying Ks (the number of clusters) and found that higher values of K resulted in better performance at a lower computational gain. The AB-VH also had an improved performance when compared to the conventional method of down-sampling of the histogram bins (equal binning) for volume rendering visualisation.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.compmedimag.2016.04.003DOI Listing
July 2016

Adapting content-based image retrieval techniques for the semantic annotation of medical images.

Comput Med Imaging Graph 2016 Apr 4;49:37-45. Epub 2016 Feb 4.

School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia; Med-X Research Institute, Shanghai Jiao Tong University, China. Electronic address:

The automatic annotation of medical images is a prerequisite for building comprehensive semantic archives that can be used to enhance evidence-based diagnosis, physician education, and biomedical research. Annotation also has important applications in the automatic generation of structured radiology reports. Much of the prior research work has focused on annotating images with properties such as the modality of the image, or the biological system or body region being imaged. However, many challenges remain for the annotation of high-level semantic content in medical images (e.g., presence of calcification, vessel obstruction, etc.) due to the difficulty in discovering relationships and associations between low-level image features and high-level semantic concepts. This difficulty is further compounded by the lack of labelled training data. In this paper, we present a method for the automatic semantic annotation of medical images that leverages techniques from content-based image retrieval (CBIR). CBIR is a well-established image search technology that uses quantifiable low-level image features to represent the high-level semantic content depicted in those images. Our method extends CBIR techniques to identify or retrieve a collection of labelled images that have similar low-level features and then uses this collection to determine the best high-level semantic annotations. We demonstrate our annotation method using retrieval via weighted nearest-neighbour retrieval and multi-class classification to show that our approach is viable regardless of the underlying retrieval strategy. We experimentally compared our method with several well-established baseline techniques (classification and regression) and showed that our method achieved the highest accuracy in the annotation of liver computed tomography (CT) images.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.compmedimag.2016.01.001DOI Listing
April 2016

Efficient PET-CT image retrieval using graphs embedded into a vector space.

Annu Int Conf IEEE Eng Med Biol Soc 2014 ;2014:1901-4

Combined positron emission tomography and computed tomography (PET-CT) produces functional data (from PET) in relation to anatomical context (from CT) and it has made a major contribution to improved cancer diagnosis, tumour localisation, and staging. The ability to retrieve PET-CT images from large archives has potential applications in diagnosis, education, and research. PET-CT image retrieval requires the consideration of modality-specific 3D image features and spatial contextual relationships between features in both modalities. Graph-based retrieval methods have recently been applied to represent contextual relationships during PET-CT image retrieval. However, accurate methods are computationally complex, often requiring offline processing, and are unable to retrieve images at interactive rates. In this paper, we propose a method for PET-CT image retrieval using a vector space embedding of graph descriptors. Our method defines the vector space in terms of the distance between a graph representing a PET-CT image and a set of fixed-sized prototype graphs; each vector component measures the dissimilarity of the graph and a prototype. Our evaluation shows that our method is significantly faster (≈800× speedup, p <; 0.05) than retrieval using the graph-edit distance while maintaining comparable precision (5% difference, p > 0.05).
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/EMBC.2014.6943982DOI Listing
July 2016

A Visual Analytics Approach Using the Exploration of Multidimensional Feature Spaces for Content-Based Medical Image Retrieval.

IEEE J Biomed Health Inform 2015 Sep 3;19(5):1734-46. Epub 2014 Oct 3.

Content-based image retrieval (CBIR) is a search technique based on the similarity of visual features and has demonstrated potential benefits for medical diagnosis, education, and research. However, clinical adoption of CBIR is partially hindered by the difference between the computed image similarity and the user's search intent, the semantic gap, with the end result that relevant images with outlier features may not be retrieved. Furthermore, most CBIR algorithms do not provide intuitive explanations as to why the retrieved images were considered similar to the query (e.g., which subset of features were similar), hence, it is difficult for users to verify if relevant images, with a small subset of outlier features, were missed. Users, therefore, resort to examining irrelevant images and there are limited opportunities to discover these "missed" images. In this paper, we propose a new approach to medical CBIR by enabling a guided visual exploration of the search space through a tool, called visual analytics for medical image retrieval (VAMIR). The visual analytics approach facilitates interactive exploration of the entire dataset using the query image as a point-of-reference. We conducted a user study and several case studies to demonstrate the capabilities of VAMIR in the retrieval of computed tomography images and multimodality positron emission tomography and computed tomography images.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/JBHI.2014.2361318DOI Listing
September 2015

A graph-based approach for the retrieval of multi-modality medical images.

Med Image Anal 2014 Feb 6;18(2):330-42. Epub 2013 Dec 6.

Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Sydney, Australia; Med-X Research Institute, Shanghai Jiao Tong University, China. Electronic address:

In this paper, we address the retrieval of multi-modality medical volumes, which consist of two different imaging modalities, acquired sequentially, from the same scanner. One such example, positron emission tomography and computed tomography (PET-CT), provides physicians with complementary functional and anatomical features as well as spatial relationships and has led to improved cancer diagnosis, localisation, and staging. The challenge of multi-modality volume retrieval for cancer patients lies in representing the complementary geometric and topologic attributes between tumours and organs. These attributes and relationships, which are used for tumour staging and classification, can be formulated as a graph. It has been demonstrated that graph-based methods have high accuracy for retrieval by spatial similarity. However, naïvely representing all relationships on a complete graph obscures the structure of the tumour-anatomy relationships. We propose a new graph structure derived from complete graphs that structurally constrains the edges connected to tumour vertices based upon the spatial proximity of tumours and organs. This enables retrieval on the basis of tumour localisation. We also present a similarity matching algorithm that accounts for different feature sets for graph elements from different imaging modalities. Our method emphasises the relationships between a tumour and related organs, while still modelling patient-specific anatomical variations. Constraining tumours to related anatomical structures improves the discrimination potential of graphs, making it easier to retrieve similar images based on tumour location. We evaluated our retrieval methodology on a dataset of clinical PET-CT volumes. Our results showed that our method enabled the retrieval of multi-modality images using spatial features. Our graph-based retrieval algorithm achieved a higher precision than several other retrieval techniques: gray-level histograms as well as state-of-the-art methods such as visual words using the scale- invariant feature transform (SIFT) and relational matrices representing the spatial arrangements of objects.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.media.2013.11.003DOI Listing
February 2014

Cellular automata and anisotropic diffusion filter based interactive tumor segmentation for positron emission tomography.

Annu Int Conf IEEE Eng Med Biol Soc 2013 ;2013:5453-6

Tumor segmentation in positron emission tomography (PET) aids clinical diagnosis and in assessing treatment response. However, the low resolution and signal-to-noise inherent in PET images, makes accurate tumor segmentation challenging. Manual delineation is time-consuming and subjective, whereas fully automated algorithms are often limited to particular tumor types, and have difficulties in segmenting small and low-contrast tumors. Interactive segmentation may reduce the inter-observer variability and minimize the user input. In this study, we present a new interactive PET tumor segmentation method based on cellular automata (CA) and a nonlinear anisotropic diffusion filter (ADF). CA is tolerant of noise and image pattern complexity while ADF reduces noise while preserving edges. By coupling CA with ADF, our proposed approach was robust and accurate in detecting and segmenting noisy tumors. We evaluated our method with computer simulation and clinical data and it outperformed other common interactive PET segmentation algorithms.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/EMBC.2013.6610783DOI Listing
August 2015

Content-based medical image retrieval: a survey of applications to multidimensional and multimodality data.

J Digit Imaging 2013 Dec;26(6):1025-39

Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Building J12, Sydney, NSW, 2006, Australia,

Medical imaging is fundamental to modern healthcare, and its widespread use has resulted in the creation of image databases, as well as picture archiving and communication systems. These repositories now contain images from a diverse range of modalities, multidimensional (three-dimensional or time-varying) images, as well as co-aligned multimodality images. These image collections offer the opportunity for evidence-based diagnosis, teaching, and research; for these applications, there is a requirement for appropriate methods to search the collections for images that have characteristics similar to the case(s) of interest. Content-based image retrieval (CBIR) is an image search technique that complements the conventional text-based retrieval of images by using visual features, such as color, texture, and shape, as search criteria. Medical CBIR is an established field of study that is beginning to realize promise when applied to multidimensional and multimodality medical data. In this paper, we present a review of state-of-the-art medical CBIR approaches in five main categories: two-dimensional image retrieval, retrieval of images with three or more dimensions, the use of nonimage data to enhance the retrieval, multimodality image retrieval, and retrieval from diverse datasets. We use these categories as a framework for discussing the state of the art, focusing on the characteristics and modalities of the information used during medical image retrieval.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1007/s10278-013-9619-2DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3824925PMC
December 2013

Designing user interfaces to enhance human interpretation of medical content-based image retrieval: application to PET-CT images.

Int J Comput Assist Radiol Surg 2013 Nov 7;8(6):1003-14. Epub 2013 May 7.

Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Sydney, Australia,

Purpose: Content-based image retrieval (CBIR) in medicine has been demonstrated to improve evidence-based diagnosis, education, and teaching. However, the low clinical adoption of CBIR is partially because the focus of most studies has been the development of feature extraction and similarity measurement algorithms with limited work on facilitating better understanding of the similarity between complex volumetric and multi-modality medical images. In this paper, we present a method for defining user interfaces (UIs) that enable effective human user interpretation of retrieved images.

Methods: We derived a set of visualisation and interaction requirements based on the characteristics of modern volumetric medical images. We implemented a UI that visualised multiple views of a single image, displayed abstractions of image data, and provided access to supplementary non-image data. We also defined interactions for refining the search and visually indicating the similarities between images. We applied the UI for the retrieval of multi-modality positron emission tomography and computed tomography (PET-CT) images. We conducted a user survey to evaluate the capabilities of our UI.

Results: Our proposed method obtained a high rating ( ≥ 4 out of 5) in the majority of survey questions. In particular, the survey responses indicated the UI presented all the information necessary to understand the retrieved images, and did so in an intuitive manner.

Conclusion: Our proposed UI design improved the ability of users to interpret and understand the similarity between retrieved PET-CT images. The implementation of CBIR UIs designed to assist human interpretation could facilitate wider adoption of medical CBIR systems.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1007/s11548-013-0896-5DOI Listing
November 2013

A graph-based approach to the retrieval of volumetric PET-CT lung images.

Annu Int Conf IEEE Eng Med Biol Soc 2012 ;2012:5408-11

Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Australia.

Combined positron emission tomography and computed tomography (PET-CT) scans have become a critical tool for the diagnosis, localisation, and staging of most cancers. This has led to a rapid expansion in the volume of PET-CT data that is archived in clinical environments. The ability to search these vast imaging collections has potential clinical applications in evidence-based diagnosis, physician training, and biomedical research that may lead to the discovery of new knowledge. Content-based image retrieval (CBIR) is an image search technique that complements conventional text-based retrieval by the use of image features as search criteria. Graph-based CBIR approaches have been found to be exemplary methods for medical CBIR as they provide the ability to consider disease localisation during the similarity measurement. However, the majority of graph-based CBIR studies have been based on 2D key slice approaches and did not exploit the rich volumetric data that is inherent to modern medical images, such as multi-modal PET-CT. In this paper, we present a graph-based CBIR method that exploits 3D spatial features extracted from volumetric regions of interest (ROIs). We index these features as attributes of a graph representation and use a graph-edit distance to measure the similarity of PET-CT images based on the spatial arrangement of tumours and organs in a 3D space. Our study aims to explore the capability of these graphs in 3D PET-CT CBIR. We show that our method achieves promising precision when retrieving clinical PET-CT images of patients with lung tumours.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/EMBC.2012.6347217DOI Listing
July 2013

A patient-centric distribution architecture for medical image sharing.

Health Inf Sci Syst 2013 10;1. Epub 2013 Jan 10.

School of Information Technologies, University of Sydney, Building J12, Sydney, Australia.

Over the past decade, rapid development of imaging technologies has resulted in the introduction of improved imaging devices, such as multi-modality scanners that produce combined positron emission tomography-computed tomography (PET-CT) images. The adoption of picture archiving and communication systems (PACS) in hospitals have dramatically improved the ability to digitally share medical image studies via portable storage, mobile devices and the Internet. This has in turn led to increased productivity, greater flexibility, and improved communication between hospital staff, referring physicians, and outpatients. However, many of these sharing and viewing capabilities are limited to proprietary vendor-specific applications. Furthermore, there are still interoperability and deployment issues which reduce the rate of adoption of such technologies, thus leaving many stakeholders, particularly outpatients and referring physicians, with access to only traditional still images with no ability to view or interpret the data in full. In this paper, we present a distribution architecture for medical image display across numerous devices and media, which uses a preprocessor and an in-built networking framework to improve compatibility and promote greater accessibility of medical data. Our INVOLVE2 system consists of three main software modules: 1) a preprocessor, which collates and converts imaging studies into a compressed and distributable format; 2) a PACS-compatible workflow for self-managing distribution of medical data, e.g. via CD USB, network etc; 3) support for potential mobile and web-based data access. The focus of this study was on cultivating patient-centric care, by allowing outpatient users to comfortably access and interpret their own data. As such, the image viewing software included on our cross-platform CDs was designed with a simple and intuitive user-interface (UI) for use by outpatients and referring physicians. Furthermore, digital image access via mobile devices or web-based access enables users to engage with their data in a convenient and user-friendly way. We evaluated the INVOLVE2 system using a pilot deployment in a hospital environment.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1186/2047-2501-1-3DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4336110PMC
March 2015

A web-based image viewer for multiple PET-CT follow-up studies.

Annu Int Conf IEEE Eng Med Biol Soc 2011 ;2011:5279-82

BMIT Research Group, School of IT, University of Sydney, Australia.

There exist many viewers for single-modal medical images that are efficient and are equipped with powerful analysis tools. However, there is a distinct lack of efficient image viewers for multi-modality images, particularly for displaying multiple follow-up studies that depict a patient's response to treatment over time. Such viewers would be required to display large amounts of image data. In this study, we present the TAGIGEN viewer--a web-based image viewer designed specifically for the visualisation of multi-modality follow-up studies. We innovate by defining a series of dynamically generated image grid layouts that display sets of related images together in order to improve the ability to compare and assimilate the myriad images. We adopted a web-based client-server image streaming technology, thus enabling interactive navigation of the images in a computationally efficient manner. Furthermore, our web-based approach is interoperable and requires no software installation. We evaluated the ability of our viewer in displaying and understanding a patient's follow-up images in a case study with combined positron emission tomography and computed tomography (PET-CT) follow-up scans. We conducted a usability survey on 10 participants to measure the usefulness of our viewer, used as an outpatient viewer e.g. viewer designed for use by the patients, in tracking a patient's disease state across four PET-CT studies. Our initial results suggest that our viewer was able to efficiently visualise the patient data over time, and that the web-based implementation was fast (loading on average within 5.6 seconds with real-time navigation) and easy to use (overall survey score higher than 4 / 5).
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/IEMBS.2011.6091306DOI Listing
June 2012

A graph-based approach to the retrieval of dual-modality biomedical images using spatial relationships.

Annu Int Conf IEEE Eng Med Biol Soc 2008 ;2008:390-3

Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Info. Tech., University of Sydney, and Department of PET and Nuclear Medicine, Royal Prince Alfred Hospital, Sydney, Australia.

The increasing size of medical image archives and the complexity of medical images have led to the development of medical content-based image retrieval (CBIR) systems. These systems use the visual content of images for image retrieval in addition to conventional textual annotation, and have become a useful technique in biomedical data management. Existing CBIR systems are typically designed for use with single-modality images, and are restricted when multi-modal images, such as co-aligned functional positron emission tomography and anatomical computed tomography (PET/CT) images, are considered. Furthermore, the inherent spatial relationships among adjacent structures in biomedical images are not fully exploited. In this study, we present an innovative retrieval system for dual-modality PET/CT images by proposing the use of graph-based methods to spatially represent the structural relationships within these images. We exploit the co-aligned functional and anatomical information in PET/CT, using attributed relational graphs (ARG) to represent both modalities spatially and applying graph matching for similarity measurements. Quantitative evaluation demonstrated that our dual-modal ARG enabled the CBIR of dual-modality PET/CT. The potential of our dual-modal ARG in clinical application was also explored.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1109/IEMBS.2008.4649172DOI Listing
May 2009
-->