Multimodal display techniques with application to breast imaging

Kimberly Rafferty
Karl Baum
Evan Schimdt
Andrzej Krol

TASS members have supported this project with their own time, energy and money; no tax dollars were spent on our work. Download the associated zip file for a fully interactive version of this document, referenced content, and high resolution images. Note: imported from RIT’s Digital Media Library running on DSpace to RIT Scholar Works in February 2013.


Application of a multimodality approach is advantageous for detection, diagnosis and management of breast cancer. In this context, F-18-FDG positron emission tomography (PET), and high-resolution and dynamic contrast-enhanced magnetic resonance imaging (MRI) have steadily gained clinical acceptance. Obtaining the spatial relationships between these modalities and conveying them to the observer maximizes the benefit that can be achieved. Traditionally the registered images are displayed side by side. However, it is believed that a combined MRI/PET display may be more beneficial. The advantage of a combined image lies in our inability to visually judge spatial relationships between images when they are viewed side by side. The process of combining the MRI and PET 3D images into a single 3D image is called image fusion. Color tables were defined for the fusion of MRI/PET images. This included color tables, which satisfy specific requirements, that were generated by a previously developed genetic algorithm. Radiologists were asked to evaluate images created using the selected fusion-for-visualization techniques. The study determined radiologists’ preference, ease of use, understanding, efficiency, and accuracy when reading images using each technique. The data studied, the data collected, the applications used to administer the study and analyze the results, and the processed results are provided through this interactive document.