MIPAV|DICOM Comunication| About MIPAV| Sample Images|

1.0 Medical Image Processing, Analysis and Visualization (MIPAV) Link to download MIPAV program. (N.B. Only NIH personell can presently download this. Not yet available to public.)

1.1 Introduction

To support imaging research in the NIH intramural program, we have made major progress in the development of an n-dimensional general-purpose, extensible image processing and visualization program. This application is platform-independent and assists researchers with extracting quantitative information from various medical imaging modalities. MIPAV (Medical Image Processing, Analysis and Visualization) is designed to be both an application and an Application Programming Interface (API) for the development and incorporation of new and innovative image processing, registration, and visualizations. As an application, MIPAV provides the researcher with a ready-made, general-purpose image analysis application to meet the majority of requirements of many researchers. Through the use of plugins the researcher is able to add required specific functionality to meet his/her goals. As an API, MIPAV provides the researcher that has programming resources, the ability to quickly build a specific application to quantify and/or visualize their data. Choosing this method requires some programming experience and image processing knowledge.

1.2 General Overall Design

MIPAV is written using the Java programming language, is modular in design, and takes advantage of Java's object-oriented features. Java is a platform independent, interpreted programming language developed by Sun Microsystems. Platform independence means that a Java application will execute on almost any computer and therefore Java applications are no longer tied to a specific platform or computer architecture.
Internally, MIPAV is designed to store images of any dimensionality (n-dimensional) and thus able to store 2D, 3D, and 4D+ medical image datasets (CT, MRI, fMRI, SPECT, PET, etc.). Examples of 3D medical images include CT and MRI datasets where all three dimensions (x, y, and z) are all spatial dimensions. Other 3D datasets include fluoroscopy volumes where the first two dimensions are spatial and the third dimension is time. While MIPAV provides storage for all types of 3D volumes, a majority of algorithms are designed for application to 3D datasets, where all three dimensions are spatial. However, MIPAV's storage techniques do not preclude developing algorithms or visualization for datasets of any dimensionality.
To directly support the varied image processing needs of the NIH intramural research community, MIPAV reads and writes the following image file types: Raw, TIFF, Analyze, and DICOM 3.0. In addition, MIPAV incorporates a DICOM 3.0 query and retrieve module enabling access to the Multi-modality Radiological Image Processing System (MRIPS) Archive & Retrieval System (MARS). The MARS database stores clinical images collected at the NIH clinical center. Access to these images is essential for researchers in the various institutes to conduct their research. Data read from the various image files are stored in MIPAV's data structure that supports all the basic data types (boolean, byte, unsigned byte, short, unsigned short, integer, float and double).
A significant portion of research at NIH relies on the analysis and quantification of data from medical images. MIPAV provides the tools to let the researcher to automatically, semi-automatically and manually identify and modify volumes of interest (VOIs). Moreover, once an object has been segmented and defined by a VOI, statistics (i.e. volume, average intensity, etc.) of the volume can be calculated. MIPAV supports over 32,000+ unique VOIs. Each VOI can be formed of multiple contours in a single slice or multiple slices. In addition, points on a VOI can be moved or deleted and new points can be easily added via the user interface. MIPAV supports the instantiation, modification, reading, writing, and grouping of VOIs.
Visualization is extremely important and is part of the segmentation process. Multiple types of visualizations are important and range from simple 2D planar displays to the more sophisticated surface and volume rendering. At present MIPAV supports planar display of image datasets including single 2D planar views In addition, MIPAV supports the lightbox view (displaying multiple slices in a single frame) and the tri-planar view (3 orthogonal slices).
Lastly, MIPAV supports a wide range of image-processing algorithms to facilitate the quantification of data from medical images. General image processing algorithms (blurring, gradient magnitude and unsharp masking, etc.) have been implemented which can be used as is or as functional blocks of a more complex processes. More sophisticated segmentation algorithms, including Watershed, Active Contours and more are also available to the user. Below is an outline of major functions of the MIPAV application:

1.2.1 Functionality

1. Image input/output
a. Read/write raw files (big-little endian)
b. Read/write Analyze images
c. Read/write DICOM images (Complete header decoding)
d. DICOM query and retrieve from MARS or PACS
e. Read/write tiff images (8bit, 16bit, indexed, RGB)
f. Read MedVision images


2. Volume Of Interest (VOI)
a. 32K+ unique VOIs
b. Can be contiguous or separate contours
c. Easily modified (move, add, and delete points)
d. Statistics of VOI can be calculated
1. # of pixels,
2. volume/area
3. average/standard deviation of intensity
4. center of mass
5. eccentricity
6. * can be extended to meets specific requirements
e. Line VOIs for intensity profiles and length measurements
f. Affine transformations (translation, rotation, and zoom)
g. Initialization to boundary evolution
h. Read/write VOIs to a file as a list of points where they can be edited with a standard word processor.

3. Image views
a. Magnification of images using pixel replication or bilinear interpolation
b. Standard 2D planar view
c. "Lightbox" all images displayed in a single frame
d. Cine loop (display images like frames in a movie)
e. LUT contrast control with "hot metal", gray, and rainbow LUTs

4. Algorithms
a. Anisotropic diffusion - blurs image but not across edges
b. Boundary evolution - Bspline snake
c. Convolution class - convolves arbitrary kernels (2D & 3D) with an image
d. Flood fill in 2D and 3D
e. Gaussian class - generate a gaussian or derivations of gaussian in each dimension with a user specific standard deviation in each dimension - used to filter images
f. Gaussian blurring - ability to set standard deviation (scale) to control the amount of blurring in each dimension.
g. Gradient magnitude - edges measurements at user defined scales
h. Histograms
i. Laplacian - zero crossings indicate locations of edges
j. Morphological operators (2D and 3D). Erosion, dilation, open, closing and more. More to come in the future.
k. Reslice 3D dataset to isotropic voxels. Linear, cubic, cubic bspline interpolation.
l. Transformation of the image (zoom, translation and rotation).
m. Threshold - single and dual
n. Watershed interactive segmentation method
o. Matrix class - SVD, LUD, eigenvalue solver, invert etc.

5. Image model
a. n-dimensional
b. 4D dataset (3D volumes sampled over time = 4D)
c. able to store: boolean, signed/unsigned byte, signed/unsigned short, int, long, float, double.



2.0 Examples

(1) Ronald Summers, M.D., Ph.D., Department of Radiology, Clinical Center, is presently using MIPAV to support the analysis of renal lesions associated with von Hippel-Lindau Disease in computed tomography (CT) datasets. A semi-interactive optimized watershed segmentation method is used first to segment the diseased kidney. Next, mathematical-morphology operators are applied to the image to fully segment the lesions, and finally the volume and size are automatically calculated. Before the use of MIPAV, it was much too time-consuming to accurately segment the lesions, thus MIPAV has provided needed functionality not readily available from other sources and provides a needed mechanism to promote intramural research.
(2) Melvyn Heyes, Ph.D., Neuro-toxicology Section, Laboratory of Neurotoxicology, National Institute of Mental Health, has used MIPAV to investigate how the immune activation mechanism within the CNS compartment is involved in the pathogenesis of a broad spectrum of neurologic diseases. MIPAV's unique interactive level-set method was used to generate regions of interest (ROIs) of the lesion area in microscopy images of gerbils' striatum in a single user interaction. Statistics of the region were then calculated and help support their hypothesis. Quantification of lesions in past studies might have taken as much as half an hour can now be accomplished under a minute.
(3) Francois Lalonde, Ph.D. and Trey Sunderland, MD, Geriatric Psychiatry Branch, NIMH, have used MIPAV in a number of ways to support their research. First, they use MIPAV's DICOM query and retrieve module to access the MRIPS clinical image database. Next, the skull is segmented in MRI datasets using either watershed or the interactive level-set methods. MIPAV is also important to this group because the Geriatric Psychiatry department has multiple types of computer platforms and MIPAV will execute on all platforms thus saving time and resources.
(4) Michael Collins, MD, Craniofacial and Skeletal Diseases Branch, National Institute of Dental and Craniofacial Research, also uses the DICOM query/retrieve module and the active level-set method to acquire and then segment lesions in CT datasets. Statistics of the segmented regions are generated over time where the longitudinal studies help to determine the efficacy of a particular treatment. The initial results and excellent performance of MIPAV has lead Dr. Collins to include MIPAV as the core application of a worldwide study and will be used by researchers in Italy and Israel.


Future goals

Significant effort has been expended in building a broad based, n-dimensional, general-purpose, extensible image processing application, however much work is still required to meet the wider needs of the NIH user community. At present we are adding important registration algorithms to MIPAV as well as adding an improved orthogonal tri-planar visualization. Future enhancements will also include sophisticated methods of surface and volume rendering. In addition, more sophisticated segmentation algorithms will be researched and implemented. A majority of segmentation algorithms will most likely be addressed at very specific requirements identified by our collaborators and be made available as MIPAV plug-ins. Lastly, a macro language will be added to enable users to build scripts to process a pool of image datasets. A scripting language will allow the application of a repeatable image analysis process, possible composed of many image-processing algorithms, to all the datasets of a particular study. An additional advantage of using macros is that they are simple and available to the end user and thus do not require low-level programming. While MIPAV has primarily addressed clinical imaging modalities it is presently capable of performing basic operations on other datasets including microarrays, microscopy images and micrographs. Future plans include adding the functionality to meet the requirements of the researchers using these technologies. More aggressive goals include utilizing Java's distributed execution capabilities to improve algorithm proformance.
Publications, Presentations and Courses
Summers, RM, Agcaoili, CL, McAuliffe, MJ, Dalal S, Yim P, Choyke, PL, Linehan, M. Helical CT of von Hippel-Lindau: Semi-automated Segmentation of Renal Lesions. Summitted to Int'l Conference on Pattern Recognition.

Dr. McAuliffe is writing a chapter for a book titled "Principles and Practices of Clinical Research" and edited by Dr. Gallin, Director of the Warren G. Magnuson Clinical Center at NIH.

Present section for Dr. Gallin's course titled "Principles and Practices of Clinical Research"

Dr. McAuliffe teaches Image Processing II as part of the CIT computer-training program.