Three-Dimensional Stereoscopic Volume Rendering of Malignant Pleural Mesothelioma
Our objective was to investigate the application of three-dimensional (3D) stereoscopic volume rendering with perceptual colorization on preoperative imaging for malignant pleural mesothelioma. At present, we have prospectively enrolled 6 patients being considered for resection of malignant pleural mesothelioma that have undergone a multidetector-row computed tomography (CT) scan of the chest. The CT data sets were volume rendered without preprocessing. The resultant 3D rendering was displayed stereoscopically and used to provide information regarding tumor extent, morphology, and anatomic involvement. To demonstrate this technique, this information was compared with the corresponding two-dimensional CT grayscale axial images from two of these patients. Three-dimensional stereoscopic reconstructions of the CT data sets provided detailed information regarding the local extent of tumor that could be used for preoperative surgical planning. Three-dimensional stereoscopic volume rendering for malignant pleural mesothelioma is a novel approach. Combined with our innovative perceptual colorization algorithm, stereoscopic volumetric analysis potentially allows for the accurate determination of the extent of pleural mesothelioma with results difficult to duplicate using grayscale, multiplanar CT images.Abstract
Although optimal treatment for malignant pleural mesothelioma (MPM) is still being investigated, multimodality therapy in the form of surgery, chemotherapy, and radiation has been shown to offer a survival advantage over any single therapy modality in appropriately selected patients.1 Surgical resection for patients with MPM can include either extrapleural pneumonectomy (EPP), which is an en bloc resection of the involved pleura, lung, ipsilateral diaphragm, and the pericardium, or radical pleurectomy with decortication (radical P/D), which removes the pleura, all gross tumor, and often the diaphragm and pericardium if there is tumor involvement. The goal of EPP is removal of all visible tumor and is reserved for patients without mediastinal lymph node involvement or chest wall invasion, and the goal of radical P/D is maximal debulking of the tumor with preservation of the lung. The limitations of current two-dimensional (2D) imaging modalities to determine the local extent of disease often result in the underestimation of the tumor extent and the intended surgery being altered at the time of thoracotomy.2 These limitations are due to the unique growth pattern of MPM, which is characterized by growth around and within the pleural cavity to involve the pleura and interlobar fissures and infiltration of the diaphragm, mediastinum, and chest wall. This growth is opposed to the spherical growth pattern of most solid tumors; therefore, the bulk of tumor seen during conventional imaging may not represent the “true” volume of disease present.
Materials and Methods
Technology
We designed our high-performance, three-dimensional (3D) stereoscopic hardware and software infrastructure to interactively volume render biomedical image data, including those originating from scanning transmission electron microscopy and computed tomography (CT) scanners. Our software environment is built on Linux, is open source, and runs on a 9-machine computer cluster built in-house using consumer-grade hardware. It can perform stereoscopic volume reconstruction using DICOM, TIFF, or binary file formats as input. The visualization infrastructure has been used previously for research in presurgical planning3 and medical education and is composed of 5 components:
-
VL3, a scalable, parallel graphics processing unit–based volume-rendering software library. VL3 accepts CT data sets from any medical scanner without specialized protocols or preprocessing. For this application, VL3 loads CT data sets and outputs two 3D volumes that represent the left and right eye virtual perspectives and whose interocular offset simulates binocular stereovision. Loading and display of the rendered volumes took approximately 20 seconds for the CT data sets described here.
-
MedVolViz, a front-end software interface for VL3 that incorporates clinical tools typically found on proprietary radiologic workstations for the manipulation of the 3D volume. These tools include multiplane clipping, rotation, zoom, pan, Hounsfield unit (HU) density-based windowing, voxel volumetrics of user-defined anatomic regions, and customizable color maps, all of which are invoked by the user through an intuitive, easy-to-use interface. MedVolViz further enhances the visualization through the use of a patented perceptual colorization algorithm4 that linearly correlates changes in perceived color luminance with changes in HU scalar density.
-
Lightsaber, a scalable, 9-node, high-performance graphics processing unit computing cluster that distributes VL3's volume-rendering duties among its 8 “task” nodes. Each task node reconstructs one quarter of an eye perspective in parallel. The remaining “master” node assembles each perspective's volume rendering while MedVolViz displays them side-by-side on Lightsaber's attached display.
-
CoWebViz,5 a Web-based, shared visualization distribution and remote control system. It has a server that processes 3D images and streams them over the Internet to a remote client, which is just a standard Web browser. The two 3D volume-rendered eye perspectives produced by Lightsaber were streamed across the University of Chicago campus to a remote computer and viewed using Google Chrome. Generically, the 2 Web browser streams are viewable via unique URLs by any Internet-enabled computer worldwide, thus enabling them to be seen and interactively manipulated by up to 3 virtual collaborators in real time. Performance is bandwidth dependent.
-
Stereoscopic visualization is accomplished by connecting an Internet-enabled computer to a 3D, consumer-grade Samsung television for use as its display. The two 3D volume renderings are fused into one 3D stereoscopic volume by Samsung's native processing. All remote manipulation tools for preoperative surgical planning remain available to the user.
Technique
After Institutional Review Board approval, 3D stereoscopic volume rendering was performed on the CT scans of 6 MPM patients (5 men and 1 woman; ages 49–89 years, median age 70.5 years) who underwent surgical treatment at The University of Chicago Medical Center (Table 1). A total of 2 patients underwent EPP, and 4 patients underwent radical P/D. These patients all underwent conventional CT imaging of the chest an average of 24.7 days (range, 5–59 days) prior to surgery. In two of these patients (patients 1 and 2), 3D stereoscopic rendering was compared with their corresponding axial 2D grayscale CT images in order to demonstrate the ability of this technique to define the tumor and its complex relationship to the surrounding structures within the thoracic cavity. All figures shown were imaged or rendered with a linear opacity ramp, a window level of 200 HU, and a window width of 1500 HU.

Results
The first patient is a 79-year-old man who presented with a chief complaint of 1 month of shortness of breath. A chest X-ray was performed that revealed a large right pleural effusion. A chest CT was subsequently performed, which was suspicious for a pleural-based tumor. Further work-up entailed thoracoscopic drainage, pleural biopsy, and talc pleurodesis. Pathology confirmed MPM with epithelioid histology. The patient was taken for surgery, at which time he underwent radical P/D. The 3D volume-rendered image clearly depicts the tumor and its relationship to the tracheobronchial tree and subcarina. This image also depicts bulky mediastinal disease despite minimal disease in the parietal pleura. When comparing this to the 2D image (Figs. 1A and 1B), one can appreciate the more clearly defined anatomy that the 3D stereoscopic-rendered images offer.



Citation: International Surgery 97, 1; 10.9738/CC66.1
The second patient is a 70-year-old man who presented with symptoms of right-sided pleuritic chest pain. CT scan of the chest (Figs. 2A and 3A) revealed a large right pleural effusion and pleural-based nodules. The patient underwent thoracoscopic drainage and pleural biopsy. After pathologic confirmation of MPM with epithelioid histology, the patient underwent EPP with prosthetic repair of the pericardium and diaphragm. The resulting 3D volume clearly shows bulky tumor within the lung involving the fissure (Fig. 2B). Additionally, extensive involvement of the diaphragm and parietal pleura is observed both in the coronal and sagittal views (Figs. 2B and 3B). When comparing this to the nonrendered images, one can recognize the difficulty in determining the local extent of disease with traditional 2D grayscale images.



Citation: International Surgery 97, 1; 10.9738/CC66.1



Citation: International Surgery 97, 1; 10.9738/CC66.1
Comment
Traditional imaging techniques are often inaccurate for MPM because of its diffuse pattern of growth and the complex shape of the pleural lining. The pleural surface is not a solid organ, and both CT and magnetic resonance imaging (MRI) may have disadvantages in depicting tumors, in differentiating them from adjacent pleural effusion or atelectatic lungs, and thus in determining the T stage of the MPM tumor. The primary imaging modality for assessment of chest wall, mediastinal, and diaphragmatic invasion, as well as the presence or absence of nodal or distant metastases in MPM, has traditionally been CT.2 MRI has been used to supplement CT for potentially resectable disease in instances in which CT findings are equivocal regarding the degree of chest wall invasion and transdiaphragmatic extension. Although MRI has been shown to be more accurate than CT in determining diaphragmatic (82% versus 55%) and chest wall (69% versus 46%) invasion,6 the optimal imaging modality for determining T stage has yet to be determined.
Three-dimensional volume-rendering techniques have been described in the medical literature since the early 1980s.7 The technique has been applied selectively for the purposes of presurgical planning to a wide range of disease processes that require detailed descriptions of anatomic relationships.8,9 The widespread adoption of 3D rendering during presurgical planning for any one disease process, however, has yet to be described. The diffuse infiltrating nature of MPM, along with the close proximity of the lung to the surrounding pleura, pericardium, and diaphragm, make accurate T staging difficult with current imaging modalities. The degree of local invasiveness and mediastinal nodal involvement not only determines the potential resectability of the tumor, but also the type of operation (EPP versus radical P/D) that can be performed.
In this article, we demonstrate how stereoscopic 3D rendering with perceptual colorization is uniquely suited to define MPM and its anatomic relationship to the pleural cavity. Although many CT scanners and commercial software packages are able to produce volume-rendered reconstructions of CT scans, these images fundamentally remain 2D representations, albeit those that provide the human viewer with a perspective not available with the native set of multiplanar (axial, coronal, or sagittal) images. Three-dimensional stereoscopically rendered images, however, present different perspectives of the image data to each eye of the viewer so that the visualized image contains true depth information, which naturally enhances the viewer's interaction and interpretation. Although surgeons are skilled at conceptualizing the 3D relationships of patient anatomy and pathology based on the clinical CT scans, the presentation of 3D stereoscopically rendered images relieves the surgeon of a task that is often challenging, especially with a disease as extensive and invasive as mesothelioma. Perceptual colorization is a necessary complement to 3D stereoscopic imaging's notable increase in complexity and information content. The tool enables surgeons to comprehensively analyze the expanded dynamic density range while concurrently delineating even finer tissue density detail in stereoscopically rendered CT data. It is important to emphasize that this standalone visualization infrastructure robustly allows for the retrospective stereoscopic 3D reconstruction of CT image data sets from any scanner; specialized scanner hardware is not required, and dedicated image-acquisition protocols are not necessary. We plan to prospectively compare the observations from preoperative 3D stereoscopic images with pathology reports and surgical notes to determine the reliability of this technique in predicting extent of disease and choice of surgical procedure. Future work will combine automated tumor volume segmentation10 with 3D stereoscopic visualization and compare these findings with actual volume of the resected tumor, and will evaluate the role of 3D stereoscopic visualization in the assessment of treatment response in the chemotherapy setting.

(A) Two-dimensional static coronal grayscale image depicting involvement of the internal mammary nodes of patient 1. Patient underwent a radical P/D 7 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that clearly depicts the tumor's relationship to the tracheobronchial tree and subcarina. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.

(A) Two-dimensional static coronal grayscale image depicting tumor involvement of the fissure and diaphragmatic parietal pleura of patient 2. Patient underwent EPP 30 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that elaborates the tumor's bulk and orientation within the lung involving the fissure. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.

(A) Additional 2D static sagittal grayscale image of patient shown in Fig. 2A. (B) Corresponding 3D volume rendering of Fig. 2B. Accurate stereoscopic representation of volume rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.
Contributor Notes