Editorial Type:
Article Category: Research Article
 | 
Online Publication Date: 01 Jan 2012

Three-Dimensional Stereoscopic Volume Rendering of Malignant Pleural Mesothelioma

,
,
,
,
,
,
,
, and
Page Range: 65 – 70
DOI: 10.9738/CC66.1
Save
Download PDF

Abstract

Our objective was to investigate the application of three-dimensional (3D) stereoscopic volume rendering with perceptual colorization on preoperative imaging for malignant pleural mesothelioma. At present, we have prospectively enrolled 6 patients being considered for resection of malignant pleural mesothelioma that have undergone a multidetector-row computed tomography (CT) scan of the chest. The CT data sets were volume rendered without preprocessing. The resultant 3D rendering was displayed stereoscopically and used to provide information regarding tumor extent, morphology, and anatomic involvement. To demonstrate this technique, this information was compared with the corresponding two-dimensional CT grayscale axial images from two of these patients. Three-dimensional stereoscopic reconstructions of the CT data sets provided detailed information regarding the local extent of tumor that could be used for preoperative surgical planning. Three-dimensional stereoscopic volume rendering for malignant pleural mesothelioma is a novel approach. Combined with our innovative perceptual colorization algorithm, stereoscopic volumetric analysis potentially allows for the accurate determination of the extent of pleural mesothelioma with results difficult to duplicate using grayscale, multiplanar CT images.

Although optimal treatment for malignant pleural mesothelioma (MPM) is still being investigated, multimodality therapy in the form of surgery, chemotherapy, and radiation has been shown to offer a survival advantage over any single therapy modality in appropriately selected patients.1 Surgical resection for patients with MPM can include either extrapleural pneumonectomy (EPP), which is an en bloc resection of the involved pleura, lung, ipsilateral diaphragm, and the pericardium, or radical pleurectomy with decortication (radical P/D), which removes the pleura, all gross tumor, and often the diaphragm and pericardium if there is tumor involvement. The goal of EPP is removal of all visible tumor and is reserved for patients without mediastinal lymph node involvement or chest wall invasion, and the goal of radical P/D is maximal debulking of the tumor with preservation of the lung. The limitations of current two-dimensional (2D) imaging modalities to determine the local extent of disease often result in the underestimation of the tumor extent and the intended surgery being altered at the time of thoracotomy.2 These limitations are due to the unique growth pattern of MPM, which is characterized by growth around and within the pleural cavity to involve the pleura and interlobar fissures and infiltration of the diaphragm, mediastinum, and chest wall. This growth is opposed to the spherical growth pattern of most solid tumors; therefore, the bulk of tumor seen during conventional imaging may not represent the “true” volume of disease present.

Materials and Methods

Technology

We designed our high-performance, three-dimensional (3D) stereoscopic hardware and software infrastructure to interactively volume render biomedical image data, including those originating from scanning transmission electron microscopy and computed tomography (CT) scanners. Our software environment is built on Linux, is open source, and runs on a 9-machine computer cluster built in-house using consumer-grade hardware. It can perform stereoscopic volume reconstruction using DICOM, TIFF, or binary file formats as input. The visualization infrastructure has been used previously for research in presurgical planning3 and medical education and is composed of 5 components:

  1. VL3, a scalable, parallel graphics processing unit–based volume-rendering software library. VL3 accepts CT data sets from any medical scanner without specialized protocols or preprocessing. For this application, VL3 loads CT data sets and outputs two 3D volumes that represent the left and right eye virtual perspectives and whose interocular offset simulates binocular stereovision. Loading and display of the rendered volumes took approximately 20 seconds for the CT data sets described here.

  2. MedVolViz, a front-end software interface for VL3 that incorporates clinical tools typically found on proprietary radiologic workstations for the manipulation of the 3D volume. These tools include multiplane clipping, rotation, zoom, pan, Hounsfield unit (HU) density-based windowing, voxel volumetrics of user-defined anatomic regions, and customizable color maps, all of which are invoked by the user through an intuitive, easy-to-use interface. MedVolViz further enhances the visualization through the use of a patented perceptual colorization algorithm4 that linearly correlates changes in perceived color luminance with changes in HU scalar density.

  3. Lightsaber, a scalable, 9-node, high-performance graphics processing unit computing cluster that distributes VL3's volume-rendering duties among its 8 “task” nodes. Each task node reconstructs one quarter of an eye perspective in parallel. The remaining “master” node assembles each perspective's volume rendering while MedVolViz displays them side-by-side on Lightsaber's attached display.

  4. CoWebViz,5 a Web-based, shared visualization distribution and remote control system. It has a server that processes 3D images and streams them over the Internet to a remote client, which is just a standard Web browser. The two 3D volume-rendered eye perspectives produced by Lightsaber were streamed across the University of Chicago campus to a remote computer and viewed using Google Chrome. Generically, the 2 Web browser streams are viewable via unique URLs by any Internet-enabled computer worldwide, thus enabling them to be seen and interactively manipulated by up to 3 virtual collaborators in real time. Performance is bandwidth dependent.

  5. Stereoscopic visualization is accomplished by connecting an Internet-enabled computer to a 3D, consumer-grade Samsung television for use as its display. The two 3D volume renderings are fused into one 3D stereoscopic volume by Samsung's native processing. All remote manipulation tools for preoperative surgical planning remain available to the user.

Technique

After Institutional Review Board approval, 3D stereoscopic volume rendering was performed on the CT scans of 6 MPM patients (5 men and 1 woman; ages 49–89 years, median age 70.5 years) who underwent surgical treatment at The University of Chicago Medical Center (Table 1). A total of 2 patients underwent EPP, and 4 patients underwent radical P/D. These patients all underwent conventional CT imaging of the chest an average of 24.7 days (range, 5–59 days) prior to surgery. In two of these patients (patients 1 and 2), 3D stereoscopic rendering was compared with their corresponding axial 2D grayscale CT images in order to demonstrate the ability of this technique to define the tumor and its complex relationship to the surrounding structures within the thoracic cavity. All figures shown were imaged or rendered with a linear opacity ramp, a window level of 200 HU, and a window width of 1500 HU.

Table 1 Patient characteristics
Table 1

Results

The first patient is a 79-year-old man who presented with a chief complaint of 1 month of shortness of breath. A chest X-ray was performed that revealed a large right pleural effusion. A chest CT was subsequently performed, which was suspicious for a pleural-based tumor. Further work-up entailed thoracoscopic drainage, pleural biopsy, and talc pleurodesis. Pathology confirmed MPM with epithelioid histology. The patient was taken for surgery, at which time he underwent radical P/D. The 3D volume-rendered image clearly depicts the tumor and its relationship to the tracheobronchial tree and subcarina. This image also depicts bulky mediastinal disease despite minimal disease in the parietal pleura. When comparing this to the 2D image (Figs. 1A and 1B), one can appreciate the more clearly defined anatomy that the 3D stereoscopic-rendered images offer.

Fig. 1. (A) Two-dimensional static coronal grayscale image depicting involvement of the internal mammary nodes of patient 1. Patient underwent a radical P/D 7 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that clearly depicts the tumor's relationship to the tracheobronchial tree and subcarina. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.Fig. 1. (A) Two-dimensional static coronal grayscale image depicting involvement of the internal mammary nodes of patient 1. Patient underwent a radical P/D 7 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that clearly depicts the tumor's relationship to the tracheobronchial tree and subcarina. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.Fig. 1. (A) Two-dimensional static coronal grayscale image depicting involvement of the internal mammary nodes of patient 1. Patient underwent a radical P/D 7 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that clearly depicts the tumor's relationship to the tracheobronchial tree and subcarina. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.
Fig. 1 (A) Two-dimensional static coronal grayscale image depicting involvement of the internal mammary nodes of patient 1. Patient underwent a radical P/D 7 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that clearly depicts the tumor's relationship to the tracheobronchial tree and subcarina. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.

Citation: International Surgery 97, 1; 10.9738/CC66.1

The second patient is a 70-year-old man who presented with symptoms of right-sided pleuritic chest pain. CT scan of the chest (Figs. 2A and 3A) revealed a large right pleural effusion and pleural-based nodules. The patient underwent thoracoscopic drainage and pleural biopsy. After pathologic confirmation of MPM with epithelioid histology, the patient underwent EPP with prosthetic repair of the pericardium and diaphragm. The resulting 3D volume clearly shows bulky tumor within the lung involving the fissure (Fig. 2B). Additionally, extensive involvement of the diaphragm and parietal pleura is observed both in the coronal and sagittal views (Figs. 2B and 3B). When comparing this to the nonrendered images, one can recognize the difficulty in determining the local extent of disease with traditional 2D grayscale images.

Fig. 2. (A) Two-dimensional static coronal grayscale image depicting tumor involvement of the fissure and diaphragmatic parietal pleura of patient 2. Patient underwent EPP 30 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that elaborates the tumor's bulk and orientation within the lung involving the fissure. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.Fig. 2. (A) Two-dimensional static coronal grayscale image depicting tumor involvement of the fissure and diaphragmatic parietal pleura of patient 2. Patient underwent EPP 30 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that elaborates the tumor's bulk and orientation within the lung involving the fissure. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.Fig. 2. (A) Two-dimensional static coronal grayscale image depicting tumor involvement of the fissure and diaphragmatic parietal pleura of patient 2. Patient underwent EPP 30 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that elaborates the tumor's bulk and orientation within the lung involving the fissure. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.
Fig. 2 (A) Two-dimensional static coronal grayscale image depicting tumor involvement of the fissure and diaphragmatic parietal pleura of patient 2. Patient underwent EPP 30 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that elaborates the tumor's bulk and orientation within the lung involving the fissure. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.

Citation: International Surgery 97, 1; 10.9738/CC66.1

Fig. 3. (A) Additional 2D static sagittal grayscale image of patient shown in Fig. 2A. (B) Corresponding 3D volume rendering of Fig. 2B. Accurate stereoscopic representation of volume rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.Fig. 3. (A) Additional 2D static sagittal grayscale image of patient shown in Fig. 2A. (B) Corresponding 3D volume rendering of Fig. 2B. Accurate stereoscopic representation of volume rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.Fig. 3. (A) Additional 2D static sagittal grayscale image of patient shown in Fig. 2A. (B) Corresponding 3D volume rendering of Fig. 2B. Accurate stereoscopic representation of volume rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.
Fig. 3 (A) Additional 2D static sagittal grayscale image of patient shown in Fig. 2A. (B) Corresponding 3D volume rendering of Fig. 2B. Accurate stereoscopic representation of volume rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.

Citation: International Surgery 97, 1; 10.9738/CC66.1

Comment

Traditional imaging techniques are often inaccurate for MPM because of its diffuse pattern of growth and the complex shape of the pleural lining. The pleural surface is not a solid organ, and both CT and magnetic resonance imaging (MRI) may have disadvantages in depicting tumors, in differentiating them from adjacent pleural effusion or atelectatic lungs, and thus in determining the T stage of the MPM tumor. The primary imaging modality for assessment of chest wall, mediastinal, and diaphragmatic invasion, as well as the presence or absence of nodal or distant metastases in MPM, has traditionally been CT.2 MRI has been used to supplement CT for potentially resectable disease in instances in which CT findings are equivocal regarding the degree of chest wall invasion and transdiaphragmatic extension. Although MRI has been shown to be more accurate than CT in determining diaphragmatic (82% versus 55%) and chest wall (69% versus 46%) invasion,6 the optimal imaging modality for determining T stage has yet to be determined.

Three-dimensional volume-rendering techniques have been described in the medical literature since the early 1980s.7 The technique has been applied selectively for the purposes of presurgical planning to a wide range of disease processes that require detailed descriptions of anatomic relationships.8,9 The widespread adoption of 3D rendering during presurgical planning for any one disease process, however, has yet to be described. The diffuse infiltrating nature of MPM, along with the close proximity of the lung to the surrounding pleura, pericardium, and diaphragm, make accurate T staging difficult with current imaging modalities. The degree of local invasiveness and mediastinal nodal involvement not only determines the potential resectability of the tumor, but also the type of operation (EPP versus radical P/D) that can be performed.

In this article, we demonstrate how stereoscopic 3D rendering with perceptual colorization is uniquely suited to define MPM and its anatomic relationship to the pleural cavity. Although many CT scanners and commercial software packages are able to produce volume-rendered reconstructions of CT scans, these images fundamentally remain 2D representations, albeit those that provide the human viewer with a perspective not available with the native set of multiplanar (axial, coronal, or sagittal) images. Three-dimensional stereoscopically rendered images, however, present different perspectives of the image data to each eye of the viewer so that the visualized image contains true depth information, which naturally enhances the viewer's interaction and interpretation. Although surgeons are skilled at conceptualizing the 3D relationships of patient anatomy and pathology based on the clinical CT scans, the presentation of 3D stereoscopically rendered images relieves the surgeon of a task that is often challenging, especially with a disease as extensive and invasive as mesothelioma. Perceptual colorization is a necessary complement to 3D stereoscopic imaging's notable increase in complexity and information content. The tool enables surgeons to comprehensively analyze the expanded dynamic density range while concurrently delineating even finer tissue density detail in stereoscopically rendered CT data. It is important to emphasize that this standalone visualization infrastructure robustly allows for the retrospective stereoscopic 3D reconstruction of CT image data sets from any scanner; specialized scanner hardware is not required, and dedicated image-acquisition protocols are not necessary. We plan to prospectively compare the observations from preoperative 3D stereoscopic images with pathology reports and surgical notes to determine the reliability of this technique in predicting extent of disease and choice of surgical procedure. Future work will combine automated tumor volume segmentation10 with 3D stereoscopic visualization and compare these findings with actual volume of the resected tumor, and will evaluate the role of 3D stereoscopic visualization in the assessment of treatment response in the chemotherapy setting.

Acknowledgments

The 3D stereoscopic visualization infrastructure was developed by the University of Chicago's Computation Institute and Department of Surgery in collaboration with the Futures Laboratory at Argonne National Laboratory. There was no additional funding necessary to perform the study. The authors performed free and independent development and evaluation of this new technology and had full control of the design of the study, methods used, outcome measurements, analysis of data, and production of the written report. S.G.A. receives royalties and licensing fees through the University of Chicago related to computer-aided diagnosis. The remaining authors have no financial relationship to companies producing stereoscopic rendering software or hardware.

References

  • 1
    Sugarbaker, DJ
    ,
    FloresRM
    ,
    JaklitschMT
    ,
    RichardsWG
    ,
    StraussGM
    ,
    CorsonJM
    , et al
    . Resection margins, extrapleural nodal status, and cell type determine postoperative long-term survival in trimodality therapy of malignant pleural mesothelioma: results in 183 patients.J Thorac Cardiovasc Surg1999. 117 (
    1
    ):5465.
  • 2
    Marom, EM
    ,
    ErasmusJJ
    ,
    PassHI
    , and
    PatzEFJr
    . The role of imaging in malignant pleural mesothelioma.Semin Oncol2002. 29 (
    1
    ):2635.
  • 3
    Parsad, NM
    ,
    MillisM
    ,
    VannierM
    ,
    DechF
    ,
    OlsonE
    ,
    PapkaME
    , and
    SilversteinJC
    . Real-time automated hepatic explant volumetrics on a stereoscopic three-dimensional parallelized visualization cluster.Int J CARS2007. 2 (
    S1
    ):S488.
  • 4
    Silverstein, JC
    ,
    ParsadNM
    , and
    TsirlineV
    . Automatic perceptual color map generation for realistic volume visualization.J Biomed Inform2008. 41 (
    6
    ):927935.
  • 5
    Kaspar, M
    ,
    ParsadNM
    , and
    SilversteinJC
    . CoWebViz: Interactive Collaborative Sharing of 3D Stereoscopic Visualization Among Browsers With No Added Software: Proceedings of the 1st ACM International Health Informatics Symposium.
    Arlington, VA
    . 2010. pp. 809836.
  • 6
    Heelan, RT
    ,
    RuschVW
    ,
    BeggCB
    ,
    PanicekDM
    ,
    CaravelliJF
    , and
    EisenC
    . Staging of malignant pleural mesothelioma: comparison of CT and MR imaging.AJR Am J Roentgenol1999. 172 (
    4
    ):10391047.
  • 7
    Knapp, RH
    ,
    VannierMW
    , and
    MarshJL
    . Generation of three dimensional images from CT scans: technological perspective.Radiol Technol1985. 56 (
    6
    ):391398.
  • 8
    Endo, I
    ,
    ShimadaH
    ,
    SugitaM
    ,
    FujiiY
    ,
    MoriokaD
    ,
    TakedaK
    , et al
    . Role of three-dimensional imaging in operative planning for hilar cholangiocarcinoma.Surgery2007. 142 (
    5
    ):666675.
  • 9
    Yamanaka, J
    ,
    SaitoS
    , and
    FujimotoJ
    . Impact of preoperative planning using virtual segmental volumetry on liver resection for hepatocellular carcinoma.World J Surg2007. 31 (
    6
    ):12491255.
  • 10
    Sensakovic, WF
    ,
    ArmatoSGIII
    ,
    StrausC
    ,
    RobertsRY
    ,
    CaligiuriP
    ,
    StarkeyA
    , and
    KindlerHL
    . Computerized segmentation and measurement of malignant pleural mesothelioma.Med Phys2011. 38 (
    1
    ):238244.
Copyright: International College of Surgeons
Fig. 1
Fig. 1

(A) Two-dimensional static coronal grayscale image depicting involvement of the internal mammary nodes of patient 1. Patient underwent a radical P/D 7 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that clearly depicts the tumor's relationship to the tracheobronchial tree and subcarina. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.


Fig. 2
Fig. 2

(A) Two-dimensional static coronal grayscale image depicting tumor involvement of the fissure and diaphragmatic parietal pleura of patient 2. Patient underwent EPP 30 days after imaging. (B) Corresponding orientation in a dynamically clipped, perceptually colorized, 3D volume that elaborates the tumor's bulk and orientation within the lung involving the fissure. Accurate stereoscopic representation of volume-rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.


Fig. 3
Fig. 3

(A) Additional 2D static sagittal grayscale image of patient shown in Fig. 2A. (B) Corresponding 3D volume rendering of Fig. 2B. Accurate stereoscopic representation of volume rendered anatomy is beyond the scope of print media because anaglyph techniques require passive 3D glasses for viewing and also distort image color.


Contributor Notes

Reprint requests: Wickii T. Vigneswaran, MD, FACS, The University of Chicago Medical Center, 5841 South Maryland Avenue, MC 5040, Chicago, IL 60637-1470, 773 795 1267, 773 702 4187; E-mail: wvignesw@surgery.bsd.uchicago.edu
  • Download PDF