Minimally invasive endoscope based abdominal procedures provide potential advantages over conventional open surgery such as reduced trauma, shorter hospital stay, and quick recovery. One major limitation of using this technique is the narrow view of the endoscope and the lack of proper 3D context of the surgical site. In this paper, we propose a rapid and accurate method to align intraoperative stereo endoscopic images of the surgical site with preoperative Magnetic Resonance (MR) images. Gridline light pattern is projected on the surgical site to facilitate the registration. The purpose of this surface-based registration is to provide 3D context of the surgical site to the endoscopic view. We have validated the proposed method on a liver phantom and achieved the surface registration error of ？mm. 1. Introduction In this paper, we develop a new method for endoscopy-MR image fusion of the liver organ for minimally invasive endoscope based surgery. Image guidance is an essential tool in minimally invasive endoscope based abdominal procedures [1]. Effective image guidance can compensate the restricted perception during the operation, which is considered a major limitation in endoscopic procedures. Without image guidance, the surgeon cannot see through the surface of the operation site and may accidentally cause damages to the critical structures of the patient. A typical procedure in image guidance is to map pre-operative high quality MR images to intra-operative endoscopic video images, or the patient thereby provides a good quality context to the real-time endoscopic images. Thus, the surgeon will be able to visually access the operation site during the procedure. As a result, the damage to the critical organs or tissues will be substantially minimized. Fusion of endoscopic video images with high quality MR images requires good match of these two modalities. In this paper, we adopt a surface based image fusion because the two modalities are different in acquisition and nature [2, 3]. In order to find the corresponding 3D surface model from endoscopic images, we utilize stereovision to snapshot the surgical site from two different angles and compute the 3D location by using triangulation [4]. Cameras are calibrated before triangulation is used [5, 6]. Although a liver phantom is used to validate the proposed technique, our method is not restricted to the liver surgery. The integrated image guidance can also be applied to other endoscopic procedures. This paper is organized as follows. Section 2 introduces the experimental setup and the camera calibration
References
[1]
M. Baumhauer, M. Feuerstein, H.-P. Meinzer, and J. Rassweiler, “Navigation in endoscopic soft tissue surgery: perspectives and limitations,” Journal of Endourology, vol. 22, no. 4, pp. 751–766, 2008.
[2]
M. Sonka, V. Hlavac, and B. Roger, Image Processing Analysis and Machine Vision, 3rd edition, 2008.
[3]
J. L. Prince and J. M. Links, Medical Imaging Signals and Systems, Printice Hall, Upper Saddle River, New Jersey, NJ, USA, 2006.
[4]
X. Maurice, C. Albitar, C. Doignon, and M. De Mathelin, “A structured light-based laparoscope with real-time organs' surface reconstruction for minimally invasive surgery,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC '12), pp. 5769–55772, 2012.
[5]
M. Feuerstein, T. Mussack, S. M. Heining, and N. Navab, “Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection,” IEEE Transactions on Medical Imaging, vol. 27, no. 3, pp. 355–369, 2008.
[6]
V. Bevilacqua, S. Cambo, L. Cariello, and G. Mastronardi, “A combined method to detect retinal fundus features,” in European Conference on Emergent Aspects In Clinical Data Analysis, pp. 1–6, Pisa, Italy, 2005.
[7]
J. Y. Bouguet, Camera Calibration Toolbox for Matlab, http://www.vision.caltech.edu/bouguetj, 2010.
[8]
D. Dey, D. G. Gobbi, P. J. Slomka, K. J. M. Surry, and T. M. Peters, “Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: creating stereoscopic panoramas,” IEEE Transactions on Medical Imaging, vol. 21, no. 1, pp. 23–30, 2002.
[9]
P. J. Besl and N. D. McKay, “A method for registration of 3-D shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239–256, 1992.
[10]
L. W. Clements, W. C. Chapman, B. M. Dawant, R. L. Galloway Jr., and M. I. Miga, “Robust surface registration using salient anatomical features for image-guided liver surgery: algorithm and validation,” Medical Physics, vol. 35, no. 6, pp. 2528–2540, 2008.
[11]
L. Maier-Hein, M. Schmidt, A. M. Franz et al., “Accounting for anisotropic noise in fine registration of time-of-flight range data with high-resolution surface data,” Medical Image Computing and Computer-Assisted Intervention, vol. 6361, no. 1, pp. 251–258, 2010.
[12]
A. Groch, A. Seitel, S. Hempel, et al., “3D surface reconstruction for laparo-scopic computer-assisted interventions: comparison of state-of-the-art methods,” in Proceedings of the Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, vol. 7964, 2011.
[13]
M. Hayashibe, N. Suzuki, and Y. Nakamura, “Laser-scan endoscope system for intraoperative geometry acquisition and surgical robot safety management,” Medical Image Analysis, vol. 10, no. 4, pp. 509–519, 2006.
[14]
M. Proesmans, L. J. Van Gool, and A. J. Oosterlinck, “Active acquisition of 3D shape for moving objects,” in Proceedings of the IEEE International Conference on Image Processing (ICIP '96), pp. 647–650, September 1996.
[15]
M. Hu, G. Penney, P. Edwards, and D. Hawkes, “3D reconstruction of internal organ surfaces for minimal invasive surgery,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 68–77, Brisbane, Australia, 2007.
[16]
W. W. Lau, N. A. Ramey, J. J. Corso, N. V. Thakor, and G. D. Hager, “Stereo-based endoscopie tracking of cardiac surface deformation,” in Proceedings of the 7th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2004), pp. 494–501, St Malo, France, September 2004.
[17]
D. Stoyanov, A. Darzi, and G. Z. Yang, “Dense 3D depth recovery for soft tissue deformation during robotically assisted laparoscopic surgery,” in Proceedings of the 7th International Conference Medical Image Computing and Computer-Assisted Intervention (MICCAI '04), pp. 41–48, St Malo, France, September 2004.
[18]
M. Hayashibe, N. Suzuki, and Y. Nakamura, “Laser-scan endoscope system for intraoperative geometry acquisition and surgical robot safety management,” Medical Image Analysis, vol. 10, no. 4, pp. 509–519, 2006.
[19]
J. Salvi, J. Pagès, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recognition, vol. 37, no. 4, pp. 827–849, 2004.
[20]
A. Bhatti, Current Advancements in Stereo Vision, InTech, 2012.
[21]
J. Salvi, S. Fernandez, T. Pribanic, and X. Llado, “A state of the art in structured light patterns for surface profilometry,” Pattern Recognition, vol. 43, no. 8, pp. 2666–2680, 2010.
[22]
P. Mountney, D. Stoyanov, and G.-Z. Yang, “Three-dimensional tissue deformation recovery and tracking,” IEEE Signal Processing Magazine, vol. 27, no. 4, pp. 14–24, 2010.
[23]
D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.
