Our purpose in this article is to superimpose a 3D model of the liver, its vessels and tumors (re... more Our purpose in this article is to superimpose a 3D model of the liver, its vessels and tumors (reconstructed from CT images) on external video images of the patient for hepatic surgery guidance. The main constraints are the robustness, the accuracy and the computation time. Because of the absence of visible anatomical landmarks and of the “ cylindrical” shape of the upper abdomen, we used some radio-opaque fiducials. The classical least-squares method assuming that there is no noise on the 3D point positions, we designed a new Maximum Likelihood approach to account for this existing noise and we show that it generalizes the classical approaches. Experiments on synthetic data provide evidences that our new criterion is up to 20% more accurate and much more robust, while keeping a computation time compatible with realtime at 20 to 40 Hz. Eventually, careful validation experiments on real data show that an accuracy of 2 mm can be achieved within the liver.
IEEE Transactions on Robotics and Automation, 2003
This paper presents a robotic vision system that automatically retrieves and positions surgical i... more This paper presents a robotic vision system that automatically retrieves and positions surgical instruments during robotized laparoscopic surgical operations. The instrument is mounted on the end-effector of a surgical robot which is controlled by visual servoing. The goal of the automated task is to safely bring the instrument at a desired 3-D location from an unknown or hidden position. Light Emitting Diodes are attached on the tip of the instrument and a speci£c instrument-holder £tted with optical £bers is used to project laser dots on the surface of the organs. These optical markers are detected in the endoscopic image and allow to localize the instrument with respect to the scene. The instrument is recovered and centered in the image plane by means of a visual servoing algorithm using feature errors in the image. With this system, the surgeon can specify a desired relative position between the instrument and the pointed organ. The relationship between the velocity screw of the surgical instrument and the velocity of the markers in the image is estimated online and, for safety reasons, a multi-stages servoing scheme is proposed. Our approach has been successfully validated in a real surgical environment by performing experiments on living tissues in the surgical training room of IRCAD.
This paper presents a robotic vision system that automatically retrieves and positions surgical i... more This paper presents a robotic vision system that automatically retrieves and positions surgical instruments in robotized laparoscopic surgery. The surgical instrument is mounted on the end-effector of a surgical robot which can be controlled by automatic visual feedback. The goal of the automated task is to bring the instrument at a desired location from an unknown or hidden position. To achieve this task, a special instrument-holder is designed with optical fibers and collimators. This instrumentholder projects laser dot patterns onto the organ surface which are seen in the endoscopic images. Then, the instrument is retrieved and centered in the image plane using a visual servoing algorithm. With this system, the surgeon can also specify a desired position for the instrument in the image. Our approach is successfully validated in a real surgical environment by performing experiments on living animals in the surgical training room of IRCAD.
We provided in [14] an augmented reality guidance system for liver punctures, which has been vali... more We provided in [14] an augmented reality guidance system for liver punctures, which has been validated on a static abdominal phantom [16]. In this paper, we report the first in vivo experiments. We developed a strictly passive protocol to directly evaluate our system on patients. We show that the system algorithms work efficiently and we highlight the clinical constraints that we had to overcome (small operative field, weight and sterility of the tracked marker attached to the needle...). Finally, we investigate to what extent breathing motion can be neglected for free breathing patient. Results show that the guiding accuracy, close to 1 cm, is sufficient for large targets only (above 3 cm of diameter) when the breathing motion is neglected. In the near future, we aim at validating our system on smaller targets using a respiratory gating technique.
In this paper, we propose a method based on multiple criteria to assist physicians in planning pe... more In this paper, we propose a method based on multiple criteria to assist physicians in planning percutaneous RFA on liver. We explain how we extracted information from literature and interviews with radiologists, and formalized them into geometric constraints. We expose then our method to compute the most suitable needle insertion in two steps: computation of authorized insertion zones and multi-criteria optimization of the trajectory within this zones. We focus on the combination of the criteria to optimize and on the optimization step.
Medical Image Analysis plays a crucial role in the diagnosis, planning, control and follow-up of ... more Medical Image Analysis plays a crucial role in the diagnosis, planning, control and follow-up of therapy. To be combined efficiently with medical robotics, Medical Image Analysis can be supported by the development of specific computational models of the human body operating at various levels. We describe in this article a hierarchy of these computational models, including the geometrical, physical and physiological levels, and illustrate their potential use in a number of advanced medical applications including image guided, robot-assisted and simulated medical interventions. We conclude this article with scientific perspectives.
IEEE Transactions on Control Systems Technology, 2006
... Page 7. GANGLOFF et al.: MODEL PREDICTIVE CONTROL FOR COMPENSATION OF CYCLIC ORGAN MOTIONS 24... more ... Page 7. GANGLOFF et al.: MODEL PREDICTIVE CONTROL FOR COMPENSATION OF CYCLIC ORGAN MOTIONS 241 Fig. ... The linearized model of the visual loop is given by the bloc diagram in Fig. 10. ... This is a strong assumption when doing direct torque control. ...
In this paper we present a guiding system for laparoscopic surgery which tracks in real time the ... more In this paper we present a guiding system for laparoscopic surgery which tracks in real time the laparoscopic tools and registers at 10 Hz a preoperative CT/MRI model of the patient. To help surgeons, this system provides simultaneously three kinds of supplementary information. It displays the relative positions of the laparoscopic tools and the 3D patient model. It provides two external views of the patient on which his 3D model and tool positions are superimposed. Finally, it displays an endoscopic view augmented with the 3D patient model. Contrary to most systems for laparoscopic surgery, this one is low cost (one PC, two cameras and a printer are necessary) and we have rigorously validated its tracking accuracy in simulated clinical conditions (tracking accuracy of an instrument tip close to 1.5 mm, and endoscopic overlay error under 1.0 mm). Besides, our system accuracy is comparable with commercial system's ones. Eventually, experiments on an abdominal phantom with surgeons demonstrate its usefulness. We plan to realize our first experiments on patients in a few months.
Medical image processing led to a major improvement of patient care: the 3D modeling of patients ... more Medical image processing led to a major improvement of patient care: the 3D modeling of patients from their CTscan or MRI provides an improved surgical planning and simulation allows to train the surgical gesture before carrying it out. These two preoperative steps can be used intraoperatively with the development of augmented reality (AR). In this paper, we present the tools we developed to provide our £rst prototypal AR guiding system for abdominal surgery.
In this paper, we present a new point of view for the conception of what a teleoperated robotic d... more In this paper, we present a new point of view for the conception of what a teleoperated robotic device should be for percutaneous intervention with computed tomography guidance. We focus our attention on very common and widely used medical acts : the radiofrequency ablation and the biopsy procedure. These acts are widely used in manual interventional radiology, but they come up with others problems. First, the guidance precision is limited for a human, second, the radiologist risks its health by being exposed to X rays during each operation. The robotic system we describe is original by its conception and its control feedback (based on force feedback and vision). A prototype is presented, together with a complete workflow of the system. Some results present the first data acquired with the force feedback sensors.
To improve the planning of hepatic surgery, we have developed a fully automatic anatomical, patho... more To improve the planning of hepatic surgery, we have developed a fully automatic anatomical, pathological and functional segmentation of the liver derived from a spiral CT scan. Materials and methods : From a 2mm thick enhanced spiral CT scan, a first stage automatically delineates skin, bones, lungs, kidneys and spleen, by combining the use of thresholding, mathematical morphology and distance maps. Next, a reference 3D model is immerged in the image and automatically deformed to liver contours. Then an automatic gaussians fitting on the imaging histogram estimates the intensities of parenchyma, vessels and lesions. This first result is next improved through an original topological and geometrical analysis, providing an automatic delineation of lesions and veins. Finally, a topological and geometrical analysis based on medical knowledge provides hepatic functional information invisible in medical imaging: portal vein labeling and hepatic anatomical segmentation according to the Couinaud classification. Results: Clinical validation performed on more than 30 patients shows that this method's delineation of anatomical structures is often more sensitive and more specific than manual delineation by a radiologist. Conclusion: This study describes the methodology used to create the automatic segmentation of the liver with delineation of important anatomical, pathological and functional structures from a routine CT scan. Using the methods proposed in this study, we have confirmed the accuracy and utility of the creation of 3 -dimensional liver model when compared with the conventional reading of the CT scan by a radiologist. This work, may allow an improvement in preoperative planning of hepatic surgery by more p recisely delineating liver pathology and its relation to normal hepatic structures. In the future this data may be integrated with computer-assisted surgery and thus represents a first step towards the development of an augmented reality surgical system.
Percutaneous radiofrequency ablation has become a frequently used technique for the treatment of ... more Percutaneous radiofrequency ablation has become a frequently used technique for the treatment of liver cancers, but still remains very difficult to plan. In this paper, we propose a robust method to delineate on the skin of a 3D reconstructed patient the zones that are candidate for an insertion, because they allow a safe access to the tumor without meeting any organ, and to compute automatically within these zones an optimal trajectory minimizing the volume of necrosis covering the tumor.
This work presents a predictive-control approach to active mechanical filtering of complex, perio... more This work presents a predictive-control approach to active mechanical filtering of complex, periodic motions of organs induced by respiration or heart beating in robotized surgery. Two different predictive-control schemes are proposed for the compensation of respiratory motions or cardiac motions. For respiratory motions, the periodic property of the disturbance has been included into the input-output model of the controlled system so as to have the robotic system learn and anticipate perturbation motions. A new cost function is proposed for the unconstrained generalized predictive controller (GPC), where reference tracking is decoupled from the rejection of predictable periodic motions. Cardiac motions are more complex, since they are the combination of two periodic nonharmonic components. An adaptive disturbance predictor is proposed which outputs future predicted disturbance values. These predicted values are used to anticipate the disturbance by using the predictive feature of a regular GPC. Experimental results are presented on a laboratory testbed and in vivo on pigs. They demonstrate the effectiveness of the two proposed methods to compensate complex physiological motion.
The application of computer-aided planning, navigation and robotics in surgery provides significa... more The application of computer-aided planning, navigation and robotics in surgery provides significant advantages due to today's sophisticated techniques of patient-data visualization in combination with the flexibility and precision of novel robots. Robotic surgery is going to revolutionize surgical procedures. Augmented with 3D image-guidance technology these tools give finer control over sensitive movements in diseased areas and therefore, allow for more surgical procedures to be performed using minimally invasive techniques than ever before. MRNV 2004 is a scientific workshop on new robotic procedures in all medical application areas. The workshop will bring together scientific, medical and application experts from university, clinical and commercial sites.
To improve the planning of hepatic surgery, we have developed a fully automatic anatomical, patho... more To improve the planning of hepatic surgery, we have developed a fully automatic anatomical, pathological and functional segmentation of the liver derived from a spiral CT scan. Materials and methods : From a 2mm thick enhanced spiral CT scan, a first stage automatically delineates skin, bones, lungs, kidneys and spleen, by combining the use of thresholding, mathematical morphology and distance maps. Next, a reference 3D model is immerged in the image and automatically deformed to liver contours. Then an automatic gaussians fitting on the imaging histogram estimates the intensities of parenchyma, vessels and lesions. This first result is next improved through an original topological and geometrical analysis, providing an automatic delineation of lesions and veins. Finally, a topological and geometrical analysis based on medical knowledge provides hepatic functional information invisible in medical imaging: portal vein labeling and hepatic anatomical segmentation according to the Couinaud classification. Results: Clinical validation performed on more than 30 patients shows that this method's delineation of anatomical structures is often more sensitive and more specific than manual delineation by a radiologist. Conclusion: This study describes the methodology used to create the automatic segmentation of the liver with delineation of important anatomical, pathological and functional structures from a routine CT scan. Using the methods proposed in this study, we have confirmed the accuracy and utility of the creation of 3 -dimensional liver model when compared with the conventional reading of the CT scan by a radiologist. This work, may allow an improvement in preoperative planning of hepatic surgery by more p recisely delineating liver pathology and its relation to normal hepatic structures. In the future this data may be integrated with computer-assisted surgery and thus represents a first step towards the development of an augmented reality surgical system.
We present in this paper an augmented reality guidance system for liver thermal ablation in inter... more We present in this paper an augmented reality guidance system for liver thermal ablation in interventional radiology. To show the relevance of our methodology, the system is incrementally evaluated on an abdominal phantom and then on patients in the operating room. The system registers in a common coordinate system a preoperative image of the patient and the position of the needle that the practitioner manipulates. The breathing motion uncertainty is taken into account with a respiratory gating technique: the preoperative image and the guidance step are synchronized on expiratory phases. In order to fulfil the real-time constraints, we have developed and validated algorithms that automatically process and extract feature points. Since the guidance interface is also a major component of the system effectiveness, we validate the overall targeting accuracy on an abdominal phantom. This experiment showed that a practitioner can reach a predefined target with an accuracy of 2 mm with an insertion time below one minute. Finally, we propose a passive evaluation protocol of the overall system in the operating room during five interventions on patients. These experiments show that the system can provide a guidance information during expiratory phases with an error below 5 mm.
Radiofrequency ablation is increasingly used in the treatment of hepatic tumors. Planning the per... more Radiofrequency ablation is increasingly used in the treatment of hepatic tumors. Planning the percutaneous intervention is essential and particularly difficult. In this paper, we focus on an automated computation of optimal needle insertion in computer-assisted surgery with 3D visualization. First, we review our method which delineates on the skin of a virtual patient the candidate zones for needle insertion, i.e., those which allow safe access to the tumor. In each case, we look for the trajectory that minimizes the volume of burnt tissue. Secondly, we introduce a quasi-exhaustive method that allies sampling and certified minimization to form a strong argument for the accuracy of our results. We also compare results of applying both methods on 7 representative reconstructed patient cases.
This work presents a predictive-control approach to active mechanical filtering of complex, perio... more This work presents a predictive-control approach to active mechanical filtering of complex, periodic motions of organs induced by respiration or heart beating in robotized surgery. Two different predictive-control schemes are proposed for the compensation of respiratory motions or cardiac motions. For respiratory motions, the periodic property of the disturbance has been included into the input-output model of the controlled system so as to have the robotic system learn and anticipate perturbation motions. A new cost function is proposed for the unconstrained generalized predictive controller (GPC), where reference tracking is decoupled from the rejection of predictable periodic motions. Cardiac motions are more complex, since they are the combination of two periodic nonharmonic components. An adaptive disturbance predictor is proposed which outputs future predicted disturbance values. These predicted values are used to anticipate the disturbance by using the predictive feature of a regular GPC. Experimental results are presented on a laboratory testbed and in vivo on pigs. They demonstrate the effectiveness of the two proposed methods to compensate complex physiological motion.
Our purpose is to provide an augmented reality system for Radio-Frequency guidance that could sup... more Our purpose is to provide an augmented reality system for Radio-Frequency guidance that could superimpose a 3D model of the liver, its vessels and tumors (reconstructed from CT images) on external video images of the patient. In this paper, we point out that clinical usability not only need the best affordable registration accuracy, but also a certification that the required accuracy is met, since clinical conditions change from one intervention to the other. Beginning by addressing accuracy performances, we show that a 3D/2D registration based on radio-opaque fiducials is more adapted to our application constraints than other methods. Then, we outline a lack in their statistical assumptions which leads us to the derivation of a new extended 3D/2D criterion. Careful validation experiments on real data show that an accuracy of 2 mm can be achieved in clinically relevant conditions, and that our new criterion is up to 9% more accurate, while keeping a computation time compatible with real-time at 20 to 40 Hz. After the fulfillment of our statistical hypotheses, we turn to safety issues. Propagating the data noise through both our criterion and the classical one, we obtain an explicit formulation of the registration error. As the real conditions do not always fit the theory, it is critical to validate our prediction with real data. Thus, we perform a rigorous incremental validation of each assumption using successively: synthetic data, real video images of a precisely known object, and finally real CT and video images of a soft phantom. Results point out that our error prediction is fully valid in our application range. Eventually, we provide an accurate Augmented Reality guidance system that allows the automatic detection of potentially inaccurate guidance.
Our purpose in this article is to superimpose a 3D model of the liver, its vessels and tumors (re... more Our purpose in this article is to superimpose a 3D model of the liver, its vessels and tumors (reconstructed from CT images) on external video images of the patient for hepatic surgery guidance. The main constraints are the robustness, the accuracy and the computation time. Because of the absence of visible anatomical landmarks and of the “ cylindrical” shape of the upper abdomen, we used some radio-opaque fiducials. The classical least-squares method assuming that there is no noise on the 3D point positions, we designed a new Maximum Likelihood approach to account for this existing noise and we show that it generalizes the classical approaches. Experiments on synthetic data provide evidences that our new criterion is up to 20% more accurate and much more robust, while keeping a computation time compatible with realtime at 20 to 40 Hz. Eventually, careful validation experiments on real data show that an accuracy of 2 mm can be achieved within the liver.
IEEE Transactions on Robotics and Automation, 2003
This paper presents a robotic vision system that automatically retrieves and positions surgical i... more This paper presents a robotic vision system that automatically retrieves and positions surgical instruments during robotized laparoscopic surgical operations. The instrument is mounted on the end-effector of a surgical robot which is controlled by visual servoing. The goal of the automated task is to safely bring the instrument at a desired 3-D location from an unknown or hidden position. Light Emitting Diodes are attached on the tip of the instrument and a speci£c instrument-holder £tted with optical £bers is used to project laser dots on the surface of the organs. These optical markers are detected in the endoscopic image and allow to localize the instrument with respect to the scene. The instrument is recovered and centered in the image plane by means of a visual servoing algorithm using feature errors in the image. With this system, the surgeon can specify a desired relative position between the instrument and the pointed organ. The relationship between the velocity screw of the surgical instrument and the velocity of the markers in the image is estimated online and, for safety reasons, a multi-stages servoing scheme is proposed. Our approach has been successfully validated in a real surgical environment by performing experiments on living tissues in the surgical training room of IRCAD.
This paper presents a robotic vision system that automatically retrieves and positions surgical i... more This paper presents a robotic vision system that automatically retrieves and positions surgical instruments in robotized laparoscopic surgery. The surgical instrument is mounted on the end-effector of a surgical robot which can be controlled by automatic visual feedback. The goal of the automated task is to bring the instrument at a desired location from an unknown or hidden position. To achieve this task, a special instrument-holder is designed with optical fibers and collimators. This instrumentholder projects laser dot patterns onto the organ surface which are seen in the endoscopic images. Then, the instrument is retrieved and centered in the image plane using a visual servoing algorithm. With this system, the surgeon can also specify a desired position for the instrument in the image. Our approach is successfully validated in a real surgical environment by performing experiments on living animals in the surgical training room of IRCAD.
We provided in [14] an augmented reality guidance system for liver punctures, which has been vali... more We provided in [14] an augmented reality guidance system for liver punctures, which has been validated on a static abdominal phantom [16]. In this paper, we report the first in vivo experiments. We developed a strictly passive protocol to directly evaluate our system on patients. We show that the system algorithms work efficiently and we highlight the clinical constraints that we had to overcome (small operative field, weight and sterility of the tracked marker attached to the needle...). Finally, we investigate to what extent breathing motion can be neglected for free breathing patient. Results show that the guiding accuracy, close to 1 cm, is sufficient for large targets only (above 3 cm of diameter) when the breathing motion is neglected. In the near future, we aim at validating our system on smaller targets using a respiratory gating technique.
In this paper, we propose a method based on multiple criteria to assist physicians in planning pe... more In this paper, we propose a method based on multiple criteria to assist physicians in planning percutaneous RFA on liver. We explain how we extracted information from literature and interviews with radiologists, and formalized them into geometric constraints. We expose then our method to compute the most suitable needle insertion in two steps: computation of authorized insertion zones and multi-criteria optimization of the trajectory within this zones. We focus on the combination of the criteria to optimize and on the optimization step.
Medical Image Analysis plays a crucial role in the diagnosis, planning, control and follow-up of ... more Medical Image Analysis plays a crucial role in the diagnosis, planning, control and follow-up of therapy. To be combined efficiently with medical robotics, Medical Image Analysis can be supported by the development of specific computational models of the human body operating at various levels. We describe in this article a hierarchy of these computational models, including the geometrical, physical and physiological levels, and illustrate their potential use in a number of advanced medical applications including image guided, robot-assisted and simulated medical interventions. We conclude this article with scientific perspectives.
IEEE Transactions on Control Systems Technology, 2006
... Page 7. GANGLOFF et al.: MODEL PREDICTIVE CONTROL FOR COMPENSATION OF CYCLIC ORGAN MOTIONS 24... more ... Page 7. GANGLOFF et al.: MODEL PREDICTIVE CONTROL FOR COMPENSATION OF CYCLIC ORGAN MOTIONS 241 Fig. ... The linearized model of the visual loop is given by the bloc diagram in Fig. 10. ... This is a strong assumption when doing direct torque control. ...
In this paper we present a guiding system for laparoscopic surgery which tracks in real time the ... more In this paper we present a guiding system for laparoscopic surgery which tracks in real time the laparoscopic tools and registers at 10 Hz a preoperative CT/MRI model of the patient. To help surgeons, this system provides simultaneously three kinds of supplementary information. It displays the relative positions of the laparoscopic tools and the 3D patient model. It provides two external views of the patient on which his 3D model and tool positions are superimposed. Finally, it displays an endoscopic view augmented with the 3D patient model. Contrary to most systems for laparoscopic surgery, this one is low cost (one PC, two cameras and a printer are necessary) and we have rigorously validated its tracking accuracy in simulated clinical conditions (tracking accuracy of an instrument tip close to 1.5 mm, and endoscopic overlay error under 1.0 mm). Besides, our system accuracy is comparable with commercial system's ones. Eventually, experiments on an abdominal phantom with surgeons demonstrate its usefulness. We plan to realize our first experiments on patients in a few months.
Medical image processing led to a major improvement of patient care: the 3D modeling of patients ... more Medical image processing led to a major improvement of patient care: the 3D modeling of patients from their CTscan or MRI provides an improved surgical planning and simulation allows to train the surgical gesture before carrying it out. These two preoperative steps can be used intraoperatively with the development of augmented reality (AR). In this paper, we present the tools we developed to provide our £rst prototypal AR guiding system for abdominal surgery.
In this paper, we present a new point of view for the conception of what a teleoperated robotic d... more In this paper, we present a new point of view for the conception of what a teleoperated robotic device should be for percutaneous intervention with computed tomography guidance. We focus our attention on very common and widely used medical acts : the radiofrequency ablation and the biopsy procedure. These acts are widely used in manual interventional radiology, but they come up with others problems. First, the guidance precision is limited for a human, second, the radiologist risks its health by being exposed to X rays during each operation. The robotic system we describe is original by its conception and its control feedback (based on force feedback and vision). A prototype is presented, together with a complete workflow of the system. Some results present the first data acquired with the force feedback sensors.
To improve the planning of hepatic surgery, we have developed a fully automatic anatomical, patho... more To improve the planning of hepatic surgery, we have developed a fully automatic anatomical, pathological and functional segmentation of the liver derived from a spiral CT scan. Materials and methods : From a 2mm thick enhanced spiral CT scan, a first stage automatically delineates skin, bones, lungs, kidneys and spleen, by combining the use of thresholding, mathematical morphology and distance maps. Next, a reference 3D model is immerged in the image and automatically deformed to liver contours. Then an automatic gaussians fitting on the imaging histogram estimates the intensities of parenchyma, vessels and lesions. This first result is next improved through an original topological and geometrical analysis, providing an automatic delineation of lesions and veins. Finally, a topological and geometrical analysis based on medical knowledge provides hepatic functional information invisible in medical imaging: portal vein labeling and hepatic anatomical segmentation according to the Couinaud classification. Results: Clinical validation performed on more than 30 patients shows that this method's delineation of anatomical structures is often more sensitive and more specific than manual delineation by a radiologist. Conclusion: This study describes the methodology used to create the automatic segmentation of the liver with delineation of important anatomical, pathological and functional structures from a routine CT scan. Using the methods proposed in this study, we have confirmed the accuracy and utility of the creation of 3 -dimensional liver model when compared with the conventional reading of the CT scan by a radiologist. This work, may allow an improvement in preoperative planning of hepatic surgery by more p recisely delineating liver pathology and its relation to normal hepatic structures. In the future this data may be integrated with computer-assisted surgery and thus represents a first step towards the development of an augmented reality surgical system.
Percutaneous radiofrequency ablation has become a frequently used technique for the treatment of ... more Percutaneous radiofrequency ablation has become a frequently used technique for the treatment of liver cancers, but still remains very difficult to plan. In this paper, we propose a robust method to delineate on the skin of a 3D reconstructed patient the zones that are candidate for an insertion, because they allow a safe access to the tumor without meeting any organ, and to compute automatically within these zones an optimal trajectory minimizing the volume of necrosis covering the tumor.
This work presents a predictive-control approach to active mechanical filtering of complex, perio... more This work presents a predictive-control approach to active mechanical filtering of complex, periodic motions of organs induced by respiration or heart beating in robotized surgery. Two different predictive-control schemes are proposed for the compensation of respiratory motions or cardiac motions. For respiratory motions, the periodic property of the disturbance has been included into the input-output model of the controlled system so as to have the robotic system learn and anticipate perturbation motions. A new cost function is proposed for the unconstrained generalized predictive controller (GPC), where reference tracking is decoupled from the rejection of predictable periodic motions. Cardiac motions are more complex, since they are the combination of two periodic nonharmonic components. An adaptive disturbance predictor is proposed which outputs future predicted disturbance values. These predicted values are used to anticipate the disturbance by using the predictive feature of a regular GPC. Experimental results are presented on a laboratory testbed and in vivo on pigs. They demonstrate the effectiveness of the two proposed methods to compensate complex physiological motion.
The application of computer-aided planning, navigation and robotics in surgery provides significa... more The application of computer-aided planning, navigation and robotics in surgery provides significant advantages due to today's sophisticated techniques of patient-data visualization in combination with the flexibility and precision of novel robots. Robotic surgery is going to revolutionize surgical procedures. Augmented with 3D image-guidance technology these tools give finer control over sensitive movements in diseased areas and therefore, allow for more surgical procedures to be performed using minimally invasive techniques than ever before. MRNV 2004 is a scientific workshop on new robotic procedures in all medical application areas. The workshop will bring together scientific, medical and application experts from university, clinical and commercial sites.
To improve the planning of hepatic surgery, we have developed a fully automatic anatomical, patho... more To improve the planning of hepatic surgery, we have developed a fully automatic anatomical, pathological and functional segmentation of the liver derived from a spiral CT scan. Materials and methods : From a 2mm thick enhanced spiral CT scan, a first stage automatically delineates skin, bones, lungs, kidneys and spleen, by combining the use of thresholding, mathematical morphology and distance maps. Next, a reference 3D model is immerged in the image and automatically deformed to liver contours. Then an automatic gaussians fitting on the imaging histogram estimates the intensities of parenchyma, vessels and lesions. This first result is next improved through an original topological and geometrical analysis, providing an automatic delineation of lesions and veins. Finally, a topological and geometrical analysis based on medical knowledge provides hepatic functional information invisible in medical imaging: portal vein labeling and hepatic anatomical segmentation according to the Couinaud classification. Results: Clinical validation performed on more than 30 patients shows that this method's delineation of anatomical structures is often more sensitive and more specific than manual delineation by a radiologist. Conclusion: This study describes the methodology used to create the automatic segmentation of the liver with delineation of important anatomical, pathological and functional structures from a routine CT scan. Using the methods proposed in this study, we have confirmed the accuracy and utility of the creation of 3 -dimensional liver model when compared with the conventional reading of the CT scan by a radiologist. This work, may allow an improvement in preoperative planning of hepatic surgery by more p recisely delineating liver pathology and its relation to normal hepatic structures. In the future this data may be integrated with computer-assisted surgery and thus represents a first step towards the development of an augmented reality surgical system.
We present in this paper an augmented reality guidance system for liver thermal ablation in inter... more We present in this paper an augmented reality guidance system for liver thermal ablation in interventional radiology. To show the relevance of our methodology, the system is incrementally evaluated on an abdominal phantom and then on patients in the operating room. The system registers in a common coordinate system a preoperative image of the patient and the position of the needle that the practitioner manipulates. The breathing motion uncertainty is taken into account with a respiratory gating technique: the preoperative image and the guidance step are synchronized on expiratory phases. In order to fulfil the real-time constraints, we have developed and validated algorithms that automatically process and extract feature points. Since the guidance interface is also a major component of the system effectiveness, we validate the overall targeting accuracy on an abdominal phantom. This experiment showed that a practitioner can reach a predefined target with an accuracy of 2 mm with an insertion time below one minute. Finally, we propose a passive evaluation protocol of the overall system in the operating room during five interventions on patients. These experiments show that the system can provide a guidance information during expiratory phases with an error below 5 mm.
Radiofrequency ablation is increasingly used in the treatment of hepatic tumors. Planning the per... more Radiofrequency ablation is increasingly used in the treatment of hepatic tumors. Planning the percutaneous intervention is essential and particularly difficult. In this paper, we focus on an automated computation of optimal needle insertion in computer-assisted surgery with 3D visualization. First, we review our method which delineates on the skin of a virtual patient the candidate zones for needle insertion, i.e., those which allow safe access to the tumor. In each case, we look for the trajectory that minimizes the volume of burnt tissue. Secondly, we introduce a quasi-exhaustive method that allies sampling and certified minimization to form a strong argument for the accuracy of our results. We also compare results of applying both methods on 7 representative reconstructed patient cases.
This work presents a predictive-control approach to active mechanical filtering of complex, perio... more This work presents a predictive-control approach to active mechanical filtering of complex, periodic motions of organs induced by respiration or heart beating in robotized surgery. Two different predictive-control schemes are proposed for the compensation of respiratory motions or cardiac motions. For respiratory motions, the periodic property of the disturbance has been included into the input-output model of the controlled system so as to have the robotic system learn and anticipate perturbation motions. A new cost function is proposed for the unconstrained generalized predictive controller (GPC), where reference tracking is decoupled from the rejection of predictable periodic motions. Cardiac motions are more complex, since they are the combination of two periodic nonharmonic components. An adaptive disturbance predictor is proposed which outputs future predicted disturbance values. These predicted values are used to anticipate the disturbance by using the predictive feature of a regular GPC. Experimental results are presented on a laboratory testbed and in vivo on pigs. They demonstrate the effectiveness of the two proposed methods to compensate complex physiological motion.
Our purpose is to provide an augmented reality system for Radio-Frequency guidance that could sup... more Our purpose is to provide an augmented reality system for Radio-Frequency guidance that could superimpose a 3D model of the liver, its vessels and tumors (reconstructed from CT images) on external video images of the patient. In this paper, we point out that clinical usability not only need the best affordable registration accuracy, but also a certification that the required accuracy is met, since clinical conditions change from one intervention to the other. Beginning by addressing accuracy performances, we show that a 3D/2D registration based on radio-opaque fiducials is more adapted to our application constraints than other methods. Then, we outline a lack in their statistical assumptions which leads us to the derivation of a new extended 3D/2D criterion. Careful validation experiments on real data show that an accuracy of 2 mm can be achieved in clinically relevant conditions, and that our new criterion is up to 9% more accurate, while keeping a computation time compatible with real-time at 20 to 40 Hz. After the fulfillment of our statistical hypotheses, we turn to safety issues. Propagating the data noise through both our criterion and the classical one, we obtain an explicit formulation of the registration error. As the real conditions do not always fit the theory, it is critical to validate our prediction with real data. Thus, we perform a rigorous incremental validation of each assumption using successively: synthetic data, real video images of a precisely known object, and finally real CT and video images of a soft phantom. Results point out that our error prediction is fully valid in our application range. Eventually, we provide an accurate Augmented Reality guidance system that allows the automatic detection of potentially inaccurate guidance.
Uploads
Papers by Luc SOLER