Yuanzheng Gong

University of Washington, Mechanical Engineering, Alumnus

Followers

Following

Co-author

Public Views

Eric Seibel

Muhammad Ayaz Shirazi

Kyungpook National University

Danish Khan

pileun kim

Georgia Institute of Technology

Yong K Cho

Georgia Institute of Technology

G. Kordelas

Woontack Woo

Korea Advanced Institute of Science and Technology

Shenman Zhang

Wuhan University

George Wolberg

Diego Viejo

University of Alicante / Universidad de Alicante

Interests

Uploads

Papers by Yuanzheng Gong

Feature-based three-dimensional registration for repetitive geometry in machine vision

Journal of information technology & software engineering, 2016

As an important step in three-dimensional (3D) machine vision, 3D registration is a process of al... more As an important step in three-dimensional (3D) machine vision, 3D registration is a process of aligning two or multiple 3D point clouds that are collected from different perspectives together into a complete one. The most popular approach to register point clouds is to minimize the difference between these point clouds iteratively by Iterative Closest Point (ICP) algorithm. However, ICP does not work well for repetitive geometries. To solve this problem, a feature-based 3D registration algorithm is proposed to align the point clouds that are generated by vision-based 3D reconstruction. By utilizing texture information of the object and the robustness of image features, 3D correspondences can be retrieved so that the 3D registration of two point clouds is to solve a rigid transformation. The comparison of our method and different ICP algorithms demonstrates that our proposed algorithm is more accurate, efficient and robust for repetitive geometry registration. Moreover, this method c...

Download

Data-Efficient Learning for Sim-to-Real Robotic Grasping using Deep Point Cloud Prediction Networks

ArXiv, 2019

Training a deep network policy for robot manipulation is notoriously costly and time consuming as... more Training a deep network policy for robot manipulation is notoriously costly and time consuming as it depends on collecting a significant amount of real world data. To work well in the real world, the policy needs to see many instances of the task, including various object arrangements in the scene as well as variations in object geometry, texture, material, and environmental illumination. In this paper, we propose a method that learns to perform table-top instance grasping of a wide variety of objects while using no real world grasping data, outperforming the baseline using 2.5D shape by 10%. Our method learns 3D point cloud of object, and use that to train a domain-invariant grasping policy. We formulate the learning process as a two-step procedure: 1) Learning a domain-invariant 3D shape representation of objects from about 76K episodes in simulation and about 530 episodes in the real world, where each episode lasts less than a minute and 2) Learning a critic grasping policy in si...

Download

Path Planning for Semi-automated Simulated Robotic Neurosurgery

Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE/RSJ International Conference on Intelligent Robots and Systems

This paper considers the semi-automated robotic surgical procedure for removing the brain tumor m... more This paper considers the semi-automated robotic surgical procedure for removing the brain tumor margins, where the manual operation is a tedious and time-consuming task for surgeons. We present robust path planning methods for robotic ablation of tumor residues in various shapes, which are represented in point-clouds instead of analytical geometry. Along with the path plans, corresponding metrics are also delivered to the surgeon for selecting the optimal candidate in the automated robotic ablation. The selected path plan is then executed and tested on RAVEN(™) II surgical robot platform as part of the semi-automated robotic brain tumor ablation surgery in a simulated tissue phantom.

Axial-Stereo 3-D Optical Metrology for Inner Profile of Pipes Using a Scanning Laser Endoscope

International Journal of Optomechatronics, 2015

As the rapid progress in the development of optoelectronic components and computational power, 3D... more As the rapid progress in the development of optoelectronic components and computational power, 3D optical metrology becomes more and more popular in manufacturing and quality control due to its flexibility and high speed. However, most of the optical metrology methods are limited to external surfaces. This paper proposed a new approach to measure tiny internal 3D surfaces with a scanning fiber endoscope and axial-stereo vision algorithm. A dense, accurate point cloud of internally machined threads was generated to compare with its corresponding X-ray 3D data as ground truth, and the quantification was analyzed by Iterative Closest Points algorithm.

Axial-Stereo 3D Optical Metrology of Internally Machined Parts Using High-Quality Imaging from a Scanning Laser Endoscope

2014 International Symposium on Optomechatronic Technologies, 2014

In vivo laser-based imaging of the human fallopian tube for future cancer detection

Endoscopic Microscopy X; and Optical Techniques in Pulmonary Medicine II, 2015

Development of Multi-segment Steering Mechanism and 3D Panorama for Automated Bladder Surveillance System

IEEE/ASME Transactions on Mechatronics, 2015

Semi-autonomous simulated brain tumor ablation with RAVENII Surgical Robot using behavior tree

2015 IEEE International Conference on Robotics and Automation (ICRA), 2015

Medical robots have been widely used to assist surgeons to carry out dexterous surgical tasks via... more Medical robots have been widely used to assist surgeons to carry out dexterous surgical tasks via various ways. Most of the tasks require surgeon&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s operation directly or indirectly. Certain level of autonomy in robotic surgery could not only free the surgeon from some tedious repetitive tasks, but also utilize the advantages of robot: high dexterity and accuracy. This paper presents a semi-autonomous neurosurgical procedure of brain tumor ablation using RAVEN Surgical Robot and stereo visual feedback. By integrating with the behavior tree framework, the whole surgical task is modeled flexibly and intelligently as nodes and leaves of a behavior tree. This paper provides three contributions mainly: (1) describing the brain tumor ablation as an ideal candidate for autonomous robotic surgery, (2) modeling and implementing the semi-autonomous surgical task using behavior tree framework, and (3) designing an experimental simulated ablation task for feasibility study and robot performance analysis.

Bound constrained bundle adjustment for reliable 3D reconstruction

Optics express, Jan 20, 2015

Bundle adjustment (BA) is a common estimation algorithm that is widely used in machine vision as ... more Bundle adjustment (BA) is a common estimation algorithm that is widely used in machine vision as the last step in a feature-based three-dimensional (3D) reconstruction algorithm. BA is essentially a non-convex non-linear least-square problem that can simultaneously solve the 3D coordinates of all the feature points describing the scene geometry, as well as the parameters of the camera. The conventional BA takes a parameter either as a fixed value or as an unconstrained variable based on whether the parameter is known or not. In cases where the known parameters are inaccurate but constrained in a range, conventional BA results in an incorrect 3D reconstruction by using these parameters as fixed values. On the other hand, these inaccurate parameters can be treated as unknown variables, but this does not exploit the knowledge of the constraints, and the resulting reconstruction can be erroneous since the BA optimization halts at a dramatically incorrect local minimum due to its non-con...

Accurate three-dimensional virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot

by Yuanzheng Gong and Eric Seibel

Journal of Medical Imaging, 2014

Brain tumor margin removal is challenging because diseased tissue is often visually indistinguish... more Brain tumor margin removal is challenging because diseased tissue is often visually indistinguishable from healthy tissue. Leaving residual tumor leads to decreased survival, and removing normal tissue causes life-long neurological deficits. Thus, a surgical robotics system with a high degree of dexterity, accurate navigation, and highly precise resection is an ideal candidate for image-guided removal of fluorescently labeled brain tumor cells. To image, we developed a scanning fiber endoscope (SFE) which acquires concurrent reflectance and fluorescence wide-field images at a high resolution. This miniature flexible endoscope was affixed to the arm of a RAVEN II surgical robot providing programmable motion with feedback control using stereo-pair surveillance cameras. To verify the accuracy of the three-dimensional (3-D) reconstructed surgical field, a multimodal physical-sized model of debulked brain tumor was used to obtain the 3-D locations of residual tumor for robotic path planning to remove fluorescent cells. Such reconstruction is repeated intraoperatively during margin clean-up so the algorithm efficiency and accuracy are important to the robotically assisted surgery. Experimental results indicate that the time for creating this 3-D surface can be reduced to one-third by using known trajectories of a robot arm, and the error from the reconstructed phantom is within 0.67 mm in average compared to the model design.

Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model

by Yuanzheng Gong and Eric Seibel

Proceedings - Society of Photo-Optical Instrumentation Engineers

Accurate three-dimensional virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot

Superfast 3-D Shape Measurement with Binary Defocusing Techniques

Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model

Proceedings - Society of Photo-Optical Instrumentation Engineers

The challenge is to accurately guide the surgical tool within the three-dimensional (3D) surgical... more The challenge is to accurately guide the surgical tool within the three-dimensional (3D) surgical field for robotically-assisted operations such as tumor margin removal from a debulked brain tumor cavity. The proposed technique is 3D image-guided surgical navigation based on matching intraoperative video frames to a 3D virtual model of the surgical field. A small laser-scanning endoscopic camera was attached to a mock minimally-invasive surgical tool that was manipulated toward a region of interest (residual tumor) within a phantom of a debulked brain tumor. Video frames from the endoscope provided features that were matched to the 3D virtual model, which were reconstructed earlier by raster scanning over the surgical field. Camera pose (position and orientation) is recovered by implementing a constrained bundle adjustment algorithm. Navigational error during the approach to fluorescence target (residual tumor) is determined by comparing the calculated camera pose to the measured camera pose using a micro-positioning stage. From these preliminary results, computation efficiency of the algorithm in MATLAB code is near real-time (2.5 sec for each estimation of pose), which can be improved by implementation in C++. Error analysis produced 3-mm distance error and 2.5 degree of orientation error on average. The sources of these errors come from 1) inaccuracy of the 3D virtual model, generated on a calibrated RAVEN robotic platform with stereo tracking; 2) inaccuracy of endoscope intrinsic parameters, such as focal length; and 3) any endoscopic image distortion from scanning irregularities. This work demonstrates feasibility of micro-camera 3D guidance of a robotic surgical tool.

High-Speed, High-Resolution 3D Imaging Using Projector Defocusing

Algorithms and Technologies, 2012

Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm

ABSTRACT To test this system, synthetic phantoms of debulked tumor from brain are fabricated havi... more ABSTRACT To test this system, synthetic phantoms of debulked tumor from brain are fabricated having spots of fluorescence representing residual tumor. Three-dimension (3D) surface maps of this surgical field are produced by moving the SFE over the phantom during concurrent reflectance and fluorescence imaging (30Hz video). SIFT-based feature matching between reflectance images is implemented to select a subset of key frames, which are reconstructed in 3D by bundle adjustment. The resultant reconstruction yields a multimodal 3D map of the tumor region that can improve visualization and robotic path planning. Efficiency of creating these maps is important as they are generated multiple times during tumor margin clean-up. By using pre-programmed vector motions of the robot arm holding the SFE, the computer vision algorithms are optimized for efficiency by reducing search times. Preliminary results indicate that the time for creating these 3D multimodal maps of the surgical field can be reduced to one third by using known trajectories of the surgical robot moving the image-guided tool.

High-resolution, superfast 3-D imaging using a phase-shifting method

With recent advancement of digital video projection technology, real-time 3-D shape measurement b... more With recent advancement of digital video projection technology, real-time 3-D shape measurement becomes one of the core areas because of its importance in numerous fields. However, because a digital video projector usually can not refresh images more than 120 Hz rate, the maximum measurement speed speed is limited to 120 Hz [1]. The most recently developed DLP Discovery technology has enabled 1-bit image switching rate at tens of kHz rate for a resolution of 1024 × 768. This innovation shows great potential for 3-D optical metrology because of its ...

High-resolution, high-speed three-dimensional shape measurement using projector defocusing

Optical Engineering, 2011

We present a high-resolution, high-speed three-dimensional (3-D) shape measurement technique that... more We present a high-resolution, high-speed three-dimensional (3-D) shape measurement technique that can reach the speed limit of a digital fringe projection system without significantly increasing the system cost. Instead of generating sinusoidal fringe patterns by a computer directly, they are produced by defocusing binary ones. By this means, with a relatively inexpensive camera, the 3-D shape measurement system can double the previously maximum achievable speed and reach the refreshing rate of a digital-light-processing projector: 120 Hz.

Ultrafast 3-D shape measurement with an off-the-shelf DLP projector

Optics Express, 2010

This paper presents a technique that reaches 3-D shape measurement speed beyond the digital-light... more This paper presents a technique that reaches 3-D shape measurement speed beyond the digital-light-processing (DLP) projector's projection speed. In particular, a "solid-state" binary structured pattern is generated with each micro-mirror pixel always being at one status (ON or OFF). By this means, any time segment of projection can represent the whole signal, thus the exposure time can be shorter than the projection time. A sinusoidal fringe pattern is generated by properly defocusing a binary one, and the Fourier fringe analysis means is used for 3-D shape recovery. We have successfully reached 4,000 Hz rate (80 μs exposure time) 3-D shape measurement speed with an off-the-shelf DLP projector. "Automated phase-measuring profilometry using defocused projection of a Ronchi grating," Opt. Commun. 94, 561-573 (1992). 17. X. Su and Q. Zhang, "Dynamic 3-D shape measurement method: A review," Opt. Laser Eng. 48, 191-204 (2010).

Download

Some recent advance on high-speed, high-resolution 3-D shape measurement using projector defocusing

Abstract High-speed, high-resolution 3-D shape measurement becomes increasingly important, with b... more Abstract High-speed, high-resolution 3-D shape measurement becomes increasingly important, with broad applications including medicine, homeland security, and entertainment. In recent years, we have made some progress, and developed an unprecedented 60 Hz rate 3-D shape measurement system with a digital fringe projection and phase-shifting method. However, a hardware bottleneck was met to further improve its speed. Since 2009, we have been studying a new method that could potentially eliminate ...

Feature-based three-dimensional registration for repetitive geometry in machine vision

Journal of information technology & software engineering, 2016

Download

Data-Efficient Learning for Sim-to-Real Robotic Grasping using Deep Point Cloud Prediction Networks

ArXiv, 2019

Download

Path Planning for Semi-automated Simulated Robotic Neurosurgery

Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE/RSJ International Conference on Intelligent Robots and Systems

Axial-Stereo 3-D Optical Metrology for Inner Profile of Pipes Using a Scanning Laser Endoscope

International Journal of Optomechatronics, 2015

Axial-Stereo 3D Optical Metrology of Internally Machined Parts Using High-Quality Imaging from a Scanning Laser Endoscope

2014 International Symposium on Optomechatronic Technologies, 2014

In vivo laser-based imaging of the human fallopian tube for future cancer detection

Endoscopic Microscopy X; and Optical Techniques in Pulmonary Medicine II, 2015

Development of Multi-segment Steering Mechanism and 3D Panorama for Automated Bladder Surveillance System

IEEE/ASME Transactions on Mechatronics, 2015

Semi-autonomous simulated brain tumor ablation with RAVENII Surgical Robot using behavior tree

2015 IEEE International Conference on Robotics and Automation (ICRA), 2015

Bound constrained bundle adjustment for reliable 3D reconstruction

Optics express, Jan 20, 2015

Accurate three-dimensional virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot

by Yuanzheng Gong and Eric Seibel

Journal of Medical Imaging, 2014

Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model

by Yuanzheng Gong and Eric Seibel

Proceedings - Society of Photo-Optical Instrumentation Engineers

Accurate three-dimensional virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot

Superfast 3-D Shape Measurement with Binary Defocusing Techniques

Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model

Proceedings - Society of Photo-Optical Instrumentation Engineers

High-Speed, High-Resolution 3D Imaging Using Projector Defocusing

Algorithms and Technologies, 2012

Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm

High-resolution, superfast 3-D imaging using a phase-shifting method

High-resolution, high-speed three-dimensional shape measurement using projector defocusing

Optical Engineering, 2011

Ultrafast 3-D shape measurement with an off-the-shelf DLP projector

Optics Express, 2010

Download

Some recent advance on high-speed, high-resolution 3-D shape measurement using projector defocusing

Yuanzheng Gong

Related Authors

Uploads

Papers by Yuanzheng Gong

Log In