Smart Mobile Microscopy: Towards Fully-Automated Digitization

PLoS ONE, 2009

Light microscopy provides a simple, cost-effective, and vital method for the diagnosis and screening of hematologic and infectious diseases. In many regions of the world, however, the required equipment is either unavailable or insufficiently portable, and operators may not possess adequate training to make full use of the images obtained. Counterintuitively, these same regions are often well served by mobile phone networks, suggesting the possibility of leveraging portable, camera-enabled mobile phones for diagnostic imaging and telemedicine. Toward this end we have built a mobile phonemounted light microscope and demonstrated its potential for clinical use by imaging P. falciparum-infected and sickle red blood cells in brightfield and M. tuberculosis-infected sputum samples in fluorescence with LED excitation. In all cases resolution exceeded that necessary to detect blood cell and microorganism morphology, and with the tuberculosis samples we took further advantage of the digitized images to demonstrate automated bacillus counting via image analysis software. We expect such a telemedicine system for global healthcare via mobile phone -offering inexpensive brightfield and fluorescence microscopy integrated with automated image analysis -to provide an important tool for disease diagnosis and screening, particularly in the developing world and rural areas where laboratory facilities are scarce but mobile phone infrastructure is extensive.

2013

This letter presents an autofocus (AF) method to position a high-magnification microscope lens that automatically captures hundreds of images from a single moving slide. These images are taken by a mobile clinic unit in a rural location, and are later automatically processed and revised by a remote specialist. This process requires high focus precision to enable image processing techniques to achieve proper results. Low focusing times are also required for the system to be operative. We propose a novel method that combines two focus measures with an adapted searching scheme to cope with both constraints.

2005

We present a new algorithm to determine, quickly and accurately, the best-in-focus image of biological particles. The algorithm is based on a one-dimensional Fourier transform and on the Pearson correlation for automated microscopes along the Z axis. We captured a set of several images at different Z distances from a biological sample. The algorithm uses the Fourier transform to obtain and extract the image frequency content of a vector pattern previously specified to be sought in each captured image; comparing these frequency vectors with the frequency vector of a reference image (usually the first image that we capture or the most out-of-focus image), we find the best-in-focus image via the Pearson correlation. Numerical experimental results show the algorithm has a fast response for finding the best-in-focus image among the captured images, compared with related autofocus techniques presented in the past. The algorithm can be implemented in real-time systems with fast response, accuracy, and robustness; it can be used to get focused images in bright and dark fields; and it offers the prospect of being extended to include fusion techniques to construct multifocus final images.

PROCEEDINGS OF THE III INTERNATIONAL CONFERENCE ON ADVANCED TECHNOLOGIES IN MATERIALS SCIENCE, MECHANICAL AND AUTOMATION ENGINEERING: MIP: Engineering-III – 2021, 2021

One of the most widely used biological and medical instruments are the microscopes. Many new modalities have been developed by constant advancement in the field of microscopy, but their relative sizes and their complexity, and their costs often hinder the usefulness of these instruments in the wider general community and many field environments. In this research, we used a 3D printer and the smartphone camera to design and construct a microscopic prototype to create a relatively low-cost, solid structure and to get pictures that are economically viable and are necessary for recording, analysis, education, and publication to acquire and distribute digital photomicrographs. The two-dimensional program, computer-aided design (Auto CAD) and the three-dimensional program (3D MAX) for 3D printed parts have been used to model and print the necessary components for the microscope prototype. The optical elements include a smartphone camera, an eyepiece, and an objective lens. The use of a traditional eyepiece facilitates device two-way compatibility of smartphones and software with a conventional microscope. The prototype microscope examined several specimens of animal tissue such as skin, follicle hair, and connective tissue. The photos were really accurate, clear, and magnified enough to see the tiny details of the biological cells and tissues. The resulting magnification was comparable to 10x of conventional microscope.

ACS Photonics, 2018

Mobile-phones have facilitated the creation of field-portable, cost-effective imaging and sensing technologies that approach laboratory-grade instrument performance. However, the optical imaging interfaces of mobile-phones are not designed for microscopy and produce spatial and spectral distortions in imaging microscopic specimens. Here, we report on the use of deep learning to correct such distortions introduced by mobile-phone-based microscopes, facilitating the production of high-resolution, denoised and colour-corrected images, matching the performance of benchtop microscopes with high-end objective lenses, also extending their limited depth-of-field. After training a convolutional neural network, we successfully imaged various samples, including blood smears, histopathology tissue sections, and parasites, where the recorded images were highly compressed to ease storage and transmission for telemedicine applications. This method is applicable to other low-cost, aberrated imaging systems, and could offer alternatives for costly and bulky microscopes, while also providing a framework for standardization of optical images for clinical and biomedical applications.

2012

Abstract Within the last few years, cellphone subscriptions have widely spread and now cover even the remotest parts of the planet. Adequate access to healthcare, however, is not widely available, especially in developing countries. We propose a new approach to converting cellphones into low-cost scientific devices for microscopy. Cellphone microscopes have the potential to revolutionize health-related screening and analysis for a variety of applications, including blood and water tests.

Journal of Biomedical Informatics, 2015

Real integration of Virtual Microscopy with the pathologist service workflow requires the design of adaptable strategies for any hospital service to interact with a set of Whole Slide Images. Nowadays, mobile devices have the actual potential of supporting an online pervasive network of specialists working together. However, such devices are still very limited. This article introduces a novel highly adaptable strategy for streaming and visualizing WSI from mobile devices. The presented approach effectively exploits and extends the granularity of the JPEG2000 standard and integrates it with different strategies to achieve a lossless, loosely-coupled, decoder and platform independent implementation, adaptable to any interaction model. The performance was evaluated by two expert pathologists interacting with a set of 20 virtual slides. The method efficiently uses the available device resources: the memory usage did not exceed a 7% of the device capacity while the decoding times were smaller than the 200 ms per Region of Interest, i.e., a window of 256 Â 256 pixels. This model is easily adaptable to other medical imaging scenarios.

Open-source technology not only has facilitated the expansion of the greater research community, but by lowering costs it has encouraged innovation and customizable design. The field of automated microscopy has continued to be a challenge in accessibility due the expense and inflexible, non-interchangeable stages. This paper presents a low-cost, open source microscope 3-D stage. A RepRap 3-D printer was converted to an optical microscope equipped with a customized, 3-D printed holder for a USB microscope. Precision measurements were determined to have an average error of 10 μm at the maximum speed and 27 μm at the minimum recorded speed. Accuracy tests yielded an error of 0.15%. The machine is a true 3-D stage and thus able to operate with USB microscopes or conventional desktop microscopes. It is larger than all commercial alternatives, and is thus capable of high depth images over unprecedented areas and complex geometries. The repeatibility is below 2-D microscope stages, but testing shows that it is adequate for the majority of scientific applications. The open source microscope stage costs less than 3% to 9% of the closest proprietary commercial stages. This extreme affordability vastly improves accessibility for 3-D microscopy throughout the world.

Diagnostic pathology, 2011

Virtual microscopy can be applied in an interactive and an automated manner. Interactive application is performed in close association to conventional microscopy. It includes image standardization suitable to the performance of an individual pathologist such as image colorization, white color balance, or individual adjusted brightness. The steering commands have to include selection of wanted magnification, easy navigation, notification, and simple measurements (distances, areas). The display of the histological image should be adjusted to the physical limits of the human eye, which are determined by a view angle of approximately 35 seconds. A more sophisticated performance should include acoustic commands that replace the corresponding visual commands. Automated virtual microscopy includes so-called microscopy assistants which can be defined similar to the developed assistants in computer based editing systems (Microsoft Word, etc.). These include an automated image standardization...

Procedia Computer Science, 2010

Modern biomedical therapies often involve disease specific drug development and may require observing cells at a very high resolution. Existing commercial microscopes behave very much like their traditional counterparts, where a user controls the microscope and chooses the areas of interest manually on a given specimen scan. This mode of discovery is suited to problems where it is easy for a user to draw a conclusion from observations by finding a small number of areas that might require further investigation. However, observations by an expert can be very time consuming and error prone when there are a large number of potential areas of interest (such as cells or vessels in a tumour), and compute intensive image processing is required to analyse them. In this paper, we propose an Abstract Virtual Instrument (AVI) system for accelerating scientific discovery. An AVI system is a novel software architecture for building an hierarchical scientific instrument–one in which a virtual instrument could be defined in terms of other physical instruments, and in which significant processing is required in producing the illusion of a single virtual scientific discovery instrument. We show that an AVI can be implemented using existing scientific workflow tools that both control the microscope and perform image analysis operations. The resulting solution is a flexible and powerful system for performing dynamic high throughput automatic microscopy. We illustrate the system using a case study that involves searching for blood vessels in an optical tissue scan, and automatically instructing the microscope to rescan these vessels at higher resolution.

IEEE Transactions on Biomedical Engineering, 1982

A new microscope system that is designed to provide images for a computer has been built and tested. This system differs from previous computerized microscopes in that the fundamental design parameters have been tuned to the computer as the receiver of the image instead of the human visual system. This solid-state automated microscope system (SSAM) simultaneously provides wide-field (2 mm), high-resolution (0.5 M), high signal-to-noise images (>53 dB) at data rates of 5 X 106 pixels/s. Various methods have been developed and used to test the design specifications of the system against the actual performance. I. HISTORICAL BACKGROUND MsAN'S first microscope was almost certainly a drop of water. Acting as a hemispherical lens on the surface of a leaf or the back of a hand, it provided magnification on the order' of 1.3 X. As early as the end of the 16th century Hans and Zaccharis Janssen of Middelburg, The Netherlands constructed the first compound (multiple lens) microscope. While Galileo is considered to be the first scientific user of a microscope [1], it was the work of van Leeuwenhoek in Leiden, Hooke in London, and Malpighi in Italy that demonstrated the usefulness and, indeed, necessity of the microscope for biological and medical studies. Van Leeuwenhoek, with an appointment as a custodian in the City Hall of Delft, used his spare time to construct over 500 simple (one-lens) microscopes. The lenses of these microscopes were exquisitely made and provided magnifications up to 200X. With these microscopes, first devised to examine drapery fabrics, van Leeuwenhoek described protozoa, bacteria, and human sperm [2]-[4]. Thus, the use of the microscope as a high-technology scientific instrument goes back at least 350 years.

Optics in the Life Sciences, 2011

In this paper we report the development of two attachments to a commercial cell phone that transform the phone's integrated lens and image sensor into a 3506 microscope and visible-light spectrometer. The microscope is capable of transmission and polarized microscopy modes and is shown to have 1.5 micron resolution and a usable field-of-view of *1506150 mm with no image processing, and approximately 3506350 mm when post-processing is applied. The spectrometer has a 300 nm bandwidth with a limiting spectral resolution of close to 5 nm. We show applications of the devices to medically relevant problems. In the case of the microscope, we image both stained and unstained blood-smears showing the ability to acquire images of similar quality to commercial microscope platforms, thus allowing diagnosis of clinical pathologies. With the spectrometer we demonstrate acquisition of a white-light transmission spectrum through diffuse tissue as well as the acquisition of a fluorescence spectrum. We also envision the devices to have immediate relevance in the educational field.

Biomedical Optics Express, 2017

In this paper we present a method for autofocusing images of sputum smears taken from a microscope which combines the finding of the optimal focus distance with an algorithm for extending the depth of field (EDoF). Our multifocus fusion method produces an unique image where all the relevant objects of the analyzed scene are well focused, independently to their distance to the sensor. This process is computationally expensive which makes unfeasible its automation using traditional embedded processors. For this purpose a low-cost optimized implementation is proposed using limited resources embedded GPU integrated on cutting-edge NVIDIA system on chip. The extensive tests performed on different sputum smear image sets show the real-time capabilities of our implementation maintaining the quality of the output image.

Scientific Reports, 2017

We present a portable multi-contrast microscope capable of producing bright-field, dark-field, and differential phase contrast images of thin biological specimens on a smartphone platform. The microscopy method is based on an imaging scheme termed “color-coded light-emitting-diode (LED) microscopy (cLEDscope),” in which a specimen is illuminated with a color-coded LED array and light transmitted through the specimen is recorded by a color image sensor. Decomposition of the image into red, green, and blue colors and subsequent computation enable multi-contrast imaging in a single shot. In order to transform a smartphone into a multi-contrast imaging device, we developed an add-on module composed of a patterned color micro-LED array, specimen stage, and miniature objective. Simple installation of this module onto a smartphone enables multi-contrast imaging of transparent specimens. In addition, an Android-based app was implemented to acquire an image, perform the associated computatio...

Methods of Information in Medicine, 2007

Summary Objectives: To increase the chance for a cure, cancer must be detected as early as possible. This can be achieved with cytopathological diagnostic methods. For a further increase of the diagnostic accuracy of these methods we introduced the multimodal cell analysis, viz, cells on the slide have to be relocalized to enable successive analysis of identical cells in different stains. For practical reasons the relocalization step must be automated. Methods: For a fully automatic acquisition of successive cell images we use a passive autofocus that is adaptive to the material, i.e., to the cells, followed by a comparison of the scenes, i.e., the cell constellation, of two such obtained images from different stains. In case that no sub-scene match can be found the search is extended to the surrounding area. A set of 1 556 scenes from seven specimens have been subject to our algorithm. The automatically relocalized and acquired images from a second stain have been manually compared...