Researchers have developed a camera that uses an array of thin microlenses and new image processing algorithms to capture 3D information about objects in a scene with a single exposure. The camera could be useful for a variety of applications such as industrial parts inspection, gesture recognition and data collection for 3D display systems.
“We consider our camera to be lensless because it replaces the bulk lenses used in conventional cameras with an array of thin and lightweight flexible polymer microlenses,” said research team leader Weijian Yang from the University. from California to Davis. “Because each microlens can observe objects from different viewing angles, it can perform complex imaging tasks such as acquiring 3D information from objects partially obscured by objects closer to the camera.”
In the review Express Optics, Yang and first author Feng Tian, a doctoral student in Yang’s lab, describe the new 3D camera. Because the camera learns from existing data how to digitally reconstruct a 3D scene, it can produce 3D images in real time.
“This 3D camera could be used to give robots 3D vision, which could help them navigate 3D space or enable complex tasks such as manipulating thin objects,” Yang said. “It could also be used to acquire rich 3D information that could provide content for 3D displays used in games, entertainment, or many other applications.”
A camera that learns
The new camera grew out of previous work in which researchers developed a compact microscope capable of imaging 3D microscopic structures for biomedical applications. “We built the microscope using a microlens array and thought a similar concept could be applied for imaging macroscopic objects,” Yang said.
The new camera’s individual lenses allow it to see objects from different angles or perspectives, which provides depth information. Although other research groups have developed cameras based on single-layer microlens arrays, it has been difficult to make them practical due to extensive calibration processes and slow reconstruction speeds.
To create a more practical 3D camera for macroscopic objects, the researchers considered the microlensing array and the reconstruction algorithm together rather than addressing them separately. They custom-designed and fabricated the microlens array, which contains 37 small lenses housed in a circular layer of polymer just 12 millimeters in diameter. The reconstruction algorithm they developed is based on a highly efficient artificial neural network that learns to map image information to objects in a scene.
“Many existing neural networks can perform designated tasks, but the underlying mechanism is difficult to explain and understand,” Yang said. “Our neural network is based on a physical model of image reconstruction. This makes the learning process much easier and results in high-quality reconstructions.”
Once the learning process is complete, it can reconstruct images containing objects at different distances from the camera at very high speed. The new camera does not need calibration and can be used to map 3D locations and spatial profiles (or contours) of objects.
see through objects
After running digital simulations to verify the camera’s performance, the researchers performed 2D imaging that showed noticeably pleasing results. They then tested the camera’s ability to perform 3D imaging of objects at different depths. The resulting 3D reconstruction could be recentered at different depths or distances. The camera also created a depth map that matched the actual layout of the object.
“In a final demonstration, we showed that our camera could image objects behind the opaque obstacles,” Yang said. “To the best of our knowledge, this is the first demonstration of imaging objects behind opaque obstacles using a lensless camera.”
The researchers are currently working to reduce the artifacts, or errors, that appear in the 3D reconstructions and to improve the algorithms to improve quality and speed. They also want to miniaturize the overall footprint of the device so that it can fit in a cellphone, which would make it more portable and enable more apps.
“Our lensless 3D camera uses computational imaging, an emerging approach that jointly optimizes imaging hardware and object reconstruction algorithms to achieve desired imaging tasks and quality,” Yang said. “With the recent development of low-cost advanced micro-optics manufacturing techniques as well as advances in machine learning and computational resources, computational imaging will enable many new imaging systems with advanced functionality.”
Researchers Create Miniature Wide-Angle Camera With Flat Metal Lenses
Feng Tian et al, Learned lensless 3D camera, Express Optics (2022). DOI: 10.1364/OE.465933
Quote: New lensless camera creates 3D images from a single exposure (September 7, 2022) Retrieved September 7, 2022 from https://phys.org/news/2022-09-lensless-camera-3d-images -exposure.html
This document is subject to copyright. Except for fair use for purposes of private study or research, no part may be reproduced without written permission. The content is provided for information only.