WASHINGTON—Researchers have developed a camera that uses an array of thin microlenses and new image processing algorithms to capture 3D information about objects in a scene with a single exposure. The camera could be useful for a variety of applications such as industrial parts inspection, gesture recognition and data collection for 3D display systems.
“We consider our camera to be lensless because it replaces the bulk lenses used in conventional cameras with an array of thin and lightweight flexible polymer microlenses,” said research team leader Weijian Yang from the University. from California to Davis. “Because each microlens can observe objects from different viewing angles, it can perform complex imaging tasks such as acquiring 3D information from objects partially obscured by objects closer to the camera.”
In the Optica Publishing Group Journal Express Optics, Yang and first author Feng Tian, a doctoral student in Yang’s lab, describe the new 3D camera. Because the camera learns from existing data how to digitally reconstruct a 3D scene, it can produce 3D images in real time.
“This 3D camera could be used to give robots 3D vision, which could help them navigate 3D space or enable complex tasks such as manipulating thin objects,” Yang said. “It could also be used to acquire rich 3D information that could provide content for 3D displays used in games, entertainment, or many other applications.”
A camera that learns
The new camera grew out of previous work in which researchers developed a compact microscope capable of imaging 3D microscopic structures for biomedical applications. “We built the microscope using a microlens array and thought a similar concept could be applied for imaging macroscopic objects,” Yang said.
The new camera’s individual lenses allow it to see objects from different angles or perspectives, which provides depth information. Although other research groups have developed cameras based on single-layer microlens arrays, it has been difficult to make them practical due to the extensive calibration processes and slow reconstruction speeds.
To create a more practical 3D camera for macroscopic objects, the researchers considered the microlensing array and the reconstruction algorithm together rather than addressing them separately. They custom-designed and fabricated the microlens array, which contains 37 small lenses housed in a circular layer of polymer just 12 millimeters in diameter. The reconstruction algorithm they developed is based on a highly efficient artificial neural network that learns to map image information to objects in a scene.
“Many existing neural networks can perform designated tasks, but the underlying mechanism is difficult to explain and understand,” Yang said. “Our neural network is based on a physical model of image reconstruction. This makes the learning process much easier and results in high quality reconstructions.
Once the learning process is complete, it can reconstruct images containing objects at different distances from the camera at very high speed. The new camera does not need calibration and can be used to map 3D locations and spatial profiles (or contours) of objects.
see through objects
After running digital simulations to verify the camera’s performance, the researchers performed 2D imaging that showed noticeably pleasing results. They then tested the camera’s ability to perform 3D imaging of objects at different depths. The resulting 3D reconstruction could be recentered at different depths or distances. The camera also created a depth map that matched the actual arrangement of objects.
“In a final demonstration, we showed that our camera could image objects behind the opaque obstacles,” Yang said. “To our knowledge, this is the first demonstration of imaging objects behind opaque obstacles using a lensless camera.”
The researchers are currently working to reduce the artifacts, or errors, that appear in the 3D reconstructions and to improve the algorithms to improve quality and speed. They also want to miniaturize the overall footprint of the device so that it can fit in a cellphone, which would make it more portable and enable more apps.
“Our lensless 3D camera uses computational imaging, an emerging approach that jointly optimizes imaging hardware and object reconstruction algorithms to achieve desired imaging tasks and quality,” Yang said. “With the recent development of low-cost advanced micro-optics manufacturing techniques as well as advances in machine learning and computational resources, computational imaging will enable many new imaging systems with advanced functionality.”
Paper: F.Tian, W. Yang, “Learned Lensless 3D Camera,” Opt. Express, 30, 18 (2022). DOI: 10.1364/OE.465933.
About Express Optics
Express Optics reports on scientific and technological innovations in all aspects of optics and photonics. The bi-weekly journal allows rapid publication of original peer-reviewed articles. It is published by Optica Publishing Group and led by editor James Leger of the University of Minnesota, USA. Express Optics is an open access journal and is freely available to online readers. For more information, visit Optical Express.
About Optica Publishing Group (formerly OSA)
Optica Publishing Group is a division of Optica (formerly OSA), Advancing Optics and Photonics Worldwide. It publishes the largest collection of peer-reviewed content in optics and photonics, including 18 prestigious journals, the company’s flagship magazine, and articles from over 835 conferences, including over 6,500 related videos. With over 400,000 journal articles, conference papers and videos to search, discover and access, Optica Publishing Group represents the full spectrum of research in the field worldwide.
The title of the article
3D camera without lens learned
Publication date of articles
Sep 7, 2022
Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of press releases posted on EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.