A new milestone has been reached in the field of computer vision thanks to an intelligent photonic chip capable of processing, transmitting and reconstructing images in a few nanoseconds. This innovation could transform machine vision applications, notably for autonomous driving, industrial inspection and robotic vision.
Researchers have developed an intelligent photonic detection and calculation chip, capable of processing, transmitting and reconstructing images of a scene in a few nanoseconds. This advancement paves the way for extremely fast image processing, beneficial for cutting-edge intelligence in machine vision applications such as autonomous driving, industrial inspection and robotic vision.
Edge computing, which performs computationally intensive tasks like image processing and analysis on local devices, is evolving toward edge intelligence by adding artificial intelligence (AI)-driven analysis and decision-making ).
Unrivaled performance
“ Capturing, processing, and analyzing images for edge-based tasks, such as autonomous driving, is currently limited to millisecond speeds due to the need for optical-to-electronic conversions “, said Lu Fang, head of the research team at Tsinghua University in China. “ Our new chip can perform all these processes in nanoseconds by keeping them in the optical domain. This could be used to significantly improve, or even replace, the traditional architecture of sensor acquisition followed by AI post-processing. »
The researchers describe the new chip, which they call an optical parallel computing network (OPCA) chip. They show that OPCA has a processing bandwidth of up to one hundred billion pixels and a response time of just 6 nanoseconds, about six orders of magnitude faster than current methods. They also used the chip to create an optical neural network integrating perception, calculation and image reconstruction.
Eliminate optical-electronic conversions
Computer vision, which uses cameras, image sensors, lights and computer algorithms to capture, process and analyze images for specific tasks, traditionally involves converting optical information into digital electrical signals using sensors. These signals are then transmitted via optical fibers for long-distance data transmission and downstream tasks. However, frequent conversion between optical and electrical signals, as well as limited advances in electronic processors, have become a major restriction for improving the speed and processing capacity of computer vision.
“ The world is entering an era of AI, but AI is very time and energy intensive Lu Fang added. “ Meanwhile, the growth of edge devices, such as smartphones, smart cars and laptops, has led to explosive growth in image data to be processed, transmitted and displayed. We are working to advance computer vision by integrating sensing and computing in the optical domain, which is particularly important for edge computing and enabling more sustainable AI applications. »
Create an all-optical input-output connection
The chip architecture allowed researchers to create an end-to-end multi-wavelength optical neural network to couple on-chip modulated light into a broadband optical waveguide, where the modulated light is summed spectrally. The multispectral optical outputs can then be used for classification tasks or to create an all-optical reconstruction of the image.
“ Because each sensing-computing element on this chip is reconfigurable, they can each function as a programmable neuron that generates a light modulation output based on the input and weight Lu Fang said again. “ The neural network connects all sensing-computing neurons with a single waveguide, facilitating a complete all-optical connection between input information and output. »
Towards practical applications
To demonstrate the capabilities of the OPCA chip, the researchers showed that it could be used to classify a handwritten image and perform image convolution, a process that applies a filter to an image to extract features. The results showed that the chip architecture can effectively complete information compression and scene reconstruction, indicating its potential for widespread applications.
Researchers are now working to improve the OPCA sense-compute chip to further improve computing performance while being more aligned with real-world scenarios and optimized for edge computing applications. They argue that for practical use, the processing capacity of the optical neural network should be increased to efficiently handle increasingly complex and realistic intelligent tasks. The size of the OPCA chip and the overall shape must also be minimized.
“ We hope that computer vision will gradually be improved to be faster and more energy efficient by using light to perform both detection and calculation Lu Fang concluded. “ Although today’s approach is unlikely to be completely replaced, we expect the sense-compute method to find its niche in edge computing where it can lead to a wide range of promising applications. »
Researchers have developed a new intelligent photonic sensing and computing chip capable of processing, transmitting and reconstructing images of a scene in nanoseconds. Credit: Wei Wu, Tsinghua University
Article : W. Wu, T. Zhou, L. Fang, “Parallel photonic chip for nano-second end-to-end image processing, transmission, and reconstruction,”11, 6, 831-837 (2024). DOI: doi.org/10.1364/OPTICA.516241
