Scientists of Samara University has developed and tested the high-speed neural network capable of analyzing the incoming video stream in real time and almost instantly recognizing and finding in it specified objects and images. Along with analyzing the “picture” from a conventional video camera, the development makes it possible to quickly, almost at the speed of light, process data obtained by hyperspectrometers – devices that see reality in multichannel spectral imaging, and allow detecting objects invisible to conventional means of observation.

The project is being implemented by order of the Russian Federal Nuclear Centre of the All-Russian Research Institute of Experimental Physics (RFNC-VNIIEF) within the research program of the National Centre for Physics and Mathematics (NCFM) to be established in Sarov of the Nizhny Novgorod Region on behalf of the President of Russia. Researching in framework of this project is funded by the Ministry of Science and Higher Education of the Russian Federation and the Rosatom State Corporation.

“Our University’s scientists have developed the optical neural network based on the analogue photonic computing system: they assembled a demo sample of the system, which proved operability of the selected scheme during experiments. This neural network is designed to analyze the incoming video stream, with subsequently recognizing and classifying specified objects and images. The key feature of the development is its ability to analyze hyperspectral data: the system is intended for working with a dual-band hyperspectrometer to be also developed at our University. The analogue photonic computing system allows analyzing and recognizing objects almost at the speed of light, which significantly exceeds by hundreds of times the speed specifications for modern digital neural networks based on conventional semiconductor computers. This is especially important for the operational analysis of hyperspectral data that are initially major data arrays”, told us Roman Skidanov, Doctor of Physical and Mathematical Sciences, Professor at the Department of Engineering Cybernetics of Samara University.

In addition to high performance and wide spectral range, analogue optical computing systems also have advantages such as complete protection from electromagnetic disturbances, low energy consumption and the option of parallel data processing. The system scheme that allows for fully optical processing of incoming data, was first proposed in 1958. This area actively developed in the 80s of the last century, but then using the devices almost ceased, because of both their bulkiness and development of digital technology. In recent years, in various countries of the world, this field of applied research has become increasingly relevant, due to new materials and compact special-structure optics to be designed.

“Our demo sample has been created by using standard laboratory optical and mechanical components, as well as various modulators and video cameras. The optical scheme of the device is designed in such a way that a number of additional tasks can be solved, due to the camera recording the intensity distribution in the frequency plane. During the first experiments, the demo sample showed recognition reliability of 93.75%. In 2024, it is planned to assemble and test an experimental sample of the system in a fairly compact case compared to a small computer system unit. Recognition accuracy and reliability of the experimental sample should increase, due to selection of components with improved specifications. A prototype of the device is likely to be ready in 2025”, noted Roman Skidanov.
 
For reference:

In hyperspectral imaging or hyperspectral Earth remote sensing conducted from a UAV or a space satellite, each pixel of the resulting image is represented as a full or continuous spectrum, which allows identifying spectral properties of the desired objects, as well as detecting objects invisible when other means of observation applied, during processing the data obtained.

For example, hyperspectrometers can efficiently detect greenhouse gases by recording methane and CO₂ emissions, as well as conduct geological exploration of hard-to-reach areas, identifying from space spectral signatures of various minerals, including those that indicate possible location of oil and natural gas deposits. Hyperspectrometers can monitor more efficiently and accurately occurrence of forest fires and the condition of forests and agricultural crops, help calculate vegetation indices and even detect from outer space stress in plants.

Samara University is a participant in the National Project “Science and Universities”.