RU 
EN 
CN 
ES 
What are the latest and most interesting projects of the Institute of Artificial Intelligence?
To begin with, one of key tasks of the Institute is finding large-scaled high-tech partners. Sber is one of the companies, which we have developed quite productive relations with. In 2021–2022, together with Sber, we participated in developing the ruDALL-E multimodal neural network; now it is known as Kandinsky. This neural network generates an image based on your text query. For example, you give a task to the chatbot on your smartphone or computer: draw “a green meadow on a sunny day” or “Peter the Great aboard a ship in Holland”, and in a few seconds, the picture is ready. This is our Russian response to Midjourney. Sber presented it at the AI Journey International Conference.
And how does it differ from foreign analogues?
It is entirely made involving the Russian National Corpus. Now there is an interesting trend: based on such large language models, one can not only engage in entertainment, but go into industry, into education, using both linguistic and visual tools to create the basis.
Please explain.
Now the GPT neural networks are used for generating text or creatively rewriting it. But this text assistant can be turned into a consultant, for example, in the legal field. There are up to 115 thousand different legislative acts in Russia, and to write something a little more complicated than a power of attorney for a child to travel abroad, you need to immediately contact a lawyer. That is, we need a specialist who keeps all the necessary laws in his/her mind. However, now it is possible to entrust this to GPT so that the network could analyze the issues and advise what to do next.
How many lawyers can lose their jobs…
On the contrary, it will relieve them, removing the routine, and allow dealing with more complex and interesting issues. I mean the initial processing of a query; even the “Gosuslugi” portal is suitable for the resource. For example, when we trained ruDALL-E, we uploaded 250 million annotated images into the neural network. In this case, you can add hundreds of thousands of legislative acts.
If the visual modality is added to the text one, the same principle can be scaled, for example, for analyzing the state of crops or forest plantations from drones for the needs of agriculture. Having uploaded different tree species into the neural network and having linked the text with the picture, we can enter, for example, the text query: the neural network, please tell us how many fur-trees are on this video, what is their average age, etc. Such neural networks are called Foundation AI Models.
So, it turns out that you are already talking about creating neural networks for drones.
Yes, and here we have been developing this theme with our key partner – Transport of the Future, LLC. This is an ambitious, large company with great potential. It has been constructing a factory for producing civilian drones in the territory of the Tolyatti Special Economic Zone. In fact, it is building a new industry in the country.
What will be your role there?
We have been making for them a Foundation AI Model that will evaluate and control how a specialist assembles and operates the drone: whether he screws right screws, follows proper sequence, spreads the blades, pours oil; how much he is generally involved in the process – listens to you or flies in the clouds… At the end of the day, this neural network can be asked to tell about all the incidents: who failed and where, what went wrong. The same system is suitable for training in conditions of production, for monitoring operational processes, wherever a lot of people and equipment are involved, where a large conveyor operates. It will be based on the Foundation AI Model bearing in its memory up to 150 billion different parameters.
If one of the drone assemblers collectors gets distracted, will the device get useless?
The main thing for the drone operator is to be in good shape, attentive and involved in the process. Attention can be trained by using VR training simulators, special sensors that show human involvement in the process.
That is, you are engaged in making a system intended for reducing so-called human factor, the probability of error to zero. It turns out that in this case artificial intelligence is designed to help a person?
That’s right.
Now you have focused on transport, haven’t you?
“Yes, we have. Our University has a large background in transport issues, and software for “smart” drones, avatars for training drone operators is the area where we have a strong scientific school and good cases”.
What kind of drones does Transport of the Future, LLC need?
First of all, agrarian drones. Since they want to establish a new industry, there are many challenges here – for example, requirement for a drone to be in the air for 10 hours, instead of 40 minutes. The question is which engine needs to be “hung” on the drone for this, and at this point a wide range of tasks opens up for our entire University.
Our IT task is developing a drone brain for its efficient and safe operation. After all, if there are several hundred planes currently flying in the country, there will be thousands of drones and UAVs in terms of the entire industry. They will need to be controlled: a special ecosystem, capable of checking codes of drones as strictly as codes of airliners, will be required. We will need to create a mission planner for the extended infrastructure. We will need to control production and operation of drones.
All this is ultimately necessary for the farmer to tell the drone: fly around this field and inform me how much fertilizer should be applied. And the vehicle must obey him. Moreover, the drone should not fly into another’s air corridor. To do this, such a task must proceed to the flight controller, that is, to the drone “brain.
The key requirement for the unmanned aviation industry to emerge and develop is maximal safety of drones for humans and the environment, for important infrastructure. I recall a similar situation in the construction industry, elevators in buildings began to be constructed everywhere, all over the world, only after Otis had invented his automatic catcher – a device that automatically stops the elevator in case of cable breakage. Before that, elevators were very unsafe, which constrained their spread. Therefore, the intelligent system to ensure safe flights of UAVs will undoubtedly stimulate development of the unmanned aviation industry in our country and over the world”.
Who currently controls air corridors for drones?
Nebosvod, LLC. It coordinates all air corridors for drones. And we are now integrating into its system.
What other partners have you developed successful cases with?
A number of our developments are being implemented at the facilities of the Russian Railway. One of them is the vision system for checking railway vehicles. Another technology allows using cameras on unmanned aerial vehicles to monitor the road infrastructure, build 3D models, and identify damage to facilities. The first technology is responsible for controlling unmanned aerial vehicles, which take photos and videos of infrastructure facilities during inspection. The second module is engaged in photogrammetric processing of the images obtained, with subsequent linear and volumetric measurements of the facilities.
What is it used for?
The Russian Railways has the longest track infrastructure in the country, which must be quickly and very carefully maintained. Relatively speaking, 150 people must be involved in checking seven thousand kilometers of the Kuybyshevskaya Railway: they go along it, checking sleepers, rivets, etc. A drone can quickly fly along, record all the parameters, and the job is done. Our neural network detects defects and damages on tracks, buildings and structures and reports them, allowing a person to make a decision on their repair.
So, again, do you mean that artificial intelligence takes people’s jobs away?
“No, I mean that employees turn from “blue collars” to “white collars”. They can engage in highly skilled work – become drone operators, get a new specialization, properly plan the entire complex process. Now the University has a contract for supplying 16 such licenses throughout the Russian Railways. On November 30, our guys went to Moscow and installed this “smart” software for analyzing drone video.
Photo: Irina Kudrina, Olesya Orina
