Shulou(Shulou.com)06/02 Report--

Machine vision is definitely one of the hottest AI technologies, while 3D machine vision is probably the most popular one.

The so-called 3D machine vision refers to the understanding of the three-dimensional model in the three-dimensional space on the basis of the general machine vision technology. This technology not only involves AI, but also a cross-discipline of machine vision, graphics and data perception technology. If you think about it, it is very valuable to let the machine know the 3D picture. After all, people's visual understanding is 3D, and 3D is a difficult problem that must be overcome in order to make AI as close as possible to human perception.

In the industrial scene, 3D machine vision is an important technical condition of unmanned and high-precision maps, and it is also widely used on VR/AR, and UAV aerial photography and mapping are also inseparable from this technology. After Apple hyped the 3D structured light technology, 3D machine vision in mobile phones has also become a must. If these powerful technologies are racing cars, then 3D machine vision is the toll booth they will encounter on the road.

At today's machine vision summit, almost half as many papers are related to 3D. Cutting-edge exploration is going on like crazy. However, the point is that in this field, it seems that most onlookers still know only one structured light.

What are the new technological trends of 3D image + machine vision that are overlooking the world in the fog of the unknown today? Today we are going to talk about some technological breakthroughs with a sense of science fiction. Maybe these capabilities will show up in your phone, VR devices and drones next year, or it may be a start-up craze kissed by capital crazily.

3D data perception of super-large scene

3D machine vision includes many aspects, including not only letting agents understand 3D data, but also how to obtain 3D model data through machine vision solutions.

In the traditional sense, 3D data acquisition, or 3D sensing technology, can generally use multi-angle photography or depth sensor to achieve 3D data collection. The limitation of this technology is that the 3D data collected cannot be too large.

However, with the continuous upgrading of 3D data requirements, the 3D data perception of super-large scenes is becoming a hot topic. For example, the high-precision map of the city used in self-driving can be seen as the splicing of super-large 3D scenes. Many urban data deductions used in the field of smart cities should also be rooted in the collection of urban 3D scenes.

Machine vision is providing many new ways to perceive 3D data in super-large scenes. For example, automated imaging methods, such as visual SLAM online processing of continuous frames of images, to achieve real-time reconstruction of a huge 3D scene. For example, point cloud segmentation and semantic understanding of point cloud data from aerial photography data can help to obtain urban 3D data quickly and cheaply.

Overall, the 3D data perception of today's super-large scenes has three main application directions, which are likely to become new investment and entrepreneurial hotspots in their respective technology fields:

1. 3D high-precision model of buildings is used in the fields of engineering supervision, intelligent design, logistics and intelligent city.

2. The combination of high-precision map and 3D data perception is an important part of self-driving.

3. 3D modeling of indoor and outdoor integration, which is of great help to smart home design, environmental monitoring and VR/AR experience.

Mobile phones and 3D vision enter the honeymoon period

Today, 3D structured light has become a well-deserved standard for high-end mobile phones, and has become another hot spot in the mobile phone industry after double photography, triple photography and off-screen fingerprinting.

However, 3D vision technology on mobile phones is far more than structured light, from algorithms, sensing hardware to image system solutions, may become the key factors for further coupling between mobile phones and 3D vision.

Today, there are three related trends that are likely to be hot next year. One is the acceleration of 3D vision algorithm based on chip, which is likely to be the next evolution direction of mobile phone AI. The second is that 3D vision algorithms with high error correction ability will be popular, and the realization of 3D data collection and local modeling in the terminal will become a new hot spot. The third is that the addition of rear cameras to 3D vision solutions is just around the corner.

From the chip side to the development platform, and then to data sets and sensor systems, 3D machine vision is affecting the trend of mobile phone battlefield in many fields. Storing technical weapons in these areas is also likely to become the key node of mobile phone battlefield fighting next year.

Pose estimation Technology in AR/VR

Why when we play the AR experience, we often feel that the things in the phone are not real in the view of the camera, as if they were floating on the floor.

This is because the pose estimation algorithm is not accurate enough to correctly locate the spatial relationship of the object. With the evolution of machine vision technology, many pose estimation techniques are evolving synchronously today. For example, based on the dynamic feature extraction algorithm, the action location achieved today has been relatively mature.

This thing sounds mysterious. What on earth is it for? Its biggest application scene is to correctly deal with the spatial relations and motion trajectories of dynamic objects in the scene in VR/AR. For example, when you play a football game in VR, where the ball should be kicked and what trajectory it should hit the wall depends on the pose estimation algorithm.

With the help of machine vision algorithms, more precise pose estimation is coming, which accelerates the arrival of a mature MR experience. On the other hand, in VR devices or mobile phones, pose estimation based on the cooperation of cameras and sensors is also a bright spot in immersion technology.

Achieve 3D modeling by scattered data

The most important thing about 3D machine vision must be to achieve 3D modeling based on data. This application is very important in the industry. Geographic information systems, exploration, engineering, and self-driving all require a lot of 3D modeling work to participate.

And consumer-level 3D modeling is also coming today, we can already see in the mobile phone side through 3D structured light to complete data collection, so as to achieve 3D modeling play.

Like 3D perception, 3D modeling uses cameras or sensors to collect data and eventually complete the modeling through different solutions.

However, there are still many problems to be solved in this area. For example, when we do 3D modeling today, we still need to collect data a little bit painfully, and we must ensure the alignment and precise arrangement of the data. Otherwise, the 3D model will be disorganized. This obviously weakens the public's enthusiasm for 3D modeling and adds a lot of difficulty to many engineering-level projects.

The arrival of AI is helping to change that. With the help of deep learning algorithm, the field of machine vision is studying how to complete 3D modeling in scattered, irregular and huge amount of data. This requires a lot of solutions such as antagonistic generation and a priori representation, but the results are worth looking forward to.

For example, today there is a 3D modeling scheme to reconstruct the dense forest with the help of deep learning. However, in the image data used for point cloud modeling, there are many parts that are obscured by leaves. At this time, AI can be used to enhance the prior knowledge of 3D modeling and actively "brain supplement" the true appearance behind the occlusion.

Not only to repair occlusion models and defective data, the integration of machine vision technology and 3D modeling, but also to make many unmanned devices have brighter "eyes". Self-driving cars, for example, may be based on the 3D modeling algorithm in the brain to make up for environments that have not yet been discovered by smart cameras. This is particularly useful in complex overpasses and parking lots.

On the consumer side, the combination of 3D modeling and machine vision will also bring new imagination, such as consumers can reconstruct accurate 3D models based on photos, or collect data used to complete modeling foolishly. The imagination behind this change is amazing to enable less professional people to build professional 3D models.

Better depth sensor solution

There is also a convergence of machine vision technology and 3D, mainly in the field of drones.

Today, when surveying, mapping and aerial photography, drones must be accompanied by the ability to understand space, otherwise it is not allowed to take pictures, and it is a big deal to hit the south wall. This ability mainly comes from cameras and sensors for spatial reading.

With the continuous upgrading of consumer-grade UAV, the demand for UAV shooting effect is also increasing. Drones must constantly take pictures over longer distances, in more extreme weather, and in more complex movements. However, the traditional sensor system solutions are almost unable to keep up with the expectations of users.

Today's consumer drones generally adopt two perception solutions, one is binocular vision technology, such as some products of DJI, and the other is structured light sensors, such as Microsoft's Kinect. These two mainstream schemes have certain limitations, such as the perception range is limited, and it is difficult to complete long-distance tasks. For example, binocular vision technology will fail in the dark, so the night shooting of drones has always been a big pit, but structured light technology can not cope with the strong light, and Shi Lezhi is also very worried about drones at noon.

A better solution is to combine sensors with smart cameras to achieve a new sensing system solution that can adapt to different weather and weather and can perceive over a long distance.

Today, using many algorithms in machine vision technology to coordinate the work of different sensing devices to turn drones into "multi-eye drones" is becoming a popular solution. Machine vision algorithms add a large number of UAV sensors, which may also improve the trajectory shooting ability of UAV to capture the overall environment, or the ability to accurately capture dynamic objects, such as moving animals and vehicles.

The above technological trends may become the next hot spot in the application of machine vision and graphics. This field may seem biased, but in fact it can affect the changes in today's technology market.

The game for machines to see the three-dimensional world has just begun, and machines and humans can one day stare at each other from the same perspective, which may be the end of the story.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.