Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how Google's Objectron uses AI to track 3D objects in 2D videos. Many people may not know much about it. In order to make you understand better, the editor has summarized the following content for you. I hope you can get something from this article.

The Google team supported by Objectron has developed a toolset that allows annotators to use split-screen views to display 3D bounding boxes (that is, rectangular borders) of objects to display 2D video frames ‍ ‍

With the start of the 2020 TensorFlow developer Summit, Google today released a pipline called Objectron, which can find objects in 2D images and estimate their posture and size using the AI model. The company says this pair of robotics, self-driving cars, image retrieval and augmented reality-for example, it can help robots in factory workshops avoid obstacles in real time.

Tracking 3D objects is a tricky prospect, especially when dealing with limited computing resources, such as smart phone systems on chips, due to lack of data and diversity, it becomes more difficult to calculate the appearance and shape of objects when the only image available (usually video) is 2D.

The Google team supported by Objectron then developed a toolset that allows annotators to display 2D video frames using a split-screen view of the object's 3D bounding box (that is, a rectangular border). The 3D bounding box is superimposed on top of the point cloud, and the annotator draws the 3D bounding box in the 3D view and verifies its position by looking at the projection in the 2D video frame. For static objects, they only need to annotate the target object in a single frame. The position of the object is positioned to all frames using the real camera pose information from the AR session data.

To supplement real-world data to improve the accuracy of AI model predictions, the team developed an engine that places virtual objects in scenes that contain AR session data, so that camera posture can be used to detect planes and estimate them. Lighting to generate physically possible locations that match the scene, resulting in high-quality composite data whose rendered objects respect the geometry of the scene and seamlessly fit the real background. In the verification test, the accuracy of the composite data has been improved by about 10%.

Better yet, the team says the current version of the Objectron model is light enough to run in real time on flagship mobile devices. Phones such as the LG V60 ThinQ, Samsung Galaxy S20 + and Sony Xperia 11 are equipped with Adreno 650mobile graphics chips, and the ‍ can handle about 26 frames per second.

‍‍‍

‍ Objectron is available in MediaPipe, and MediaPipe is a framework for building cross-platform AI pipes consisting of fast reasoning and media processing, such as video decoding. Models that are trained to identify shoes and chairs and end-to-end demonstration applications are available.

The team said that in the future, it plans to share other solutions with the research and development community to stimulate new use cases, applications, and research work. In addition, it intends to extend the Objectron model to more categories of objects and further improve performance on their object devices.

After reading the above, do you have any further understanding of how Google's Objectron uses AI to track 3D objects in 2D videos? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.