Technology

A marker-less augmented reality method that is operable in mobile applications is proposed in this invention. In particular, the invention works through a projection of color-coded light on a 3D object. A camera captures this image, which is then analyzed to obtain the correspondence between the deformation of the observed light pattern with respect to the projection and reveal 3D information on the object. The 3D information then becomes part of the initial feature map of the 3D object and be tracked from frame to frame for reality augmentation, where the feature map, 3D mesh and camera pose are updated. The method is tuned for mobile applications where factors such as speed are preferred over accuracy. The confidence level of the solution is ensured through the use of two-dimensional, eight-color DeBruijn spatial grid and two-level color label correction process. The DeBruijn encoded color pattern is used to perform rapid one-shot, real-time 3D (three dimensional) reconstruction with a focus on local optimization. Color detection and correction is carried out using a special vote majority rule while a Hamming distance minimization method enables finding missing topological information very quickly. All together help reveal accurate 3D information that is important for object tracking and pose estimation. Simulations carried out using this method show that it is very efficient and achieves desirable accuracy even when applied to complicated scenes.

Potential Application

Augmented Reality (AR) enables real-world images to be enhanced with virtual reality (VR) by combining live video imagery with computer-generated graphics. The incorporation of features in smart phones such as GPS, cameras and image processing capabilities has given new life to the use of AR in various applications, especially those consisting of special headsets that use mobile devices as the displays.

n addition to entertainment applications such as gaming; AR is finding uses in a variety of other areas - training technicians to conduct repairs of equipment and machinery in different environments, assisting medical personnel in preparing for surgeries and enhancing the experience of sightseeing and viewing objects displayed in museums.

Opportunity

The market for AR and VR is being driven by the mobile phones market. Mobile AR applications are expected to drive AR/VR growth to US$ 108B by 2021. Internet, social media and gaming enterprises such as Google, Facebook and Sony are leading the development and incorporation of AR methods and applications in products such as the Daydream View, Oculus and Playstation VR. The mobile phone is expected to be the standard platform for AR/VR vs. other platforms such as eyeglasses. Rowan University is looking for a partner for further development and commercialization of this technology through a license.