Online Image Learning - The Next Big Leap in Mobile AR?

Online Image Learning - The Next Big Leap in Mobile AR?

Mobile, image recognition based, augmented reality is very cool, as evident from the Popcode's demos we posted yesterday. However, creation of a model used by the mobile phone to recognize a new image still requires a desktop, hindering realtime creation and sharing of AR content.

Thanks to the work of researchers from the Korean Gwangju Institute of Science and Technology and the Swiss EPFL, this needn't be the case anymore. In a paper titled "Point-and-Shoot for Ubiquitous Tagging on Mobile Phones" accepted to ISMAR 2010, they present a method to scan surfaces and create "recognition-models" by using your phone (no data is sent to a remote server).

You don't even need to take the perfect straight-on picture. As the video below shows, this means you can augment hard to reach surfaces. Best of all, you can share those models with your friends.



A little bit more detail over Wonwoo Lee's blog.

Augmenting Deformable Surfaces

Augmenting Deformable Surfaces

I envy those students that create a video demonstrating their research results and then upload it to Youtube. It's a great a way to show others what have you been working on for all those years. Julien Pilet of Keio University, Japan, is one of those students. He recently uploaded two videos showing off his PhD thesis (done at the EPFL), that explores registering and augmenting non-rigid objects using only one regular camera:



AS you can see in the video above, not only the virtual EPFL sticker bends according to the contours of the shirt, its illumination model is adjusted as well (shadows cast on the shirt are cast on the sticker), and it handles occlusions nicely , most of the times. The only requirements are that the augmented object has a non-monotone texture, that it is locally planar and that it lacks holes.

In the next video, the illumination model is more obvious and a bit trickier to handle, since it now interpolates the shadow cast on a 3d model, from the 2d information in the image:



Pilet's work has been used in several projects. One of them you might recall if you were following AR for the last year. It's Camille Scherrer's "Magic Book":



According to Pilet's, he used standard Macbooks Pros to achieve decent rendering times. For example, handling occlusion on a 2.0Ghz single core computer, gets him a rendering rate of 18 frames per second for a resolution 360*288 pixels (slightly worse results are achieved with his illumination algorithms). So, AR of deformable surfaces is probably not coming soon to an iPhone near you, but Pilet's research is a step in the right direction.