Haptic Augmented Reality

Haptic Augmented Reality

Yesterday, I've stumbled upon the following paper - "Haptic Augmented Reality: Modulation of Real Object Stiffness" (Jeon and Choi), presented last week in the Eurohaptics conference, and thought it is an interesting idea. In the paper, the researcher use a pen like tool to change how stiff objects feel, as a first step for a more general haptic AR.

Suppose that you are drawing something on a table with a penshaped
tool in your hand. Would it be possible that you may feel as
if you were drawing on a smooth piece of paper with a ball-point
pen or on a soft rubber pad with a marker? The tool may also guide
your hand to teach the art of East Asian calligraphy, preferably with
the feel of using a calligraphy brush on a piece of traditional East
Asian paper. Creating such haptic illusions belongs to the realm of
haptic augmented reality, although how to is yet to be researched.

Being so familiar with visual AR, I had hard time thinking of use-cases for haptic AR (apart from the one given in the paper above). For a while, I thought it shouldn't deserve to be called augmented reality. Then, two ideas popped to my head -

• The magnifier glass can be considered as one of the first AR devices, allowing us to see details beyond our normal perception. How about a magnifier glass for the sense of touch? Using a special glove, one would be able to feel every small imperfection in any surface. It can be a wonderful device for antique dealers.
  
• Haptic advertisements. In the last year we saw tons of ads augmented using visual AR techniques, it could be the same for haptic AR. Think about an ice-cream ad that feels cold, about an ad for a tropic resort that feels hot or about a stand-up club having tickly ad.

What do you think, is that augmented reality? Do you have ideas for haptic AR? If so, don't be shy, and add a comment!

Weekly Linkfest

Weekly Linkfest

Some other augmented reality news from around the web:

• Microsoft uses a cheap AR trick to promote the new SQL server
• YDream's "Playing in an Augmented World" (which we mentioned here) wins honorable mention at the MIX09 showoff competition. Congrats Antão and team!
• Games Alfresco: ArToolKit Wins the Oscar of Virtual Reality
• Inspiring low-income youth using augmented reality part one and two, an initiative by these guys.
  

And we could not avoid embedding videos of two of the augmented reality games presented at the Game Development Conference (Ori was first to report about those two):

• Killing zombies on Nvidia's Tegra (by Georgia Tech):
  


• PIT Strategy, an augmented reality board game, by Beyond Reality
  

Oxford Augments your Museum Visit

Oxford Augments your Museum Visit

I know, I used the "* Augments your Museum Visit" template just yesterday. However, unlike Alcatel-Lucent, Oxford really does have a working technology demo to brag around with. Robert Castle of the Active Vision Labratory at Oxford University has posted a few videos earlier this week showcasing his "Parallel Tracking and Multiple Mapping" technique for augmented reality. In a nutshell (and I can't say I understand it in any deeper sense), this technique uses SIFT for object identification and SLAM for camera tracking, in order to identify and augment the current scene.
This technique has won the ISMAR 2008 tracking competition, and the best paper award at ISWC 2008. Apart from that, the following video is cool -



You could see the other 18 videos and read more details here.

Alcatel-Lucent Augments your Museum Visit

Alcatel-Lucent Augments your Museum Visit

As I predicted here, Alcatel-Lucent just posted another AR concept video to Youtbue. The video shows how a visit to the Centre Pompidou would like using an augmented reality device. Nothing really revolutionary is shown, and the interface looks clunky, but I do like the transformation of Picasso's Guernica to three-dimensions.



This is yet another video attributed to Stephane Dufosse, who, according to his Linkedin profile is a business development manager at Alcatel-Lucent, responsible for "business development of video applications" including augmented reality.

Vuzix and Metaio Team Up to Create AR Goggles

Vuzix and Metaio Team Up to Create AR Goggles

Vuzix and Metaio have announced yesterday at the Game Developers Conference an accessory kit for the Vuzix VR920 eyewear. The kit contains a clip on camera (named CamAR) for VR920, which allows the user to see the real world while wearing those goggles, and PhasAR, which is some kind of three-dimensional, six-degrees-of-freedom joystick, in order to interact with the augmented objects. Currently the system looks for fiducial markers, but theoretically, it shouldn't be limited to only those.



The full press release is here.
So, good news for Noha who doesn't need to wait for Vuzix's AV920 Wrap, to play with AR goggles, but of course, he didn't wait and implemented his own version of AR kit for VR920 a while ago.
Updating the score board: Metaio: 2, Total Immersion: 2.

Update: Ori shot the following clip at GDC -

Alcatel-Lucent's Augmented Reality Efforts

Alcatel-Lucent's Augmented Reality Efforts

Something is brewing up in Alcatel-Lucent. The company that tried last year to bring second-life avatars into the real world (video here), but failed to impress the crowds, and had some cooperation with Total-Immersion (video here), is now pushing onwards in multiple AR frontiers. First, there's Alcatel-Lucent Touchtag initiative (until last month it was known as Tikitag). Touchtags are cheap ($1 per tag at Amazon), programmable RFID tags that Alcatel-Lucent hopes will help create the internet of things (here's a commercial).
Moving on to visual markers based AR, Alcatel-Lucent presented their "Enhanced Reality" application for Symbian cellphones at Barcelona's GSM Mobile World Congress last month -



And then, there's this video Alcatel-Lucent released a few days ago, presenting their vision for the (near) future



I have tried contacting Stéphane Dufossé (the Alcatel-Lucent manager who posted the last video) for comments, but failed to get any reply. However, it's clear that the behemoth company is interested in AR and invests resources in this field. After relaunching Tikitags as Touchtags and posting those videos to Youtube, I'm expecting to see more of them in the coming months, making their AR strategy clear.

Weekly Linkfest

Weekly Linkfest

Some other news from around the web:

• Sergey Ten opened a mobile AR group at Linkedin.
• Games Alfresco: Games of the future are mostly augmented
• The World Wildlife Foundation (WWF) is going in the way of GE - Augmented earth hour
• Looking for an AR job? Neogence is looking for an OpenGL guru.
• Looking for an AR research position? The Friedrich Wilhelm Bessel Institute in Bremen, Germany is looking for a researcher on the field of augmented welding helmets.
  
• An audio recording of SXSW's "Emerging Trends of Mobile Technology" panel, previously mentioned here.

Quote of the week:

Designers may be used to the technology behind augmented reality and understand how an icon on paper can be transformed into a 3D animation you can move yourself. But we must never forget how our target audience thinks. For ‘the masses’ augmented reality feels like magic. They don’t think in technologies, but just experience the result. And when they do this, they are overwhelmed by the coolness of augmented reality.

From "The viral effect of augmented reality" at Johnny Holland Magazine.

And just to spice things up, here's a video of an augmented fossball table using Goblin XNA someone did as an homework assignement -



Have a nice week!

Sekai Camera for Android - Still Looks Terrible

Sekai Camera for Android - Still Looks Terrible

Once again Techcrunch (or rather MobileCrunch) has an exclusive on Tonchidot's Sekai Camera (here's last time). This time, the news are coming from an Android event held by Nikkei, apparently Japan's leading business newspaper, where the first version of Sekai Camera for Google's Android was exposed. Unlike the iPhone version, this one utilizes the built-in compass to narrow the number of items displayed on the screen. The result is still far from impressive



Yes, I know this is only the first version, and things will look better when the application is released. However, I still suspect that without any image recognition, the usefulness of Sekai camera will be limited to the outdoors (where it would compete with Wikitude), whilst it would need a way to filter spam in indoors environments.

Robert Rice: Augmented Vision and the Decade of Ubiquity

Robert Rice: Augmented Vision and the Decade of Ubiquity

Robert Rice wrote a very inspiring article on Curious Raven today. In it, he discusses the marvelous future we are facing and some of the hurdles ahead:

The Decade of Ubiquity is defined as the next ten years where every aspect of our lives will be permeated by digital, mobile, media, data, information, augmented, virtual, and so forth. It will be everywhere and accessible almost instantly. Everything will be connected, labled, monitored, tracked, tagged, and interactive to some degree or another. We will break away from the desk, we will throw away our monitors, and our children will laugh at how large our IPhones are. They will struggle with how we ever managed to get work done with “windows” “webpages” and keyboards. They will be unable to fathom the concept of vinyl disks, typewriters, and landlines. But it all starts, and accelerates, during this next decade. Imagine everything that happened in the last decade, and multiply it. You haven’t seen anything yet. The next decade will make the last one pale in comparison.

He then calls for action:

The world is nearing another dramatic paradigm shift and explosive growth in technology and economics, but we need to wake up. Demand more, better, stronger, faster, smaller. The future is ours to invent. Don’t be satisfied with mediocrity or lazy development.

I would like to suggest another call for action. Augmented reality and ubiquitous computing are coming fast our way. As Rice notes, they will change almost every aspect of our daily life. We cannot afford that the infrastructure for such a revolution will be controlled by a few, whether those are giant corporations such as Microsoft or Google or small companies like Rice's Neogence. For augmented reality to be truly successful, we need an open environment, where developers, artists and visionaries can create applications without total dependence on the infrastructure providers. As web developers are not committed to routers manufacturers, AR developers should be free to create whatever they deem right, without the prior approval of the patent holders and infrastructure overloads. We need an OpenAR initiative, and we need it now.

Have you written a paper for ISMAR09 yet?

Have you written a paper for ISMAR09 yet?

Ori of Games Alfresco has recruited me, and others to remind you, dear AR fanatics, that now is the time to write a paper for ISMAR09 (deadline is less than two months away). The best thing - you don't have to be a computer scientist to do so (though they are always welcomed). Quoting Ori:

The call for papers is on, and this year it targets well beyond the typical research papers audience and into interactive media and art folks.
There are plenty of opportunities such as:

• Art Gallery
• Demonstrations
• Tutorials
• Workshops
  

It's a huge opportunity to give AR the break it needs.

Check the call for papers, and let us all meet up in Orlando!

Playing with AR at YDreams

Playing with AR at YDreams

Yes, I linked to the following video on Sunday


Playing in an Augmented World from YDreams on Vimeo.

Usually I refrain from re-posting old news, but Antão from YDreams (the guy in the red shirt) has contacted me and gave some technical info, so I'll let this one slip by in order to encourage others to send me news and other AR tidbits. My address is right there on the right panel. Anyway, so says Antão:

The applications use face and blob detection. Although the virtual objects move on a 2D plane, they are 3D objects and react using 3D physics. The interaction can be extended to be in real 3D. The applications take advantage of the modern multi-core CPUs (computer vision, behaviors and physics run in separate cores) .
...
We are using a proprietary development platform, codenamed YVision. It was developed on .NET 3.5 and uses open-source libraries like OpenCV, Ogre and ODE. The computer vision algorithms are the ones supplied by OpenCV, with a little magic added by us. ;-)
The applications were running on a Intel(R) Core(TM)2 Duo E7200 @ 2.53GHz with an NVIDIA GeForce 9400 GT (a regular off-the-shelf PC). Computer vision running on one core, while behaviors and physics were running on the other core.

They have submitted this video to MIX09 ShowOff contest, so feel free going over there and to vote for them.

Augmented Reality Goes Underwater

Augmented Reality Goes Underwater

Fraunhofer, the German institute we met at CeBIT , presented a few days ago their newest project, involving augmented reality for scuba-divers. After all, when you're scuba-diving, you already wear goggles, so they might as well be AR goggles (and I thought the underwater world is beautiful on its own).

A diver's mask with a special display lets the diver see his or her real submarine surroundings overlaid with computer-generated virtual scenes. In the pilot application, an AR game, the player sees a coral reef with shoals, mussels and weeds, instead of a plain indoor pool. Applications for professional divers are being investigated.

However, the implementation is a little bit lacking. The divers need to carry an ultra mobile computer on their back, and visible markers are used as reference points for the AR algorithm. In the future, Fraunhofer are planning to adapt this setup to aid in underwater maintenance tasks.

Fraunhofer press release is here, where you can read further details about the technology and the game using it.

Augmented Reality War Game

Augmented Reality War Game

Yanko Design reported on designer Frantz Lasorne's thesis project for his master in interaction design (I hope I get this right, my thesis work was on a much duller subject). Lasorne is using Virtools and the ARToolKit for creating (it's still a work in progress) a war game where two children can have virtual armies fight each other on an augmented battlefield (meaning, their parents' living room). Real life items become obstacles for the armies and the goal is to kill the opponent using every mean made possible by the game rules and the battlefield. It's better than Warcraft!


Augmented Reality Toys (Work in progress) from Frantz Lasorne on Vimeo.

According to Lasorne, he combined 3Dvisor's glasses with a web camera in order to have a makeshift AR visor, and he hopes to present the full project at this year's Laval Virtual (22 April). If he is successful, this could very be a defining moment for AR games, so let's wish him goodluck.

Via Yanko Design where you can read the rules of the game.

Augmented Reality in SXSW 2009

Augmented Reality in SXSW 2009

I hate those short posts that don't have real content. However, since my traveling budget doesn't allow me to attend SXSW 2009 in person (or any other conference), I'll settle for quoting others this time.

On Saturday, a panel named "Emerging Trends of Mobile Technology" was held, with two directors from Sapient, and one Ryan Stewart.
Among the emerging trends are applications stores, mobile Flash, augmented reality and image recognition.

What's the difference between AR and image recognition? The first will allow you to "play ping pong in the middle of the street using your phone as your control and screen", while the latter will enable you to "snap a photo of your favorite store’s logo, and its Web site will pop up" or "point your phone at a monument and it will tell you information about it". Oh. Probably I should rename this blog to "Image Recognitions Times" :)

Sources are here, and here. Couldn't find a recording of this session yet, but if you know of any, please leave a comment.

Weekly Linkfest & Site News

Weekly Linkfest & Site News

Some other AR news from around the web:

• Mobilizy the company behind Wikitude, is now developing an application named Zenith to augment your night skies. "You point your camera at the night sky - and get a star map", according to this article from the BBC. A very cool (and relatively simple) idea!
• Robert Rice relaunched Neogence's site. Still not a lot of info about what they are all about.
• YDreamers are playing in an augmented world.
• In 2002 we were promised augmented reality navigation systems for our cars. I, for one, am still waiting.
  

Some news concerning this site:

• Please welcome our newest tag - "Novelty AR". I'll use this tag to mark any post that concerns augmented reality that was done in the sake of augmented reality and not much more. Usually this definition fits all those AR projects that augment a very specific item, and don't really augment reality, such as Topps' augmented baseball cards, the Mini ad and GE's smart grid. I took the term "Novelty AR" from Robert Rice who defined it as “cool for five minutes but ultimately a waste of time.”
• If you have any AR related news, please contact me. My address is right there on the right panel.
  

ARForce - another Infrared AR Concept

ARForce - another Infrared AR Concept

The following video tells it all (but I'll write a short blurb to justify my non-existent salary). ARForce is a sensor coming from the university of Tokyo, that can tell the position, magnitude and direction of the force applied to it by the user. It also has multi-touch capabilities.
ARForce is embedded with an array of dots, visible under infrared light and a camera. When pressed, the dots slightly realign, enabling the calculation of the pressure vector. Different initial arrangements of the dots can identify different input devices.



Though such a device is not strictly related to AR (apart from its name, and the demo presented above), it does show the viability of infrared markers, at least for indoors AR. Any thoughts on the subject?

Futuristic Vision of Projected Augmented Reality

Futuristic Vision of Projected Augmented Reality

Mac Funamizu, an internet star designer we previously featured here, has posted his own take on projected augmented reality. According to Funamizu, his worked was not influenced by MIT's projected AR, and he did come up with some unique use cases.
Using a special wristband (codenamed Snowcorn), one can seamlessly get a definition for any word encountered in a book (apparently, books are still present in the future):

One can even plan a train ride right on a map:


More images at Funamizu's post; Images reposted here under Creative Commons' Attribution 3.0 Unported.

Two Interesting AR Talks

Two Interesting AR Talks

Short update for today (I need some rest, too many posts in the last few days).
First, TED uploaded MIT's Pattie Maes' talk, about projected augmented reality, which we talked about here (and so did the rest of the blogosphere).



Next, Ori Inbar of Games Alfresco, has uploaded his talk from the WARM 09 conference. In his talk he discusses his vision of AR games as an educating tool (if I got it right), and why we should target kids as the early adopters of augmented reality. It's really interesting, though I'm still not that sure AR is the best technology for enriching kids' education experience (see this talk from TED 2009 for another compelling concept). Ori's talk is embedded below, but you should go read his post and the comments following it.


Augmented Reality Today: WARM '09 from Ori Inbar on Vimeo.

New Video from the Artvertiser

New Video from the Artvertiser

The guys from one of my favorite augmented reality projects, the Artvertiser have published a new video (well, actually, it's about a week old, but it's new to me). The Artvertiser, unlike all those pseudo AR projects we recently reported about, truly aims at augmenting our reality. The concept is quite amazing, using a hand held device, outdoor advertisements are replaced with art pieces. After all, ads are the prefect canvas for AR - they can be (relatively) easy to identify, they are everywhere, and most of us will not miss them.


The Artvertiser: Urban beta - Report 1 from Julian Oliver on Vimeo.

True, this demo was done using a PC and a high-end camera, but for a project that is barely one year old, it seems very promising. Moreover, Julian Oliver, Clara Boj and Diego Diaz, who are behind the project, aim to bring it to either Android or Symbian based phones by the end of the year. I for one, wish them good luck with that!

Augmented Baseball Cards

Augmented Baseball Cards

Total Immersion and trading cards company Topps have struck a partnership to augment Topps' baseball cards. Starting today (Monday), collectors that have special "Topps3DLive" baseball cards, can point their web camera at them, and see a three dimensional rendition of the player featured on the card. Moreover, they can use their keyboards in order to play some simple games featuring those 3D avatars.
The implementation seems to be quite quirky, you have to go to this site, download a plugin for your browser, choose the card that is in front of your camera (can't they identify it themselves), and only then you are able to experience the "magic". Anyway, it's supposed to be a breath of fresh air for a dying industry:

Topps needs to augment reality because baseball cards are struggling in the Internet age. Today’s collectors, most of whom are still boys, can just as easily and less expensively find the sports facts they want online. (source)

Unfortunately, there is no video I'm able to embed, so you'll have to go to the NY Times article, or to Total Immersion's site, to see how it looks like. If you happen to work for Total Immersion, please contact your PR division and explain them about the power of blogs. Nevertheless, we can update our scoreboard: Metaio: 1, Total Immersion: 2.

Updates:

• Engadget uploaded a video that I can legaly embed -
  
  


• Blair of "Living in an Augmented Reality", has a nice piece on the subject.
  

  The problem with this concept is not the technology (although the graphics are pretty lame). Rather, it is the failure to seemingly even consider trying to answer the question I drill into my students: So What? Why would anyone care?

  
  

Weekly Linkfest

Weekly Linkfest

Some other news from around the web:

• Augmented reality drum kit usign FLARToolkit, from Squidder.com
• Arnold Schwarzenegger and Angela Merkel are playing with augmented Lego (via Games Alfresco)
• Microsoft imagines the technology of 2019, and it's going to be an augmented future (via istartedsomething)
• How popular is augmented reality right now? It's too popular! Japanese spammers add "augmented reality" to the description of some of their erotic videos on YouTube (here's a slightly NSFW one). Apparently, augmented reality is one of the top tags right now, thanks to that GE campaign.
  

Augmented Reality at CeBIT 2009

Augmented Reality at CeBIT 2009

Fraunhofer, the German research organization that brought us the MP3 format, is presenting two augmented reality technologies in CeBit this year.
First is a nifty magic mirror, presented in the video below (no horizontal, English version yet, sorry):



The mirror is actually a touch screen, a camera, and an algorithm that samples the camera feed 30 times per second. The algorithm looks for the original logo imprinted on the shirt in order to replace it with another logo (or even with a video), while changing the shirt's color at the same time. Once you understand it's marker based, the demo looks less cool.

On the other hand, Fraunhofer demoed a mobile AR technology, aimed to augment historic tourist attractions. In the future, using an application installed on an iPhone (or currently on a mini Sony Vaio), tourist will be able to examine the Reichstag or Brandenburg Gate and see them as they looked through the years.

The software recognizes the buildings, as well of the perspective of the shot and delivers the historical equivalent of the photo, covering the original. The rendering of the historical snap is done is real-time – so if the user walks around the building, the picture moves with him. The picture is sent to a server via the 3G network, which delivers the “Augmented Reality View”.
Views of the Reichstag just after the Second World War, during the separation of East and West Germany, or after the fall of the Berlin Wall are available. The user can scroll back and forth between the different time periods and short explanatory texts round off the offering. (source)

Fraunhofer presented the same concept for ancient Rome in SIGGRAPH 2008, but it seems their technology has matured since then.

Ubisoft and Microsoft augment your messenger

Ubisoft and Microsoft augment your messenger

It's been a rather slow news week, but Ubisoft and Microsoft France may bring some spice to its end. The two companies announced the integration of Ubisoft's YouUp with MSN Messenger.
Ubisoft has been developing YouUp, a technology that relies on face detection in order to insert virtual objects into web camera feed, for the last couple of years. An early demo of YouUp can be seen below -



Following the integration with Messenger, users can now have video chats with an added fun factor (if this is your sort of fun). As Yves Guillemot, chief executive officer at Ubisoft puts it (source):

YouUp will allow users to truly enjoy chatting via webcam. This playful application is fun, offering disguises and hilarious 3D effects that adapt easily to each user’s face

I know, it's not a ground breaking technology (though it seems to be well implemented), but it's further evidence that augmented reality is creeping its way to the mainstream.

(download it here (in French))

Weekly Linkfest

Weekly Linkfest

Other news from around the web:

• Discovery on Augmented Reality. I like the fifth comment - "Moreso, as I think after one whole generation is brought up in an AR world, they would stop valueing image and fashion autogether. It would be the ultimate victory of function over form."
• Boingboing's Offworld: Introversion's DefconAR: mutually destructive augmented reality. I don't know where they are going with this, but wish them good luck.
• ISMAR 2009 call for papers, deadline is May 16. (via Games Alfresco)
• Toyota and Nissan envy the Mini AR ad. (via Games Alfresco)