Augmented T-Shirts Show Twitter Updates

Here's a cool idea from the guys at, t-shirts that when looked at through a web camera show your most recent Twitter update. They simply printed a marker that codes a Twitter account, and let FLARToolKit do the heavy lifting. I don't know why, but this idea just clicks with me. The markers are not that ugly, and if you were able to see the tweets through a cellphone, it could become a fashion statement of sorts.

PaperTweet3d: Augmented Reality T-shirts from squidder on Vimeo.

More info, here.

Total Immersion Augments K'NEX Boxes

Let me first apologize for posting old news, I totally missed that a month ago. Avid fans of augmented reality surely remember that Lego and Metaio are collaborating, setting up specialized kiosks to enable buyers to look at a Lego box and see the completed model in 3d. It seems that Total Immersion won't let themselves be outplayed (sorry) by this collaboration. At CES 2009, Total Immersion used their technology to augment K'NEX boxes. Now, a cool video surfaced on Youtube showing it in action:

There's nothing on Total Immersion's site to indicate that this is a true cooperation, and not just a technology demo, but it's enough to make us update the scoreboard - Total Immersion: 1, Metaio: 1.

Microsoft Demos AR Applications in Techfest 2009

Some cool news come this week from Microsoft. Apparently, the beloved friendly giant holds its annual Techfest this week, where it demos some of the research projects it has been working on. This year, two augmented reality are featured, NoteScape and Core Tools for Augmented Reality.

NoteScape is a project by Darren Edge, from Microsoft Research Beijing which brings us AR sticky notes. NoteScape lets a user attach virtual sticky notes to her laptop screen so that they follow the user wherever she chooses to go. They can be viewed at any time by using a web cam or a mobile phone camera. More info can be found here, but I don't see why the user can't just use a notepad document (or if she's adventurous, a wiki) to jot down her ideas.

Next comes Core Tools for Augmented Reality, which I guess is a codeword for using Photosynth as a service to drive augmented reality. To the uninitiated, Photosynth is a really cool project by Microsoft, where multiple images of the same object taken from different angles and positions are merged together into a point cloud, indicating where each photo was taken, and provide an almost three-dimension reconstruction of the object. All that, without having meta data about where the images were taken (no compass reading), using only image processing techniques. Here's a good video than explains it better than I could ever could.

Now, imagine a room or an outdoor site, previously "mapped" using Photosynth, for example, Piazza San Marco in Venice. When a user takes another photo at the piazza, Photosynth can quickly tell where the user was located and at what direction he was looking at, with accuracy that can rival civilian GPS devices and compasses, leading the way to augment that picture. That's exactly what Michael Cohen, a principal researcher from Redmond, thought of, as he explains in the next video, where he uses this technology for a virtual indoors treasure hunt:

Only downside? This could probably not run on a cellphone anytime soon, but otherwise, I think its a breath of fresh air, especially when compared to other AR demonstrations we recently encountered.

Weekly Linkfest

Other news from around the web:

More on Infrared Marker Based AR

Previously I blogged about a short Youtube video, demoing a project at Beijing Institute of Technology. The project uses infrared markers for augmented reality in maintenance tasks. Few days ago I found this paper describing the project in more detail than provided in its Youtube page. Here again is the video clip:

Basically, they use an infrared projector in order to project (the surprise!) a number of infrared dots (usually 6 of them) on some object, serving as a marker. The hand held device is composed of two cameras, one scenic, and another infrared one. The IR camera is looking for the projected dots, and calculates the distances between them. The device calculates using those distances the alignment and position of the augmented object, and thus can annotate images taken by the scenic camera. Which is pretty cool.
The obvious downsides of this method is the use of two cameras (though, I don't think it's impossible to combine them both into one), and the limited number of markers. Unlike the visual makers we all learned to hate, with their many white and black bits, with the IR markers, one cannot simply rearrange the "dots" in order to get a new marker, since the dots' positions determine the direction of the object. Maybe adding few more dots to the jumble can relieve this problem. Another major problem with this technique is that a projector can only mark a limited number of objects, so if we wanted to implement this method in order to augment museum items, we would need numerous projectors.

Wang, Liu, Wang, Infrared Marker Based Augmented Reality System for Equipment Maintenance, International Conference on Computer Science and Software Engineering, 2008

Tonchidot’s Sekai Camera Spotted in Tokyo

Tonchidot, the Japanese company that created quite a buzz half a year ago when presenting its "Sekai Camera" at Techcrunch50 is at it again. The Sekai Camera is an iPhone application which enables the user to add tags to any place (be it a store or a tourist attraction) and see how other users previously tagged the same place by looking at it through the iPhone's camera. All that, without any image recognition and keeping in mind that the iPhone does not have a built-in compass to detect the direction the user is looking to. No wonder some have thought the following video was contrived and the Sekai Camera is too good to be true :

Guess what, it is too good to be true (but at least it's not vaporware). Techcrunch now reports about a private demo that Tonchidot arranged in Tokyo. Apparently, the Sekai Camera really doesn't know at what the user looks through her iPhone, and she has to pick from the available options:

Many people speculated how Sekai Camera works technically. The answer is simple: The user’s location is identified through GPS (no cell-tower triangulation or image recognition technology is being used). As the iPhone doesn’t have an internal compass, the direction of where the viewfinder is pointed at can’t be measured: Users need to flick fingers left or right to find relevant tags that are around them (as demonstrated in the video I took below). Tap a tag and the information it contains appears in the form of a window, for example a picture with a comment box below it or a voice message someone left earlier.

It can still work and become a great success, but I'll go with Wikitude for now. Anyway, here's how it looks now, you may compare it in your spare time against the previous one


Augmented Reality for the Hearing Impaired

Internet celebrity designer, Mac Funamizu, has done it again. In a new post in his series "Future of Mobile Internet Search", Mac envisions a device that enables the hearing impaired, via some futuristic text-to-speech technology, to see what people around them are saying, and the tone they are using to do so

See many more images, here.

Infrared Marker Based Augmented Reality

There's not much that I can say about it, a project done at Beijing Institute of Technology harness the ability of cameras to see infrared light in order to augment an industrial environment.
The following (subtitled) video demonstrates using this method for maintenance tasks.

Augmented Valentine's Day

The Warehouse, a retail chain store from New Zealand had a neat idea for Valentine's day, especially if you are a lazy boyfriend.
This weekend they published a heart shaped marker on (New Zealand's) "The Weakend Herald"

Using the a web tool, one can create a dedicated message for his girlfriend, to be retrieved by going to a specific URL. Now, the (douche bag) boyfriend points his web browser to that URL, calls on his girl, and lets her watch the paper through the lens of a web camera. To her surprise, flowers and hearts pop out to life, followed by the message of love previously written by her boyfriend. Tears come to the girl's eyes, as never before someone had published his love for her on the weekend newspaper.

Of course, later that day, the girl hears the same story of public love from one of her friends, and what was once a romantic gesture from her boyfriend becomes another selfish act from her ex. Well, at least he didn't buy her a nail-clipper.

Go to this site, to see the unembedable video, and learn more.

Augmented Reality for the Blind

Often, when we talk about augmented reality, we think of ways to augment our reality. The following application for the G1 phone is a noble idea - augment the reality of those experiencing the world in a different way than most of us. Translating shapes and colors into pitch and volume, the application provides (some) information usually not accessible to the blind.

More information (more than you could care for) can be found here. I think this is a great idea, but the implementation could get better. How would you improve the life of others using AR?

MIT's Projected Augmented Reality

A very cool innovation from MIT's labs consisting of a carry on projector, a wristband that can read RFID tags, and a cellphone, enables one to augmented reality like never before. When the user takes a book into her hands, the wristband identifies the book, communicating this information to the cellphone which pulls reviews about the book from Amazon, and then those reviews are projected back onto the book itself.
Furthermore, placing colored caps, the phone's camera (I guess it's the phone's camera, they don't provide much information), can detect the user's gestures, enabling her to draw a circle on her wrist in order to project a watch; Creating a frame using your two hands, tells the phone to take a picture.
It's really amazing, and you must take a look at the following video:

True, carrying a projector around your neck can be considered cumbersome, but that's an amazing proof of concept. Once truly mobile projectors become cheaper, or head mounted displays become more popular, this future vision may turn to our everyday reality.

Link to Wired's report

Is Augmented Reality Over Exposed?

An interesting blog post at Curious Raven, comparing nowadays augmented reality with the early 90's virtual reality. Virtual Reality was in its infancy (and some say, still is) when it hit mainstream, being featured in films, newspaper articles and commercials. The expectations were greater than the technology could support, and virtual reality flopped. Is augmented reality over exposed, is it risking the same fate?

Anyway, Augmented Reality is at risk of getting people excited about the future potential and vision now, while we are still crawling around with basic concepts and marker technology. The industry doesn’t even have a decent lexicon and is generally one giant unexplored area of technology that has not really been trailblazed and pioneered yet. Sure a lot has been done in the last 20 years, but this is still an embryonic market.