picture-231As a developer's conference, GDC also attracts a number of game researchers such as Blair MacIntyre who is an Associate Professor at the School of Interactive Computing at Georgia Tech.

We met with Mr. MacIntyre about his research into Augmented Reality and how it might apply to the iPhone. The demos and possibilities he introduced were truly remarkable.

Augmented Reality is described as a combination of real-world and computer-generated data, where computer graphics objects are blended into real footage in real time.

He and his graduate students have put together some proof of concept demos on the iPhone which could pave the way to future gaming concepts.

This embedded video demonstrates a Augmented Reality Virtual Pet application on a current generation iPhone. The iPhone overlays an animated image of a dog on realtime video input from the iPhone's camera. The result is a pet that appears to reside in the real world in front of you. MacIntyre then shows off a far more sophisticated Zombie shooter (what else?) using non-iPhone NVIDIA reference design platform.

The Zombie-shooter demo serves to show the more impressive possibilities of the technology. In the game, tapping on the screen shoots the zombies in your crosshairs. Moving the camera around and in/out changes your view and if you stay still too long, the zombies start throwing limbs at you.


[ Full HD version | Low Bandwidth version ]

MacIntyre reports that his research has found that those playing these Augmented Reality games with others were more social and interactive as compared to those playing more traditional games against each other.

More extensive demonstrations of the above concepts are provided in these videos: Augmented Virtual Reality Pet on iPhone, Augmented Reality Zombies on Tegra.

Some of MacIntyre's students plan on taking these concepts and executing a true iPhone game based on the technology, especially if the rumors of a more powerful iPhone with advanced video processing capabilities are true. The next generation iPhone is rumored to be released as early as this summer.

  • Chad

    Anyone remember Eye of Judgment?

    http://www.youtube.com/watch?v=ebHilAXNHSY

    Hopefully they avoid trash this time.

    • blair

      Oh yes, I hope we (and others) avoid that. Eye of Judgement had some good points, but the big problem is that the game didn't really _use_ AR for much beyond eye candy.

  • Chad

    Anyone remember Eye of Judgment?

    http://www.youtube.com/watch?v=ebHilAXNHSY

    Hopefully they avoid trash this time.

    • blair

      Oh yes, I hope we (and others) avoid that. Eye of Judgement had some good points, but the big problem is that the game didn't really _use_ AR for much beyond eye candy.

  • http://www.transapp.com transapp

    Wow. I'm fascinated by this topic.
    Found an even more impressing demo of an argumented reality game, which is also planned to be released for the iPhone this year. Check out this YouTube video: http://tr.im/hP6f

    • http://sudokugrab.blogspot.com/2009/03/getting-started-with-version-11.html Chris

      That's pretty impressive. I really wanted to do my Sudoku Grab game as augmented reality - I was planning to show the solved sudoku super imposed on top of the video stream coming from the phone's camera - but there's no way in the official SDK to access that data.

      I wonder how these apps are doing it? If they are using any private frameworks they might not even make it onto the app store.

  • http://www.transapp.com transapp

    Wow. I'm fascinated by this topic.
    Found an even more impressing demo of an argumented reality game, which is also planned to be released for the iPhone this year. Check out this YouTube video: http://tr.im/hP6f

    • http://sudokugrab.blogspot.com/2009/03/getting-started-with-version-11.html Chris

      That's pretty impressive. I really wanted to do my Sudoku Grab game as augmented reality - I was planning to show the solved sudoku super imposed on top of the video stream coming from the phone's camera - but there's no way in the official SDK to access that data.

      I wonder how these apps are doing it? If they are using any private frameworks they might not even make it onto the app store.

  • Phreakuency

    http://www.youtube.com/watch?v=aPENA1Bpm68

    That's how the tech works when it's done well. Very well.

    • blair

      Yes, the EyePet looks pretty cool! I'll be interested to see how it really works when the real version comes out. When you've got a stationary camera, a powerful console behind it, and a flat table, it's SO much easier to do cool things! But, I think you lose a lot when you aren't looking through a mobile device, because the "magic mirror" effect is much less engaging than looking "through" the phone onto the table.

      But, I'll be buying it! :)

  • Phreakuency

    http://www.youtube.com/watch?v=aPENA1Bpm68

    That's how the tech works when it's done well. Very well.

    • blair

      Yes, the EyePet looks pretty cool! I'll be interested to see how it really works when the real version comes out. When you've got a stationary camera, a powerful console behind it, and a flat table, it's SO much easier to do cool things! But, I think you lose a lot when you aren't looking through a mobile device, because the "magic mirror" effect is much less engaging than looking "through" the phone onto the table.

      But, I'll be buying it! :)

  • Rog_Smashy

    You can see an example as well if you have a webcam
    http://www.boffswana.com/news/?p=392

  • Rog_Smashy

    You can see an example as well if you have a webcam
    http://www.boffswana.com/news/?p=392

  • http://nodpad.com No DPad

    Holy crap, this is crazy. Anyone else reminded of that video a long time ago that was proven to be fake that did this sort of effect? Nice to see some real development on it. I think there are some really great possibilities here.

  • http://nodpad.com No DPad

    Holy crap, this is crazy. Anyone else reminded of that video a long time ago that was proven to be fake that did this sort of effect? Nice to see some real development on it. I think there are some really great possibilities here.

  • http://cb.nowan.net/blog Sébastien "cb" Kuntz

    Another game that will commercially be available this summer for iPhones, Windows Mobile and other smartphones by Int13 :

    http://cb.nowan.net/blog/2009/02/17/kweekies-mobile-augmented-reality-game-by-int13/

    • blair

      How can this be made available on the iPhone? There is no video api, and Apple won't let you release a game that uses undocumented APIs. The folks at Metaio have a few apps in the store that use images and their own tracking technology, but they would likely have done "real" AR if they could have.

      Anybody know more about Kweekies? It doesn't really appear to leverage AR very much, just letting you have a little game on the desk around the marker. I'd be curious what else it does.

  • http://cb.nowan.net/blog Sébastien "cb" Kuntz

    Another game that will commercially be available this summer for iPhones, Windows Mobile and other smartphones by Int13 :

    http://cb.nowan.net/blog/2009/02/17/kweekies-mobile-augmented-reality-game-by-int13/

    • blair

      How can this be made available on the iPhone? There is no video api, and Apple won't let you release a game that uses undocumented APIs. The folks at Metaio have a few apps in the store that use images and their own tracking technology, but they would likely have done "real" AR if they could have.

      Anybody know more about Kweekies? It doesn't really appear to leverage AR very much, just letting you have a little game on the desk around the marker. I'd be curious what else it does.

  • Greg

    I've seen demos of this with web cam chat software but never games. I imagine you could do some sweet networked games of controlling little battling robots or something to that effect with the 3.0 software.

  • Greg

    I've seen demos of this with web cam chat software but never games. I imagine you could do some sweet networked games of controlling little battling robots or something to that effect with the 3.0 software.

  • palapalooza

    I can see a game where the players use coin-like tokens on any flat surface. You lay out your coins, wave your iPhone over them and their figures pop up on the screen along with a terrain map full of obstacles and objects. You move your tokens around to fight the other player, move the iPhone around to get a better view, etc. Kind of like that holographic chess game in Star Wars, but you can carry it around in your pocket.

    I'd also like to see stickers that you could put on walls, or on your friends so when you turn the camera on them you see various augmented realities frolicking over their bodies.

    • lance

      That is the kind of thing I would love to see. I am imagining a tower defense game where you could have a set of tokens, one could be the spawn point and one could be the goal, and the others could be the tower placements. You could set your own difficulty depending on how far apart you placed the goal and spawn points respectively. This actually might require a board to define the paths, so you could use a board like settlers where you could rearrange the path available.
      For that matter you could license out games like settlers, risk, ticket to ride, etc.. and create a stand alone iphone game with those. Then I assume you could use the existing boards of those games, that people already have to create a recognizable map for the AR, and have the easter egg ability to play with the actual game boards but without all the messy pieces. Then you could watch little risk armies actually doing battle across your game board and either pass the iphone around to take turns or network a few together so everyone gets their own view of the action.
      There are so many gaming opportunities if the AR works fluidly. What if you could even have a thimble like token with symbols on all sides that would project a sword, lightsaber, etc.. from your finger and you could actually do battle with enemies on your desk? It will be exciting to see as this technology is improved if developers take full advantage of it.

      • blair

        Good idea ... in fact, we just submitted a paper to the SIGGRAPH Games track on just such a game. We had a first version shown at ISMAR last year that used sketching on tiles (http://www.augmentedenvironments.org/lab/research/handheld-ar/artofdefense/). We've since changed it based on testing and design revisions to use tokens (circular, coin-like tokens) in conjunctions with the hex-based marker board. The game is now two-player coop. It's still a bit ugly, but the game was pretty fun. I showed video of it in my GDC talk, and we'll get the video on youtube after the SIGGRAPH review process is done.

      • lance

        Wow. This kind of stuff gets me so excited! I love your game project, can't wait to see the newer version, let us know when the new video is available.

        I am a huge board game player and this fusion of real-virtual gaming environments seems to add so much to the experience. I'd love to pick your brain just to get a taste of what is coming down the mobile AR pipeline. I am a forum member here under Johannvonstranovic if you want to PM me. I am wondering:

        Is it possible to use an existing game board that people already have, like settlers for instance, and program the AR to recognize those tiles?

        Would your game come with a stand? It would help out a lot in making it possible for one player to play. For a multiplayer game board I am imagining a circular board with a track on the outside edge that a stand holding the device could rotate around like a lazy susan. You could have turn based games or coop games where you must quickly rotate the device around to the person with the best view to complete a certain task etc... Or vice versa, the stand holding the device could be stationary, and the board could rotate around in a circle. This would be ideal for one player games.

        Is the virtual sword/lightsaber possible? Can one 3d projection controlled directly by moving the marker effect the other 3d projected elements? Sorry about the poor wording, but hopefully you know what I mean.

        Thanks for listening and for the great AR work you are doing!

  • palapalooza

    I can see a game where the players use coin-like tokens on any flat surface. You lay out your coins, wave your iPhone over them and their figures pop up on the screen along with a terrain map full of obstacles and objects. You move your tokens around to fight the other player, move the iPhone around to get a better view, etc. Kind of like that holographic chess game in Star Wars, but you can carry it around in your pocket.

    I'd also like to see stickers that you could put on walls, or on your friends so when you turn the camera on them you see various augmented realities frolicking over their bodies.

    • lance

      That is the kind of thing I would love to see. I am imagining a tower defense game where you could have a set of tokens, one could be the spawn point and one could be the goal, and the others could be the tower placements. You could set your own difficulty depending on how far apart you placed the goal and spawn points respectively. This actually might require a board to define the paths, so you could use a board like settlers where you could rearrange the path available.
      For that matter you could license out games like settlers, risk, ticket to ride, etc.. and create a stand alone iphone game with those. Then I assume you could use the existing boards of those games, that people already have to create a recognizable map for the AR, and have the easter egg ability to play with the actual game boards but without all the messy pieces. Then you could watch little risk armies actually doing battle across your game board and either pass the iphone around to take turns or network a few together so everyone gets their own view of the action.
      There are so many gaming opportunities if the AR works fluidly. What if you could even have a thimble like token with symbols on all sides that would project a sword, lightsaber, etc.. from your finger and you could actually do battle with enemies on your desk? It will be exciting to see as this technology is improved if developers take full advantage of it.

      • blair

        Good idea ... in fact, we just submitted a paper to the SIGGRAPH Games track on just such a game. We had a first version shown at ISMAR last year that used sketching on tiles (http://www.augmentedenvironments.org/lab/research/handheld-ar/artofdefense/). We've since changed it based on testing and design revisions to use tokens (circular, coin-like tokens) in conjunctions with the hex-based marker board. The game is now two-player coop. It's still a bit ugly, but the game was pretty fun. I showed video of it in my GDC talk, and we'll get the video on youtube after the SIGGRAPH review process is done.

      • lance

        Wow. This kind of stuff gets me so excited! I love your game project, can\'t wait to see the newer version, let us know when the new video is available.

        I am a huge board game player and this fusion of real-virtual gaming environments seems to add so much to the experience. I\'d love to pick your brain just to get a taste of what is coming down the mobile AR pipeline. I am a forum member here under Johannvonstranovic if you want to PM me. I am wondering:

        Is it possible to use an existing game board that people already have, like settlers for instance, and program the AR to recognize those tiles?

        Would your game come with a stand? It would help out a lot in making it possible for one player to play. For a multiplayer game board I am imagining a circular board with a track on the outside edge that a stand holding the device could rotate around like a lazy susan. You could have turn based games or coop games where you must quickly rotate the device around to the person with the best view to complete a certain task etc... Or vice versa, the stand holding the device could be stationary, and the board could rotate around in a circle. This would be ideal for one player games.

        Is the virtual sword/lightsaber possible? Can one 3d projection controlled directly by moving the marker effect the other 3d projected elements? Sorry about the poor wording, but hopefully you know what I mean.

        Thanks for listening and for the great AR work you are doing!

  • http://beesbuzz.biz/ fluffy

    I'm surprised it's taken this long for phone game developers to rediscover this. Toyspring wrote some fun AR games for the Palm Treo, released in a collection called "Arcade Reality."

  • http://beesbuzz.biz/ fluffy

    I'm surprised it's taken this long for phone game developers to rediscover this. Toyspring wrote some fun AR games for the Palm Treo, released in a collection called "Arcade Reality."

  • Gadzooks

    Imagine walking down a street with your iphone, and stores and other things have these key object shapes in windows and such. Holding up your phone would let you see 3d advertising, product listings, or all kinds of creative things.

  • Gadzooks

    Imagine walking down a street with your iphone, and stores and other things have these key object shapes in windows and such. Holding up your phone would let you see 3d advertising, product listings, or all kinds of creative things.

  • http://www.blakespot.com Blake Patterson
  • http://www.blakespot.com Blake Patterson
  • http://loerchner.com loerchner

    I have started a facebook group petitioning Apple to open the API for video augmentation. This will allow agmented reality apps in the store! Please join and pass it on:

    http://www.facebook.com/group.php?gid=105162007611

  • http://loerchner.com loerchner

    I have started a facebook group petitioning Apple to open the API for video augmentation. This will allow agmented reality apps in the store! Please join and pass it on:

    http://www.facebook.com/group.php?gid=105162007611