TPR10: ugmented Reality�Merging the Virtual World into Ours

All right. We're going to go through a variety of things here. So let me just kind of dive in to see what's going on.

First thing you need to know: everything is uploaded already. It's online and you can actually go to the conference website, the presentation links are already there. And all my presentations are available off my website here. That will be available again at the end of the slides so you don't need to write it down right away. But I do keep everything together.

So, Augmented Reality. We all have a pretty clear idea or at least a basic understanding of what we consider reality, so what the heck's the augmented part? Well, you have to kind of step back a little bit. So for example, we'll do this.

[Demo Music]

Dan Frommelt: And we'll replay that.

[Demo Music]

Dan Frommelt: OK. So, that's a major chord. Let's augment that.

[Demo Music]

Dan Frommelt: It's a little dissonant. Additional notes are in there. More meaning, more information supplementing. The major chord is still in there, but additional information is in there, too.

So really that's what we're talking about. There's a difference between what we do everyday like driving down the street. What could we do if it was augmented? There is a whole lot more you can get. Sometimes it's a distraction, sometimes it's a lot of beneficial information. It depends on how we interpret and interact with that particular piece. Let me clean up as we go along here.

So that's what we want to talk about. How do we augment our current reality? It makes something that is greater than before. Notice it doesn't say 'better'. It says 'greater'. It is increase because we're adding more information. We are amplifying. There's all different ways to do it. And we can also improve.

So we're actually enhancing, modifying, improving our current reality, and we still, believe it or not, we actually have Augmented Reality in use today that everybody is understanding and they know.

For example, if we do a heads-up display--now this is a very advanced concept. If I'm driving down the street and I see a business and I have some kind of augmentation device that I can look through, I can get supplemental information added on to what's going on.

Heads-up display actually exists already. Our fighter pilots have it. It's available on some automobiles. It actually projects up direction, gas, mile per hour, little basic information for the driver. It's like a mini-cockpit.

It's not that complicated of a technology. It's just not really necessarily available to the masses. We aren't walking around with little helmets on. You know, "Hey, greetings. I see your IM."

[Laughter]

Dan Frommelt: Laugh. It might come really soon, though. NFL first down marker! That yellow line really doesn't exist, just in case you haven't been to an NFL game.

[Laughter]

Dan Frommelt: My son was confused. "What?" [Laughter] Yeah, it was kind of a riot. "Why does that yellow line come out?" I was just, "Never mind."

The puck marker. How many people here watch hockey? You can't see the puck on TV, I'm sorry. If they don't put that little tracer on there, you're gone. You'd go, "What? What's going on?" It's like watching a fly, you know?

Museum tours. This is actually really interesting. I went on a museum tour up on the Pompeii Exhibit up in the Twin Cities. This is a while ago with my children. And we got there and they have the audio augmented system. You kind of plug in the headphone. But this isn't the one where you push 'play' and go to the next thing. You walked around and it knew where you were, and if you stopped in front of something, it would start with a story.

And then you walked over, "Oh!" then it carried on to something else. It knew where I was in the exhibit and would augment my interpretation of that at that time. The kids loved it! They spent more time walking around that stupid thing. Here I thought they wanted to go see the dinosaur bones. No. The technology that talked to me and told me what was going on and told me a story meant more to them. Then once the batteries went out, dinosaurs.

[Laughter]

Dan Frommelt: So that's all right.

So, really what we're looking for when we're using augmented technology, we need some kind of simple input/output mechanism. So we're looking at a video, input and output, and an audio, input and output. That's all we really need. Although who wants to go around wearing that? I mean, looks like Robocop reject or something.

We're getting closer, though. These things are more and more prevalent in our society, and we don't really even realize it yet. Most of you actually might have an Augmented Reality device on you now.

This actually started a while ago--I've been looking at Augmented Reality--and at the Presidential election, CNN released this little thing. It was really kind of cute. They're calling it a hologram. As soon as it came on TV, I'm like, "That's not a hologram." I mean, come on. Watch Star Trek and I know to be "That ain't it."

[Laughter]

Dan Frommelt: OK? I even know what that is. And actually I messaged my student Adam who has been working on this stuff and I'm like, "Dude, look. Augmented Reality in CNN. It's pretty sweet."

Will.i.am became the first hologram, although he'll correct that term eventually. But the masses don't really understand that. So this is my tweets that moment. As soon as it came out, I was like, "I know what it is. Hey, this is what's going on."

They also did an Augmented Reality of the Capitol Building, kind of showing the number of seats available on what had gone, which direction--it was a really cool display. It was very engaging, very inviting. They could have just said the numbers and had the same impact, but it meant more having that visual display in a 3D environment that was very new, draw-me-in. It was kind of fun.

So, the very next day, they had so many people calling CNN. "We need to find out more about this thing. How do you actually do hologram technology?" because I think they were thinking back to the Hollow Deck or something. I'm like, "No, that's not what's going on."

If you noticed and you watched that day--this is actually a whole presentation. You click the thing it'll open up and show the YouTube of that. We're not going to go through that now. It's mostly a promotional sale for this guy here and his company. I don't really care about that.

But what it was happening--if you remember that clip of Will.i.am when it happened, there was a little blue circle in the corner of the room, and the camera had to keep that blue circle in view at all times. And as the camera was moving around, Will.i.am, I believe, was in Chicago, if I remember correctly. I don't recall all the details completely. But he was in a tent and he was in the dead center of that tent. And he had to stand on a similar circle.

And all these cameras are around him, kind of like how they did all the Matrix shots for the movies. And then as the camera moved in was it Atlanta or New York, one of them, as they're moving around, it would supplement his current information inside that green screen tent into the livestream. So in the studio, they didn't see anything. They just saw a blue thing.

If you looked at what was being televised, supplemental information from somebody else in another location, streamed real-time. And for Will.i.am, when he was there, he had a little heads-up display in the corner so he could kind of look ahead and actually engage, because it's hard to have a conversation with a wall. It's a step in the right direction, but it was really kind of neat.

So what we really need is some kind of marker. Something like this. It's nothing too fantastic. And this is the basic marker. We're really at the baby steps here. The foothills of the food chain.

It has to be asymmetric. So if I just throw it down on the table, I should be able to determine automatically which side is up, because that's one of the first things. It's going to grab video input display and these are determined first. Is the marker present? And second of all, which way is up? If I'm going to supplement additional information, I need to know exactly what it is, its shape, its size. So from closer it gets bigger. If I'm further away, it gets smaller. If I'm spinning it, it rotates correctly. So that's the marker part of this piece.

And when it actually recognizes if you turn on the trace mark, it will actually highlight, "OK, I have identified the marker. I know where it's at." Then it can supplemental this additional information on top of it. It's kind of a cool piece.

So the simpler the pattern, the more you can have a margin of error. The more complex the pattern, the margin of error gets thinner, you know? You have to get more of the pattern to actually make it clear.

So you want something as very basic as possible. That's why most markers today that are in use are very big and blocky. They're not too exciting, but hey, you can make them look exciting. It's just your margin of error turns up.

So now that we have a marker concept in piece, the video can be modified based upon user input, meaning that the user can control the depth of the piece to the camera, or they can manipulate it, or if I add multiple markers, I can have different things happen. There's all kinds of things you can do with this utility. It's really kind of fun.

And it uses VR stuff, which we've been moving in the movies and schools and CAD for years. We have experience in VR. We just see to get out there and play a bit more. There's even the virtual reality modeling language. If you really want to go old school, it's a very painful way to learn.

If you do want to learn, go to that presentation page. I've actually done full research study on VRML. I wouldn't recommend doing it. [Laughter] I have easier recommendations, but you can do that.

This is the tutorial for that Vermel thing that I did build. That was one of the first things when I was hired as a webmaster back in '97. One on my position description, it said, "You must learn," because they thought that was going to be the next big technology. I'm like, "Are you kidding me?" Because there were no tutorials. And I had to re-W3C documentation and invent tutorials. You want to talk about a tough way to learn, you know, "Here's a mountain, scale it. Oh, by the way, here's a spoon."

[Laughter]

Dan Frommelt: "Thanks! OK, we'll figure it out as we go."

The thing that I do recommend is Blender, and this is something that my student Adam came across. It is a pretty simple-to-use modeling program. It's free! And it meets my budget requirements. So, that's what's going on there.

And when augmentation first came out the HITLab, Human Interface Technology, at University of Washington, they created an Augmented Reality Toolkit, and it was all written in C. It was fine, it works great, although it ran on one specific application.

If I had rendered it on that machine, then it would run on that machine, but it wasn't really portable. I couldn't move it to another machine as easily. It had all the little different pieces to it.

So it worked, but it wasn't really what we wanted. I wanted something where, OK, just go to a webpage and here you go. Tada! It works. So somebody took this piece and reworked it into Flash Augmented Reality Toolkit. That's a lot easier. Flash works pretty much everywhere. We even have Flash on mobile devices, so we're getting closer.

So now this is what I would recommend. Just pop up onto that site and you can play with the Flash Augmented Reality Toolkit. And Flash has benefits because if you've ever played with it, Flash can do microphone input, it can do video input, and it obviously does video and speaker output, so we've met our qualifications, input and output.

And what you do is create in Blender the different object that you want. You can then use Paper Vision 3D. It's a library that comes along with the Flash AR Toolkit. And it has a lot of additional information. So from there you're actually just doing basic JavaScript. It's in Flash's ActionScript, and you're manipulating different pieces and compiling this together to make a 3D implementation using Flash. So now I can do supplemental information with Flash. I have some examples we'll have in just a minute.

Let me walk through the usage. And I'll tell you now: this is not going to go into details of how to write every single line of code. This is just general concepts because I think this is going to be a relatively new concept for most of you. If you take this presentation and hit 'print', most of the details are in the notes when you print. So if you want that, just go here, print, you'll get a whole bunch of more information, and we'll get Adam's email address because he's got 99% of all the information there. Anyway.

So, there are some sample files when the Flare Toolkit is downloaded, so I recommend get back at your sample. It's usually a simple cube and it will have a marker similar to this, and you hold it above and a box appears on it. Ooh, whoopee! But, you know, it's kind of cool the first time you do it. You're like, "Oh, wow, a box!"

Then you start messing with stuff and you start playing with ideas and things really get a lot of fun really quickly. So you use that sample marker and the supporting library files and you can get that marker, turn it on, run this simple test, and the webcam window will be launched, you can verify that it is working.

When you export the Flash package, so all the ActionScript files are compiled with all the other stuff that makes it work, it will create a swift with the supporting libraries to the ActionScript and then it opens up in a browser window, and you're good to go.

Except there's a little hiccup there. It usually--this player toolkit works under USB videocam. So you have to open up your Flash and change the settings on Flash to not render as whatever the higher quality you have to render as lower quality.

If you think about it, it has to analyze every single frame real-time and identify a tag and supplement Virtual Reality on top of it, and scale it according to how far, I mean, that's just an amazing amount of processing it has to do. I can understand why it needs to be lower quality, but I just wanted to explain that if you don't adjust that to a lower setting on your camera, it won't work initially, so...

That's the only trick that I have stumbled across this. And I sent it off to some friends, "Oh, this is cool!" and they're like, "I don't see anything." "Oh, yeah, the USB thing. You just switch the camera to a lower mode, we're good to go."

I'll go through basic steps, but again, not details. So each one of these steps is probably like 15 sub-steps, but we're going to gloss over those. You create the basic object in the Blender so you're creating whatever rendering object you want. You get the supporting ActionScript for Blender inside your Flash program. You make any adjustments to the scale, the size, location.

And then you move the generated files and import it into the master. It's actually a ton of steps in that piece right there. I'm really minimizing the detail there. You attach the base node to the master AS file, so everything is compiled all together in one big punch. And then you upload it.

You also have the ability to make custom patterns. Initially we just swapped out objects for the pattern that they gave us. But if you go back to the HITLab's Flare kit, they have a program in there written in C that you can actually create any image you want. Keep it black and white, keep it square, keep it simple, and you can hold it up and it will try to capture anything it thinks is a tag and it will help you create the tag. So you create the pattern file using this makepath executable inside that HITLab Toolkit. So I do recommend you play with that, too. It's kind of cool.

And then as it pops up, the red and green angles appear, and it will kind of tell you which side is up on that. And it will not pick up color, so right now it's just black and white is all it's looking course. So that's kind of a down part for the marker, but again, we're in the infancy of this technology, so it's kind of cool.

You click 'accept', it will make that pattern file that the AR can now use. So you don't even have to really do anything fancy. It's just really just the design piece, hold it up, capture, move that pattern file into your Flare kit, and now you have your pattern ready to go. So it's not really complicated that way. You just merge, you know, if you need to do a file name change or whatever for your pattern and replace it.

So, here is an example. For the first time I did this out at Penn State over the summer and we had a blast. It was talked about a lot. And let me pull it up. Make sure my camera comes up. OK. And so as we have supplemental information, I now can augment the reality.

And you could see it's not really on the paper, is it? But I have the ability to change it up. OK, so lighting, lighting, lighting, there we go. Too bad it didn't look a little brighter in here. That's all right.

So as you move around, it's great. As soon as I cover up part of it, though? Gone. It needs to capture all of that tag in order for it to work. Adam, how long did it take you to put this together?

Adam: The model?

Dan Frommelt: Yes. The model. This is kind of cool.

Adam: Two hours?

Dan Frommelt: And that was really the toughest model you have done so far, so it was a crash course. It wasn't a ton of time, so if somebody wanted something funky, you can play with it. And this is a little more complicated than most things. But just to kind of show you the example.

And if you go back... Oh. That's kind of creepy leaning out on yourself. Heeey.

[Laughter]

Dan Frommelt: If you go to the marker thing there, that'll grab a PDF of this sheet, and it has the URL at the bottom. So if you want to go home and play with it, go home and play with it. It's there. Just print it off.

I have a clipboard here so it stays straight. If I bend the paper a little bit, it alters the reflection of the pattern, so it's a little touchy yet. But again, we're at its infancy, so it's loads of fun.

Now to do this, it's a little bit different. You had to create a skin for this thing that wraps around. So if you think about what we just had, these are all the pieces to the puzzle that were used as if it was unwrapped into one big thing. And then this skin is then applied to the object real-time as it is rendering.

So the more complicated you make the object, the bigger the pattern, the slower the render, so you can kind of see how it gets out of hand a little quickly. So you do have to kind of be careful when you're playing with it. You don't want to get it too far out-of-control. There's a fine line between useful and silly. I don't know where that is at, but that's kind of a case-by-case basis.

Here's another one. Once we had this piece together and I was showing, my supervisor's all "Oh, this is really cool," the next thing I know the Foundation office calls. "We have a donor dinner coming up next week. I understand that you can do something cool for us. Can you build the new Engineering hall that we just had all the--" Essentially a thank-you dinner.

[Laughter]

Dan Frommelt: How much time did I give you? Like an afternoon?

Audience 1: Maybe.

Dan Frommelt: I'm mean.

[Laughter]

Dan Frommelt: Get the camera on, OK. So, let's see. Let's get this beast out of the way. Tada. And so then we had the logo so you can kind of see what we're using. This is the university logo. It just had to be asymmentric. It was nothing fancy.

But as soon as I hold it up, we've got the Engineering hall, and actually they had this printed along with the URL, and everybody could take home the Engineering hall that they helped donate to and build. And it's a little touchy there on lighting, but that's all right.

This one's a little more complex, so it takes a little bit longer to render. And you'll notice the triangles that are on different spots. That's because it's having a difficult time trying to figure out the texturing. So, the simpler you make it, the better, but again, they wanted a new Engineering hall, we did our best. Of course then it's like, "Well, can you make it do--?" "No."

[Laughter]

Dan Frommelt: "You're lucky we pulled anything off in a half day." I mean, we had most of the pieces of the puzzle already figured out, though. I mean, don't think that we started and finished the project in one day learning all the AR. It took a little bit of time. But you get the concept, and this is a neat application.

I never thought of using this. We were initially building the library. We thought it would be kind of cool that if we built little cards, one for every building, and you just threw it down on the table, you can kind of have a camera up there and you can rearrange campus?

[Laughter]

Dan Frommelt: And then the thought of the campus planner freaking out, I just scrapped the idea.

[Laughter]

Dan Frommelt: All right. I could've been fun, but, yeah. Anyway. OK, so we'll get that thing out of the way.

So, current AR. Did I skip one? Back? I guess not. Current AR applications. There are some stuff that are already out there using AR.

So, General Electric has this website that they have been pushing called 'Plug into the Smart Grid' trying to encourage people to use alternate forms of energy in trying to show how they're being environmentally conscious. So let me pull up a couple of these pieces. And you can go to the 'Plug into the Smart Grid' website and I saw this in a couple of places. I saw a TV ad with this thing.

A very brief window. This gives a brief overview, a little movie to explain, you know, "Here's how to turn on a web camera, here's how to change the setting to USB." But I can come in to here and launch the wind turbine and this is a normal thing unless I change the settings. I have to click 'allow' because asking Flash, 'Will you accept this?' I can go in once I've accepted the thing. I can set the properties, 'Always accept from this site'. That's why it never appeared for the other ones. So I just allow the camera access, and so now I've got this.

[Demo Sound Effects]

Dan Frommelt: There's a company that builds these specifically. Oh, well, you get the concept. But it's really kind of cool.

[Demo Sound Effects]

Dan Frommelt: Wow!

Demo Voice 1: In!

Dan Frommelt: Yeah.

[Laughter]

Dan Frommelt: They have another one for solar energy. Same basic concept, but that's the GE one. And a little bit later, I was flipping through, I think it was Nick magazine or something my kids get at the house, and it was kind of funny. And let me--I saw in the magazine this ad for Your Zone and it was the tag itself.

So let me... I think it does the same basic... Yes. Well. Let's move it over here.

And this is what came. I actually--you can see I just ripped it right out of the magazine. It's actually got a Wal-Mart ad in the back, too. But on this one, same basic thing, as soon as it recognizes.

[Demo Sound Effects]

Dan Frommelt: It's a lighting issue.

[Demo Music]

Dan Frommelt: This is the room designer... and... No. OK.

[Demo Sound effects]

Dan Frommelt: So, what happens is the room will keep walking through the different styles of curtains and bed sheets and everything else they currently have available for their products. And it's playing a little music in the background, a lot more noise. You know, kind of makes you feel like you're at home. The kids' little study area.

A really neat way of highlighting some of their new technologies that they have.

So we were thinking about how can we possibly take some of these technologies and actually use them at a university setting? There is a lot that we could do, and one of them--let's see if we can get this thing come back to life. One of them was, we were thinking, well, there's a natural VR application a lot of schools are familiar with, and that would be Second Life. And there is a way to use AR with Second Life.

There's also ways to use AR with mobile phones, mobile devices. So if you think about it, your cell phone has audio input and output, video input and output--meets the requirements, so should be able to do it. That's really all we're looking for.

So, here is probably the first one I want to talk about is the Second Life, and this is just a YouTube. Hopefully this works.

Demo Voice 2: What would happen if a Second Life avatar could come out of the screen and end up sitting on your desk? That's the question that I started asking when I visited Julian Stadon's SLARiPS installation at the John Curtin Gallery.

Julian very kindly allowed me to film the installation. He was a student in the Master of Arts and Electronic Arts at Curtin University. The exhibition runs from the 20th of June to the 1st of August, 2008. SLARiPS is the Second Life Augmented Reality in Physical Space project, and he describes it as the development of interactive Augmented Reality construct that appear 3D in physical space when viewed through a head-mounted display.

It uses Second Life as a starting point for an image projection in 3D of an avatar. It also uses the avatar template from Second Life. If you've done any work in Second Life, you would recognize the bigger as using the avatar template from Second Life. However, it just uses it as a starting point. I think at the moment you're not really grabbing an avatar out of Second Life but the what-ifs that had asked and--

Dan Frommelt: So, there is Second Life applications that are built as well. And actually I have a different demo a little bit later that somebody did on YouTube. It's really cool. They actually had a marker laying on the floor, they had a video camera kind of walk up, and it's like you're at this edge of a cliff looking into a world of Second Life. You can actually see the avatars moving around in real-time.

So if you think of a virtual classroom that we already have in Second Life, you could actually have somebody monitoring it from above.

[Laughter]

Dan Frommelt: Which is really kind of creepy.

[Laughter]

Dan Frommelt: Anyway. Let's take a quick look at this.

[Demo Sound Effects]

Dan Frommelt: Pretty much different product modifications you can do. Just using your cell phone looking at the ad in a magazine, you can customize it, pare it. When you're done, you can have it sent off to a customer sales representative. They can get back in touch with you about how soon you want it delivered.

I mean, that's kind of...

[Demo Music]

[Demo Sound Effects]

[Demo Song]

Dan Frommelt: So think of your campus planning group. You could put up an ad. There's the next band coming up or homecoming activity. You can have a video supplement on top of that. You can have audio. You can have pretty much anything you want. There's a ton of stuff.

And if you're interested more in Augmented Reality, just go into YouTube, type in 'Augmented Reality', you will find there are tons of stuff out there. It is amazing.

In fact, I just found some new stuff this morning. Somebody took Augmented Reality and they are supplementing angle information for a pool table. So as you're playing pool, it's kind of showing where your shots are going to be. I'm like, "Ohh!"

[Laughter]

Dan Frommelt: I might actually have a chance!

[Laughter]

Dan Frommelt: There's also here the AR Toolkit running on a phone. We'll kind of pass that one for a moment, but... There's some really fun stuff coming out, too. And Augmented Reality is a lot more than just computers.

And probably the one that I would like to show you first will be this one. The BMW Augmented Reality.

[Demo Sound Effects]

Dan Frommelt: This concept, I can't wait.

[Demo Sound Effects]

Dan Frommelt: You hake your car to the dealer, they have to do something, but the dude doesn't really know what he's doing. It doesn't matter.

All right, now it looks cool.

[Laughter]

Dan Frommelt: But now we have the instructions in the heads-up display.

Demo Voice 3: Release two screws.

[Demo Music]

Dan Frommelt: And if you think about situations, for example, astronauts. They're in space, they're in a limited situation. They have to know more at the drop of a hat. This would be a possible technology they could use to do and repair something.

If you think of medical staff, it may be in Antarctica or something, and they need to do something basic, they could do basic things using simple technologies. The capabilities are amazing! Although the glasses are kind of dorky. You get the concept. But I'll walk you through and go to the next step. So it's kind of neat.

Gizmodo, this game never made it to market. And I'm really mad because I want to play it bad. So, you get your webcam, you get these little cards, which are just AR cards, you lay it down, and you build your towers around it.

And then you get somebody else's towers and you come forward and you have catapults attacking each other.

[Laughter]

Dan Frommelt: I'm like, you think about fun card games that kids used to play with? These are even cooler.

[Laughter]

Dan Frommelt: So, I think the company went belly up before this ever came out, unless--

Audience 2: Well, there was a preview in Playstation 3.

Dan Frommelt: Yeah. There's also another one I'll show here in a little bit. But, yeah, you see the concept. It's just fun, fun stuff. Because there's a lot out there and we just have to think differently about how can we use some of this.

And one of them would be something simple, like, how do I supplement education? If I have a book, how do I bring that book more to life? So, here is a concept on an interactive--and this is one kind of showing the concepts of what you need equipment-wise: webcam, computer, your book with the appropriate tags.

You open the book and you can see that the information is there. So they can have some information in the book, but some of the information is supplemented, augmented in through the camera. So as I'm looking at it, I can study different things and they will behave differently based upon what I'm doing.

There's another demo. I don't have that one loaded up here. It's one on the study of volcanoes. And it's in the lab. You put on the headset and you can actually--you read about the volcano, you can adjust different sliders to show or reveal more or less of the volcano. You can show the intensity of the volcano erupting and they actually have smoke coming out. It's just amazing!

You think about how much more you could learn about a particular city or a particular event, or if you're trying to learn about geography. It means a whole lot more when I can play with it, I can manipulate it. It's more engaging.

So I think we are going to see more pieces like this come into play.

Again, lots of fun stuff out there, and there's a ton of those pieces in it. There's one on GIS maps. This one's kind of cool. Again, this product will be to market I think by Christmas. This is their future ad.

[Demo Sound Effects]

[Demo Music]

Demo Voice 4: Awesome move!

Dan Frommelt: Play all your friends. Are you ready? Start moving.

[Demo Sound Effects]

[Demo Music]

Dan Frommelt: There's a lot of fun stuff that's going to be coming down the pike, and you know the way our students are now, they're going to start expecting more and more. So we have to at least know and understand the basic concepts of the technology.

We don't have to build it all right now. We have to be aware that this is out there and it's going to become more and more prevalent as we're playing with this.

There's a section on GIS maps. If we have a little bit of time, we'll come back to that. That's really neat, too. There is so much... Just a second here, let me go back a half step. Did I miss part of it? Future. OK.

So now, here is the thing. What do we do now to make AR... I mean, what can we do? What can we supplement? What can we play with? What's out there now? What would students use? What would they find beneficial?

So, one thing we were thinking about, is it possible to grab Twitter on the fly and augment it? So, we did--I think we did this, what, last week? And so I have another hash tag. I don't know what the tweet is. The search is for whatever the last tweet was. For heweb09. If we can pop--I don't even know who it is. Maybe we don't want to know. Cool.

So if you keep refreshing--we don't have an auto-refresh to grab more recent information. We could do that in ActionScript to have it, see if there's more relevant information. But this is interesting. There's a lot we can do, but we're trying to figure out how could you take this one step further.

Now, we didn't have time to work out this next piece. I was kind of busy planning this particular event so we ran out of time. There is a program available for Facebook that will allow users as you upload photos that will help identify who the people are on the photos so there's facial recognition.

And there is a company who's been trying to play with that concept in iPhone to actually have it as an application. You launch it, you pull it up, and if there's somebody who you know, you can associate that name with their Twitter or their whatever status they want with their business card. So as you're walking around the room, it's like, "Oh, hey, Grayson, I see you're thinking about--" whatever, you know, a thought bubble appears.

[Laughter]

Dan Frommelt: So as you walk around the room, frink!

[Laughter]

Dan Frommelt: Maybe you'd tweet a little bit more carefully, I don't know.

[Laughter]

Dan Frommelt: We were playing with that technology a little bit, but we just did not have time to rip it apart. It does not have a really easy interface at this stage. But it is something I do want to continue to play with, because, really, right now the limitation is the tag. The tag has to be square, it has to be whatever. We can get rid of that limitation. We can start supplementing anything, we would be good.

So I was hoping, because everybody's got the fleeces, I tried to design the fleece in the square. It didn't go over so well.

[Laughter]

Dan Frommelt: But we were hoping that we could get by with just the shape of the Hyatt Web logo there, but it is looking for a square. So it did not work. We tried many different ways. That Flare Toolkit is limited in that regard.

But that's available. And again, the marker is there, too, so if you want to grab that later. Right now that's just currently searching for whatever the last tweet is on that particular group. It's kind of fun. Oops! OK.

So now, AR resources. There's a bunch of stuff out there. HITLabs has their resource available. There's a Flash Toolkit, a full Wiki available out there, another Flare Toolkit startup guide, basic information about it.

A Second Life on Georgia Tech. They did a ton of stuff there and even have an Augmented Reality island in Second Life, and they have camera objects. So if you want to set up something, you just grab the object, you set it up, you just configure it to match a website, and then you download their application. You put in the plug for whatever the code is and then all of the sudden you're seeing into Second Life in that stage.

So you're bringing it outside of that environment and now you can interact with it a little bit more. So much fun stuff is available.

And we'll kind of pause here a little bit and we'll take some questions, so...

Yes. So it scales according to the tag size. So when we did the Twitter thing initially, the bubble was really tiny. And then so we had to scale it a little bit bigger and we had to force carriage returns in certain spots to make it fit inside the bubble. But it wasn't really that complicated, that piece. So the larger you scale the tag, the larger the object is. So it's kind of neat.

There is a... Oh, let's see if we got it here. Let me walk back a couple of slides. BMW Pit strategy. I'm missing one that I really wanted to show you. Here it is!

Baseball cards will never be the same. Topps has these out right now. Of course, they're a little marketing thing. Yeah, you know, you can't sell this.

[Demo Music]

Demo Voice 5: San Francisco Giants pitcher, Tim Lincecum.

[Demo Sound Effects]

Dan Frommelt: Not only do they deal with the player, but I also put a game based upon what position they play.

[Demo Sound Effects]

[Laughter]

Dan Frommelt: All of the sudden, baseball cards are cool again, you know?

Demo Voice 5: Baltimore Orioles outfielder, Nick Markakis.

[Demo Sound Effects]

Dan Frommelt: All right. Good catch. All right.

[Laughter]

Dan Frommelt: Let's see... And I think...oops! Back, back. I'm going to show you this one, too. And this is one of the first companies that came out and tried to do an ad.

[Demo Music]

Dan Frommelt: This is their entire production film development sequence.

[Demo Music]

Dan Frommelt: And if you actually search you can actually find that ad still, then you can print it off, pull it up, and it still works. They put it on the back cover of a magazine, and that was one of the biggest draws for an ad company. It was brand new. They really went out on the edge. And that same company was the one who worked with General Electric and also with Wal-mart, developed their ads, so that was essentially a brand new field.

You know, we're talking about what can we do, we're thinking about other ways to use this. So if we can do Second Life and we can get this whole piece to work, why couldn't we give somebody a more real virtual tour? I mean, if you can't make it to us, we can come to you. Let me email you a PDF and just go to this webpage and hold it up and, tada, I can be there. I can walk you around campus and show you what I need to show you.

How more real can you possibly get? How much more personalization can you get?

It would be a one-to-one relationship, but those are concepts. There's lots of other things we could do. I see a lot of possibility in education, though. And if you augment some classroom material with some of the--you know, here's the concept of a molecule and how it forms and what happens when this--you know. A whole lot more real when it's right there in front of my face and I can show interaction and motion.

You do have the ability to have multiple AR tags. So if we had up one tag and I hold up another one, it can supplement and augment the next one. So the more tags that are together, I can alter things. Tons of possibilities out there.

As we get into the question period--how much time do we have left? About five, 10? Just a few more minutes? OK. How about if I have Adam come on up? He's the student who I saddled with this beast. So, Adam is the one who's worked mostly on this, so I appreciate Adam for doing his hard work. So, is lighting going to work here today?

Programmer: Oh, I don't know.

Dan Frommelt: Oh, come on, Milton. Don't fail us now. All right. We'll try it this way, then.

[Laughter]

[Applause]

Dan Frommelt: It needs to have clear lighting, because remember, it's trying to grab information on the fly and alter it. And this is a very--we want to have Milton. Come on, it's funny!

But the tag is very complicated. So if we get brighter lighting, we can show you that in more detail. So, questions? Yes.

Audience 3: [43:40 Unintelligible]

Dan Frommelt: It's just a webpage and it's just a Flash file, with supporting ActionScript files. There is a pattern file that goes with it. What else is in that bundle?

Programmer: It's pretty much just a Flash file with an ActionScript file with the Flash, the Swift calls. And then it has imported the Flare toolkit, line barriers and stuff. When you talk about multiple markers, what it does it just build an array of all the markers that exist or that you see in window and then it throws out the models based on that, and then the pattern files, the ActionScript files, which is the export of the 3D model. That file just has points and the phases and then ActionScript editing, it pumps it out.

Dan Frommelt: Yes?

Audience 4: I'd like to determine, how do you get this to work if it doesn't have a marker. It just uses GPS to know where you are, a magnetometer to know what direction you are pointing. It's designed in a 3D model. You look at it as a 3D model up here.

Dan Frommelt: Right.

Audience 4: It's really exciting.

Dan Frommelt: That iPhone development kit that's out there for the Flare kit, they've got some information there, too. So if you have an iPhone and you want to play with it, grab that. There's lots of fun stuff available. I mean, I really look at this as a new technology evolving, that's why I wanted to kind of present on it a little bit, and we're dabbling...but lots going on. Other questions? Yes.

Audience 5: In the engineering demo, did you go for the modeling Blender?

Dan Frommelt: Yeah.

Programmer: Yeah. All the models that we used, they were all used, built in Blender, and there's actually a plug-in for Blender that exports to the ActionScript file. So, you don't even have to create that yourself. You just say 'file export' and then you have the model 3D.

One additional problem that we ran into was, I could export files but then all of the sudden it was in bits and pieces. Like, half of every face was missing because of the squares, two triangles, and it was only exporting one triangle per face.

And there's a command in Blender that changes everything to triangles, so then it ended up exporting smoothly. But, yeah. So...

Dan Frommelt: Small learning curve. But all those notes are inside the presentation if you hit prints. You're good to go. Other questions. Go ahead.

Audience 6: What's your website url?

Dan Frommelt: Sure! Let me close this. Come on back. It doesn't want to let go right now. There we go. That's the URL. And let's see...who asked questions here?

Who asked questions? Here's a Milton shirt.

[Laughter]

Dan Frommelt: Who else?

[Laughter]

Dan Frommelt: We had one over here. Who asked a question? All right, there you go. Sorry about that.

[Laughter]

Thank you, everyone. If you have questions, give me a holler. We're here.