With all this sudden interest in 3D because of films like Avatar, and hearing things such as the newly coined term ‘augmented reality’ it’s time for me to clear up a few things.

The Invention of Augmented Reality.

The idea Star Trek’s Holodeck has been around for a long time.

Here I’m going to put my foot down, which I seldom do, but I think I’ll be doing a lot more of this come the New Year. Firstly I must admit, that I didn’t come up with the name ‘Augmented Reality’ but I did come up with the concept. The trouble I had way back then was selling the idea…

Before I get started I must say that Star Trek’s ‘Holodeck’ obviously pre dates my version of ‘Augmented Reality’ but then I wasn’t planning on attempting to suddenly give photons mass and then have them suspend themselves in mid-air, that really would be clever, if not actually my idea any longer…

Coming from the video games industry I had an idea that would take the world of Virtual Reality away from the stagnant location of sitting in an arcade machine booth and move it to a place where people could interact with each other as they do in the real world – only instead of looking through their ‘normal’ eyes the player would see a display drawn by a computer that represented a view of the world that their eyes would show them. The sounds they would hear are the normal sounds around them only with added computer generated sound effects.

The year was 1992 and having completed Back to the Future III on the C-64 and then Technocop for the NES (which

The idea is to have real objects in this room which would also be much larger.

was a dreadful game but it was a conversion to the NES. I did my best with it really – and Jeroen Tel did a great job of the music, I still know all the tunes now!)

I was talking to my boss at the time Fergus McGovern who was M.D. of Probe Software based in Croydon U.K. and I did my best to describe what I had in mind…

“Imagine you are sitting in a room at home. Everything you see around you; your bookcase, the desk lamp, your chair etc… everything you see with your eyes are actually displayed by a computer. A bit like wearing a pair of glasses only the light that comes to your eyes are not photons that have traveled through your lenses, but from a computer generated image that is set into your glasses. Now, imagine that your head, body and joints are all being tracked (the basics of motion tracking capture was available at the time, the best of which was in a game ‘The Prince of Persia’.”

I was suggesting, rather than simply filming a person running and digitise the footage, we actually tracked them. Each joint would have a ‘marker’ on it (in fact my way of doing this is still far better than the multi-camera setup that is currently used for motion capture at present so I won’t go into detail here). The idea of motion tracking is that the exact location/position could be fed into a computer of any person (or object) – and then placed into a virtual computer landscape.

The difference here is that the ‘virtual landscape’ would not just be any landscape, it would be an exact computer modelled  landscape of where you are right now (in this example I’m sitting in my sitting room with my chair, bookcase, table etc…)

The point here is this:

If everything I’m seeing is now drawn by a computer, I can see my bookcase, I can even go over to my bookcase and touch it – because in the real world – it really is there! However, what I’m seeing is the computer displaying my bookcase to my eyes exactly as I would see it in the ‘real world’. I can still feel my bookcase with my fingers using my normal sense of touch.

So what’s so good about that?

To be blunt back in the 1990’s I’d already lost most people by the time I’d got this far – really. No one could grasp what I was talking about, but there were a few that could and Fergus McGovern was one of them bless him.

Now we get to really open up the door of what’s possible… Now that we are ‘Augmenting’ (not my word for it) Reality we can really start to play.

Now let’s imagine that I’m not in my living room, I’m in a large multi-room complex that has been carefully mapped out to the fraction of a millimetre so that the computer model of the complex matches exactly my environment. Let me spell this out. I’m now standing in say for example a warehouse; there are several floors and there are many large rooms: Every detail, every surface area of this warehouse has been modelled into a computer.

The room in which I stand is displayed to me through my stereoscopic 3D glasses. The computer knows my exact position. It knows where I’m facing, where my eyes are looking and the exact location of all my joints (elbows, knees, feet, fingers etc…) Only this is where it’s a bit different: Rather than see a computer generated image of me, I see an alternate version of me (now that Avatar’s been out and people are more likely to grasp what I’m saying I could use the word Avatar – but I won’t – you know what I mean)

My head, legs, arms – all now look different (if I wanted them to, but then this is the video games industry so yes, of coarse I want them to).

Here’s where it gets really good.

I could instruct the computer either not to display me or make me look semi-transparent to others (invisibility). I could tell the computer to add to the display of what it’s showing of the world around me – not just the room I’m in – but to add all the other rooms around me to my display (x-ray vision). I could add stuff to the display that isn’t really there – but then I wouldn’t be able to tell – unless I tried to touch it… You see what’s happening? We can now give ourselves super powers, we can make objects appear that aren’t really there, we can do all sorts of things…

The idea for a video game was this:

I would need a large place – I had the EPCOT centre in Disney World as my first thought. The game would consist of 8 players per team and there would be two teams. Each player would be armed with a ‘plasma riffle’ with which they could happily blast away at opponents.

The aim of the game is to infiltrate the enemy base (a lot like a paint gun combat game in the real world), photograph secret documents, retrieve a lost/stolen item, eliminate enemy forces, blow up the enemy base etc… You get the idea.

The game would also be Internet based whereby players at home could dial-in and play minor roles such as droids with limited weaponry and movement – and the battle commences…

The Pitch:

Having a good track record as a video games programmer I approached people like Nintendo. I had a meeting between Fergus McGovern and myself and a couple of high up people from NOA Nintendo of America (who shall remain nameless).

My response back in 1992?

“Simon, you go ahead and develop it, make it happen all by yourself. If you make a success of the company we’ll just buy you.”

And that’s it ladies and gentlemen. Nintendo have absolutely no interest in being creative, they don’t want to invest any time or effort in to making something great. All they’re interested in is money. That’s it pure and simple, if your company is making money they’ll just buy you out. If you don’t sell they’ll squeeze you out. Either way they’ll get rid of you.

If I sound like a complete and utter cynic – that’s because I am. The video games industry even in those days had absolutely no vision, no balls to do something new. If I wanted to do a film license, say for example – Die Hard 57 – sure no problem. The money men could understand what you were talking about, even they could sit in a chair and watch a screen in front of them and stare at an already successful film license – It doesn’t take any stretch of the imagination for them at all. All they would have to do as money men, would be to invest a few pounds, get the appropriate ‘management’ in place to make sure I wasn’t working any less than 18 hours a day and they could sit back, kick their heels up in the air and wait for their day; so that they could screw 10’s if not 100’s of millions of pounds out of their investment. I don’t like the video games industry much any more (you can tell can’t you). Still I’ll never give up being a programmer and one of the wonderful things about being a ‘creative type’ is that we create, and I’ve got some things in the pipeline that are so exciting I can’t wait to get started. Anyway I digress…

I spoke to a few other select people about this from SONY and even Mike Hughes a TV game show producer here in the UK. Unfortunately although I had attentive ears it was extremely difficult for people to grasp the essence of what I was saying as there was nothing current to refer to, and so after a very pleasant meeting after which no one had any clue as to what I was talking about, I decided that maybe this idea was not yet ready to seek funding for so it got ‘shelved’.

I sincerely believe I could have made that project a reality back in ’92 and I believe I could do an even better job of it now. There are still a few hurdles to overcome and if today’s technology isn’t there to do it – then I’ll create it.

After all that’s what I do.


Image used is from the website futureupdate.wordpress.com – many thanks.


XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>