Background
In December 2006 the Nintendo Wii was released. This included the Wiimote controller which provided an interesting new motion tracking system. The Wiimote connects to the Wii over Bluetooth and includes two kinds of sensors. A 3-axis accelerometer and an IR camera. For the purposes of this post the camera is the more interesting part.The infrared camera has a resolution of 1024x768 with built-in hardware blob tracking of up to 4 points at 100Hz. This is used in combination with the Wii sensor bar which is actually just an array of 6 IR LEDs, 3 at each end, to provide accurate motion tracking of the Wiimote when it is pointed at the sensor bar.
In December 2007 a develope called Johnny Chung Lee posted a video showing how the Wiimote could actually be used in reverse to accurately track a users head. This involved mounting IR LEDs on some glasses to take the place of the sensor bar and placing the Wiimote in front of the screen pointing at the user.
This demo was built on top of the Wiimotelib C# library for using the Wiimote on windows. A forum which discusses the head tracking demo is available here.
In a recent episode of Gadget Geeks on Sky One they used this technology to build a virtual window for a hotel room. This inspired me to investigate the technology myself.
My Hardware and Results
To build my own IR glasses I bought the following components:- 400mAh Lithium polymer battery
- Lithium polymer mini USB charger
- 950nm IR LEDs
- 15ohm resistor
- SPDT Switch
The complete glasses:
Close up on the side with the switch and the battery.
Full view of the side with the battery.
I then setup a Wiimote on top of my TV with some masking tape holding it in place. I used some more masking tape to hold an iPhone wide angle legs in front of the camera to give it a wider field of view - the camera normally only has a ~45 degree field of view.
A view of the TV including the attached Wiimote:
A close up of the Wiimote:
The results can be seen in this video.
Conclusions
I was quite pleased with how good this looked. However, the are a few key limitations.
- The lack of stereoscopic depth cues means that in person the 3D effect is much less effective than the video would suggest. If I combined this head tracking with a 3D TV I expect the combined effect would be really impressive.
- The field of view of the Wiimote camera is a really limiting factor as it is easy to move out of its view at which point obviously the head tracking stops working.
- The previous limitation is made much worse by the fact that the camera only works over fairly short distances. I found that once I moved much more than a meter away from the camera it started to struggle to keep track of me. This forces you to stay much closer where the narrow field of view is more annoying.
Other Consoles
The Wii has sold the highest number of units out of the current generation of games consoles (vs the Xbox 360 and PS3). However, as of Autumn 2010, both of the rival consoles have released their own motion tracking systems. Interestingly, both of these systems work in a similar way to the camera based head tracking covered in this post. In fact, the original creator of the head tracking demo, Johnny Chung Lee, ended up working on the Xbox Kinect development team.Xbox Kinect
The Xbox Kinect system uses an infrared projector to project a pattern of infrared light. This is then captured by a 640x480 pixel 30fps infrared camera. This input is augmented with a 640x480 pixel 30fps rgb camera. The advantage of this approach is that the IR light source and detector can be contained in the same package and the player doesn't need to wear/hold anything to be tracked.
It is interesting to note that both the resolution and frame rate of the IR camera is lower than the one used in the Wiimote.
Playstation Move
The Playstation Move uses an approach which is much closer to the one described in this post. The Playstation Eye camera provides a 640x480 60fps rgb picture which works even in very low light. This enables the use of the Playstation Move controllers which have a coloured sphere on the end. The system automatically picks the best color to stand out against the current background and then uses simple image processing to track the coloured sphere within the camera image. The known size of the sphere allows for the accurate measurement of depth.
Once again the actual resolution of the camera being used is lower than the one used in the Wiimote.