Kinect Hackers – Dennis Ippel interview

Tell us a little about your work with Kinect.

I started experimenting with Kinect as soon as the open source libfreenect library was released. The Kinect Sensor hardware offered so many new possibilities that I spent a lot of evenings playing around with its data.

The first goal I set was to understand how I could use the depth information to interact with 3D objects. This resulted in an augmented reality style game where you have to slap bananas that are thrown at you.

The next thing I wanted to experiment with was face tracking and gesture recognition. I used the classic Tetris game as a starting point and integrated face tracking to move the camera in 3D space. This really enhanced the 3D experience. I was able to use gesture recognition to move the blocks and rotate them.

Some speaking engagements at conferences followed for which I had to create some demos that used Adobe Flash. In order to get Kinect working together with Flash I wrote a piece of software that acted as an intermediary between the Kinect Sensor and the Flash player.

Now that Microsoft have released an official SDK I am experimenting more with that.

What are your thoughts on the current crop of games available for Kinect? Are they using the technology innovatively, or are their more inventive uses of the technology we’re not seeing from developers?

Kinect Sports is pretty sweet, but it wears me out!  It’s better than the other gesture tracking sports games because it tracks your whole body.  I’m looking forward to some new games that were announced at E3 that will have Kinect interaction, such as Star Wars and SSX.

Do you see Kinect as a more useful and interesting tool outside of videogames? Why?

Absolutely. Just take a look at all the experiments out there. You can see that intrigued people from different backgrounds and industries have picked it up to investigate the potential. I’ve seen Kinect used for robotics, medical research, playing virtual instruments and live visuals for music shows amongst others. From an advertising point of view it offers many possibilities for interactive installations in shops and exhibitions.

What non-videogame related applications of Kinect particularly interest you?

Because I work for a digital media agency I am mostly interested in how I can use it for in-store and exhibition installations – engaging audiences in a new and powerful way. We have a lot of opportunities to use if for major events too.

What can you imagine Kinect and its technology being used for in five years time?

I’m sure that Kinect or similar technology will be integrated in laptops and television sets. This will allow users to interact with the device in a more natural way by using gestures. Think Minority Report.

Seeing that Kinect is also used for robotics experiments I could see the technology being used in toys as well.

If Microsoft hired you to make a game for Kinect, what kind of game would you be interested in making?

I haven’t got a particular concept in mind but I’d love to work on a game that uses facial, voice recognition and skeleton tracking together. It would probably be a niche game, experimental both in gameplay and the use of visuals.

Microsoft recently announced Halo 4. If you could add one Kinect feature to a Halo game, what would it be?

I would like to see military-style gesture recognition for communication in multiplayer mode. Ideally some of these would be configurable. It would be great if you could configure gestures that target specific team members.