Interactive virtual martial arts video game system, based on realtime markerless motion analysis. FreeFighter enabled the user, wearing no markers or equipment, to play a martial arts game by making fully real martial arts movements in front of a large screen, and engage a virtual opponent in a realistic fight.

The system components were a dual-camera stereo vision rig, an video processing server, and a video game server with a large screen output.

The motion classifier algorithm was implemented using stereo contour matching, and a stochastic state machine approach with a training database of known motion datasets carefully selected from thousands of capture sessions with a variety of test users.

In addition, FreeFighter was prototyped using a wired Polhemus magnetic sensor system and Digital Signal Processing algorithms

Demo videos:

Jab Hook Uppercut StraightKick SideKick RoundKick Demo1 Demo2 Demo3