26 January 2014 by Woody Chow on KLT | SIFT | OpenCV | computer vision | homography | tangible tiles | berkeley

Realtime Tracking of Tangible Tile using SIFT+KLT


Figure 1: Tile tracking by SIFT+KLT feature tracking.

Tangible Tiles is a liquid meditated touch surface created by Campbell et al. at Berkeley. Because of deplacement of the colored liquid, a finger press on the device would result in a color change in the nearby area. This tangible tile could become an input device if the partial color change on the device could be tracked efficiently.

Figure 2: Tangible Tile in action, image courtesy of Cesar Torres.

In my last semester at Berkeley, I have worked with Cesar Torres, one of the creators of the tile, creating a Computer Vision based system for turning the tile into an input device. I was responsible for tile tracking and Torres was responsible for features detection on the tile.


Below are videos of my tile tracking in action.

Note: Red square denotes a SIFT(corrector) frame, while blue square denotes a KLT frame in the video.

Implementation Details

If you are by any chance interested, please read the final writeup (link) for details.