Video Filtering of Infant Social Games

Video-based behaviors, especially social interactions, are widely analyzed in screenings and diagnosis of developmental disorders, as well as studies on child development. Existing methods rely on human's frame-by-frame tagging and editing of video content, which is time-consuming and labor-intensive. Automatic retrieval of infant social games is a logical first step towards automatic analysis of social interactions for three reasons. First, these games typically arise between 8 and 12 months of age, when infants have not yet achieved full mobility, making it easier to monitor the game with a single relatively-stationary camera. Second, many infant social games such as peek-a-boo and so-big are characterized principally by gross motor movements. Third, these games are fairly structured interactions and they typically follow a regular turn-taking pattern. These analysis suggest that analysis of repetitive patterns of gross motion in video could be sufficient to identify the instances of social games in a video collection, and distinguish social games from other video content.

Currently, we define social games as re-occurring social interactions between a child and an adult, or among any number of children.

Retrieval system

The retrieval system consists of two modules: Module 1 – converting videos to discrete symbolic sequences; Module 2 – Mine quasi-periodic patterns from the sequence.
The diagram of Module 1 – Convert videos into discrete symbolic sequences is shown below.

Module 2: Results of mined patterns from videos of games. We show examples of a pattycake game and a peekaboo game. The mined sequential patterns are highlighted on top of each image. Each row is one occurrence of the pattern in the video. You will see pattern 2-4-6-9 appears three times in the pattycake video and pattern 2-1-5-1-7 appears twice in the peekaboo video.

Retrieval Performance
We test the performance on three videos recorded with adult-child in a laboratory setting with hand-hold camera. From left to right, the performance curves are for video E,K,M respectively.

Experiment1-YouTubeGame.wmv
Experiment2-HSI07222009.wmv
Experiment3-GDA07232009.wmv
youTubeVideos.zip

Last modified 23 October 2009 at 4:28 pm by Ping