Detecting and tracking unconstrained hands in videos is a basic technique for sign language recognition. In current hand detection methods, AdaBoost classifier based on Haar-like features is known to be fast and robust against scale change and rotation. However, its performance drops sharply when the background is complicated or the hand and other skin-color parts overlap. Insufficient training data also decreases the performance. This paper proposes a new training method for Haar-like features based AdaBoost classifier with insufficient data, and a hand detector integrating Haar-like features, skincolor and motion cue together. Also we present a novel hand tracking technique. Experimental results have shown that the proposed method obtains a promising detecting rate of 99.9%, and more than 97.1% of the tracked hands are extracted in proper size. In summary the proposed method is more robust than AdaBoost classifier against
complicated background, scale change and rotation.