Abstract: Considering the poor stability of the video object tracking methods in outdoor scenes when the background circumstance variation in object images occurs, a new method is presented for object tracking based on infrared and visible dual-channel video. It extracts the Hue, Saturation, Value (HSV) color feature in visible image and the contour feature in infrared image, and combines the Mean Shift (MS) algorithm and the level set evolution algorithm to realize object location, also, the object scalar and model update mechanism is presented. To address the multiple object occlusion problems, a method is presented to locate multiple objects by determining the object merger and separation. Experimental results on infrared and visible dual-channel video demonstrate that the proposed method can successfully cope with the cases in complex environment such as illumination changes, shadow, occlusion, etc..