Abstract: A new local saliency texture descriptor named Local Ternary Number (LTN) is defined in this paper, and a new object tracking method combining LTN with hue information is proposed. Derived from Local Similarity Number (LSN), the new local saliency texture descriptor makes full use of pixel intensity relationship in a 8-connected neighbourhood. Compared with LSN, the LTN could distinguish different texture structures with the same local saliency degree. LTN mask not only extracts the key object pixels on edges, lines and corners to enhance the distinguish ability of texture feature, but also preserves the unity of the object. Then a new Mean Shift (MS) tracking method is built by utilizing the new object model, which integrating LTN with hue information from the object pixels within this mask. A large number of experiments are conducted and the experiment results demonstrate that the proposed method improve the performance of the traditional MS tracker, and the proposed method tracks object continuously and accurately even in the scene with similar color and illumination changing.