Abstract: To reduce side effects of background information included in the outer parts of tracking rectangular boxes, a weighted block compressed sensing feature extraction method is proposed based on normalized gradient features. The compressed sensing measurement matrix is converted to a block diagonal matrix. Appropriate weights are assigned to different blocks according to the importance of the blocks. It aims to reduce the measurement matrix size, weaken background interference and simplify feature extraction. Then the extracted features are inputted into Bayesian classifier with adaptive priori probabilities, which is proposed to make full use of existing tracking results. To some extent the classifier with variable priori probabilities can predict the direction of the moving targets, and reduce the ambiguities of target candidates. Each frame classification function changes according to the results of the previous track to improve the classification accuracy. In the experiments compared with four state-of-the-art tracking algorithms on 8 commonly used tracking test sequences, the proposed target tracking algorithm has higher accuracy and stability in terms of tracking results and success rate.