Abstract: Registration of the consecutive frames is quite essential in dynamic video super resolution. In this paper, a multi-scale least square affine-based block-match method is proposed. An index Dmv is defined to evaluate global and local matching performances of the images. Then a multi-scale scheme is designed to adjust the block size automatically according to the motions between frames, which guaranteed well performances in both planar and un-planar regions. Different from the traditional block-match method, the affine-based least square estimation algorithm is introduced for registration of each pair of blocks. Convergence of the estimating process for different size of blocks is resolved by unification of the update step, which results in improvements in both estimating precision and speed. Finally, the proposed algorithm is evaluated in both the registration performance and its affects to the performance of the super resolution algorithm. Experimental results show that the proposed algorithm not only can provide more accurate motion estimation, when be applied to the Maximum A Posterior (MAP) based super resolution method, it shows obvious enhancement in both reconstruction performances and efficiencies.