Abstract: The Improved Non-Subsampled Contourlet Transform (INSCT) algorithm combines redundancy ascending transform and multi direction analysis. Neville-operator is used in the part of redundancy ascending and direction information is lost in the process of multi-scale decomposition, which is bad for follow-up analysis. To solve this problem, a new set of operators which effectively preserves direction information in frequency band decomposition part is designed in this paper, and a new image fusion algorithm using the operators is proposed. First, the histogram equalization method is used to enhance grey values of the target zone in infrared image. Second, the multi-scale decomposition is performed on visible image and the enhanced infrared image using the new set of operator rather than the Neville-operator. Finally, low-frequency sub-band is fused using a new method incorporating the activity level based on Weighted Average-Window Based Activity measurement (WA-WBA) rather than the simple weighted sum method. The use of neighborhood homogeneous measurement realizes the adaptive fusion of each sub-band coefficient, and the proposed method effectively makes up the disadvantages of pixel-based image fusion method. The experimental results show that the proposed method preserves details of the visible image and obtains clear target information in the infrared image.