Abstract: The digital channelizer processing on-board is adopted by the broadband flexible transponder, which is the next generation communication satellite payload. This payload can flexibly transpond signals within any frequency band and any bandwidth, which can not be well solving by the traditional payloads. Based on analyzing performance of the modulated filter banks in flexible transponder, an algorithm for designing prototype filter with Near Perfect Reconstruction (NPR) is proposed through converting the filter banks to the design of prototype filter. This method include two step: Firstly, the low order two-subchannel filter h(2)(n) and the real symmetric FIR filter g(n) are designed using the Parks-McClellan algorithm. Then the desired NPR prototype filter h(M)(n) is calculated through upsampling non-zero values to the coefficients of h(2)(n) with a factor of M and factor-2 downsampling according filter g(n). So the proposed algorithm can efficiently avoid the performance loss caused by image components with Interpolated Finite Impulse Response (IFIR) approach. The simulation results prove that the modulated filter banks designed by the proposed algorithm present better performance than conventional methods.