Abstract: A feedback-based Adaptive Reference Picture Selection (ARPS) is proposed for improving error resilience in real-time video transmission. The novel predictive structure which the reference picture selection can be optimized with rate-distortion method is designed by taking advantage of classic reference picture selection. The bit-rate of coding with difference reference pictures is analyzed, and its impact on source distortion is discussed. By first introducing the concept of average channel cost, the propagation of channel error and its impact on channel distortion is studied. Within joint source-channel rate-distortion optimization framework, the video picture that leads to the minimum rate-distortion cost is selected as the current reference picture, which can efficiently enhance the video transmission quality and decrease the number of reconstruction frame buffers of encoder/decoder. Simulation results show that the proposed ARPS algorithm outperforms the classical ACK and NACK RPS method in different bit-rates and lost-rates in PSNR.