Abstract: A new single-channel double-color image encryption algorithm is proposed by combining Discrete Fractional Random Transform (DFrRT) with linear congruence theory. The two input RGB images are converted into their indexed image formats, and one of the 2D indexed image is encoded in amplitude part and the other is encoded into the spatial domain phase mask. The two color map matrixes are embedded in the fractional domain phase mask which is generated by Linear Congruential Generator (LCG). The amplitude-phase encoding is introduced into the optical encryption system to implement the double-color image encryption algorithm without increasing other optical elements. The fractional order of DFrRT and the four parameters of the linear function are the keys to enhance the security of the proposed algorithm, and the Mean Square Error (MSE) between the decrypted images and the input ones for all keys are calculated. The performance of the proposed scheme is analyzed against ciphertext-only attack, noise addition and occlusion of the encrypted image, respectively. Numerical simulation results demonstrate the feasibility and effectiveness of the proposed method.