In order to improve the ability of anti-chosen plaintext attack and decryption quality under unknown attack in current optical encryption technology, an optical image encryption algorithm based on chaotic Gyrator transform and differential mixed mask is proposed. The input plaintext is converted into its corresponding Quick Response (QR) code. The chaotic phase mask is generated according to the Logistic map. At the same time, the radial Hilbert and the zone plate phase function are combined to fuse with the chaotic phase mask for constructing the mixed phase mask. Then, a random sequence of Logistic chaotic maps is used to calculate the rotation angle of the Gyrator transformation, and the QR code is modulated to form Gyrator spectrum by combining the mixed phase mask. The Gyrator spectrum is divided into two components by introducing the equivalent decomposition technique, and two differential spiral phase masks are obtained by setting up different orders. Then, the Singular Value Decomposition (SVD) is introduced to process one of the Gyrator spectral components so that its corresponding orthogonal matrix is encoded by combining two differential phase masks. Finally, by combining the encoded orthogonal matrix and diagonal matrix, the  encrypted cipher is outputted based on thereversible SVD technology. The ability of resisting plaintext attack and clipping attack, as well as the sensitivity level of the encryption results to key change is analyzed theoretically. Experimental results show that the algorithm has good security performance.