Abstract: A Q-learning based Joint Radio Resource Management (JRRM) algorithm is proposed for the autonomic resource optimization in a B3G system with heterogeneous Radio Access Technologies (RAT). Through the trial-and-error interactions with the radio environment, the JRRM controller learns to allocate the proper RAT and the service bandwidth for each session. A backpropagation neural network is adopted to generalize the large input state space to reduce memory requirement. Simulation results show that the proposed algorithm not only realizes the autonomy of JRRM through the online learning process, but also achieves well trade-off between the spectrum utility and the blocking probability.