Abstract: Combination of acoustic models trained from different unit sets is studied in this paper. For Chinese continuous speech recognition, Prevailing unit sets include context-dependent initial-final unit set and context-dependent phone unit set. Through experiments it is discovered that some Chinese syllables have higher recognition rates under initial-final model while some have higher recognition rates under phone model. In this paper, a method is proposed to combine these two acoustic models. On one hand the two acoustic models can be fully utilized during the recognition process; on the other hand, some models that lead to low recognition rate will not be used. Experiments show that in comparison with initial-final model and phone model, syllable error rate is reduced by 9.60% and 6.10% respectively after using the provided method.