This paper presents an approach of Multi Space Distribution Hidden Markov Model (MSD-HMM) for Vietnamese recognition. An MSD-HMM prototype with four independent streams is proposed for modeling the Vietnamese phonemes which embedded tonal information corresponding to its syllable. These phonemes are built by adding tonal symbol to each phoneme syllables based on the International Phonetic Alphabet (IPA). This approach improves 2.49% accuracy compared to the baseline system. A process of tonal feature extraction that is suitable for modeling by MSD-HMM is also described. The result shows that the performance of MSD-HMM and tonal phoneme is better than the baseline system,  and the tonal phoneme and tonal feature are important components for Vietnamese recognition.