Advanced Joint Model for Vietnamese Intent Detection and Slot Tagging

This paper aims to propose BiJoint-BERT-NLU, an advanced BERT-based joint model for Vietnamese intent detection and slot tagging, which extends the state-of-the-art JointBERT-CRF model. This model leverages the bi-directional relationships of these two tasks by: (i) adopting an intent-slot attention layer to explicitly incorporate the simple intent output (but with a temporary intent loss) into slot tagging (with a slot tagging loss) from the JointIDSF model, and (ii) introducing an advanced intent classification layer (with a final intent loss) that uses the slot tagging results to improve the accuracy of intent classification. The slot tagging outputs of all tokens, i.e. slot probability, will be summed up for each slot to build the final slot vector for the intent classifier. During the training phase, the coefficients of the three losses are optimized by grid search. The experiments have been done on the recently (and only) published PhoATIS dataset, the Vietnamese version of ATIS. The experimental results show that the proposed model using PhoBERT encoder on word-level on the syllable-level variant of the dataset gives a significant enhancement of Intent accuracy compared to state-of-the-art baseline models, i.e. JointBERT-CRF and JointIDSF. The Sentence accuracy has a considerable improvement on both syllable-level (using XLM-R encoder) and word-level variant.