Bridging context gaps in low-resource language chatbots through multilevel attention and hybrid embedding approaches

Abstract

Conversational agents for low-resource languages (LRLs), such as Igbo, face major challenges, including limited annotated data, code-switching, and weak contextual coherence in multi-turn dialogue. This study proposes a multilevel-attention and hybrid-embedding framework that integrates FastText subword representations with multilingual BERT (mBERT) to improve semantic understanding and context retention. The architecture applies hierarchical attention at the word, utterance, and dialogue levels, enabling effective modeling of conversational dependencies and reducing context drift. The model was evaluated on a curated Igbo--English conversational dataset and benchmarked against long short-term memory (LSTM), Transformer, FastText, mBERT, and XLM-R baselines. For response generation, the proposed framework achieved a bilingual evaluation understudy (BLEU) score of 44.1%, a longest-common-subsequence recall-oriented understudy for gisting evaluation (ROUGE-L) score of 60.3%, and a context-retention accuracy (CRA) of 81.5%. For intent classification, it attained an F1-score of 87.3% and an area under the receiver operating characteristic curve (ROC-AUC) of 0.91; for context-dependency detection, it achieved an F1-score of 84.3%. The framework also reduced inference latency and was robust to code-switching and noisy conversational input. Human evaluation confirmed improvements in response clarity, cultural relevance, and multi-turn coherence. The findings show that hybrid embeddings combined with multilevel attention provide an effective and scalable approach to conversational AI for LRLs, with potential applicability to other African languages.