Application of the SBERT Model for Optimizing Answers to Frequently Asked Questions in Campus Academic Services

Abstract

This study designs a Telegram academic chatbot based on dense retrieval using SBERT (paraphrase-multilingual-MiniLM-L12-v2) combined with a category-menu navigation for quick exploration. The system is orchestrated to be classroom-ready and device-friendly (runs on Google Colab without a webhook) and supports selective answering through a similarity threshold (τ) to control fallback and wrong-confidence. A curated question–answer corpus is indexed as vectors; at inference, the query is embedded and matched via cosine similarity. Evaluation on 140 questions (7 categories) uses Top-1 Accuracy, fallback rate, wrong-confident rate, and latency. Results show τ=0.15–0.30 yields Top-1 72.14% with 0% fallback but 27.86% wrong-confident; τ=0.50 reduces wrong-confident to 20.71% at the cost of Top-1 67.14% and 12.14% fallback. By category, the highest accuracy is Academic (90%) and the lowest is Organization (55%); median latency is 26.7 ms. Main contributions: (i) integrating Telegram menus + SBERT in a single interaction flow, (ii) explicit τ calibration for selective answering, and (iii) a lab-ready blueprint that is easy to replicate. Findings confirm a clear risk–coverage trade-off and recommend intent-adaptive τ plus reranking/calibration as future work.