Enhancing Sentiment-driven Recommender Systems with LLM-Based Feature Engineering: A Case Study in Drug review Analysis

SAMUEL MATIA KANGONI 1, OBED TSHIMANGA TSHIPATA 2, PIERRE SEDI NZAKUNA 3(Member, IEEE), VINCENZO PACIELLO 4 (Member, IEEE), JEAN GILBERT MBULA MBOMA 5, JEAN ROBERT MAKULO 6, KYANDOGHERE KYAMAKYA 7
(Member, IEEE)
1Dpt. of Electrical and Computer Engineering, University of Kinshasa, Kinshasa, DR Congo (e-mail: ssamat0020@gmail.com)
2Dpt. of Electrical and Computer Engineering, University of Kinshasa, Kinshasa, DR Congo (e-mail: obedtshipata33@gmail.com)
3Dpt. of Industrial Engineering, University of Salerno, Fisciano, Italy (e-mail: psedinzakuna@unisa.it)
4Dpt. of Industrial Engineering, University of Salerno, Fisciano, Italy (e-mail: vpaciello@unisa.it)
5Dpt. of Electrical and Computer Engineering, University of Kinshasa, Kinshasa, DR Congo (e-mail: mbula.gilberto@gmail.com)
6Unit of Nephrology, Kinshasa University Hospital, Kinshasa, DR Congo (e-mail: makulo.rissasi@unikin.ac.cd)
7University of Klagenfurt, Austria (e-mail:kyandoghere.kyamakya@aau.at)
Corresponding authors: Samuel Matia Kangoni (e-mail: ssamat0020@gmail.com), Kyandoghere Kyamakya
(e-mail:kyandoghere.kyamakya@aau.at)

ABSTRACT

Sentiment analysis is vital for evaluating user feedback in drug reviews because understanding patient experiences leads to more personalized treatment recommendations by providing insights into the real-world effectiveness and tolerability of medications, which are often overlooked in clinical trials. This study evaluates the effectiveness of word-level and sentence-level embeddings for feature extraction in sentiment analysis. These embeddings are used in sequential models (Bi-LSTM, CNN) and non-sequential models (Random Forest, DNN, ExtraTreesClassifier). The Random Forest model with LLM2Vec achieves the best performance, with 0.93 accuracy, F1-scores of 0.95 (positive) and 0.88 (negative), and precision scores of 0.93 (positive) and 0.94 (negative). This approach detects subtle negative feedback often missed by standard models. To capture social consensus in patient feedback, we introduce Adaptive ConfidenceWeighted Scoring. This method leverages social validation as an implicit confidence signal, enabling sentiment scores to reflect both individual experiences and community agreement. It enhances trust and interpretability while standardizing sentiment polarity on a [−5, +5] scale. Clinically validated drug-related information, scraped from trusted sources, is encoded using the Llama-3.2-3B-Instruct model to extract
context-aware representations. Features derived from this external medical knowledge improve semantic
understanding, ensure grounding and safety, and enhance robustness even when reviews are sparse or noisy.
The final vectors are employed within a cosine similarity framework to recommend relevant drugs, aligning
recommendations with user satisfaction. This work demonstrates how LLM-based feature engineering can
advance clinically valid, patient-aligned healthcare recommender systems informed by real-world feedback.

This article has been accepted for publication in IEEE Access. This is the author’s version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3590326