Context-Aware Attentive Deep Learning for Enhanced Sentiment Analysis in Multimodal Social Media Data

Akash Verma

Context-Aware Attentive Deep Learning for Enhanced Sentiment Analysis in Multimodal Social Media Data

Authors: Akash Verma

Published: 2025-04-28 19:25:57.262

Abstract

Sentiment analysis, the computational task of identifying and categorizing opinions expressed in text, has seen significant advancements with deep learning. However, its effectiveness is often hampered by the reliance on textual data alone, neglecting the rich information conveyed through other modalities like images and videos prevalent in social media. Moreover, existing approaches often lack the capacity to effectively capture the contextual nuances inherent in multimodal data. This paper introduces a novel Context-Aware Attentive Deep Learning (CAADL) framework for enhanced sentiment analysis in multimodal social media data. CAADL leverages deep learning models with attention mechanisms to extract salient features from both textual and visual modalities. Furthermore, it incorporates contextual information by employing a hierarchical attention network that models inter-modal and intra-modal relationships. The framework is trained and evaluated on a large-scale multimodal sentiment analysis dataset. Experimental results demonstrate that CAADL significantly outperforms state-of-the-art baselines in terms of accuracy, F1-score, and precision, highlighting the importance of context awareness and attention mechanisms in multimodal sentiment analysis. The proposed framework provides a robust and effective solution for understanding and interpreting sentiments expressed in the complex and dynamic landscape of social media.

Keywords

Sentiment Analysis, Multimodal Data, Deep Learning, Attention Mechanisms, Context Awareness, Social Media, Natural Language Processing, Feature Fusion, Emotion Recognition

References

Turney, P. D. (2002). Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (pp. 417-424).
Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing (Vol. 10, pp. 79-86).
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1746-1751).
Baltrušaitis, T., Ahuja, C., & Morency, L. P. (2018). Multimodal machine learning: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence, 41(12), 2724-2742.
Zadeh, A., Liang, P. P., Mazumder, S., Poria, S., Cambria, E., & Morency, L. P. (2018). Memory fusion network for multimodal sentiment analysis. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1414-1424).
Tsai, Y. H., Liang, P. P., Zadeh, A., Morency, L. P., & Salakhutdinov, R. (2019). Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 6355-6365).
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1480-1489).
Liu, B., Zhang, L., Luo, X., Wang, J., & Zhao, W. X. (2015). Context-aware sentiment lexicon expansion. In Proceedings of the 2015 conference on empirical methods in natural language processing (pp. 1874-1884).
Hazarika, D., Poria, S., Zadeh, A., Cambria, E., Majumder, S., & Morency, L. P. (2018). Conversational memory network: Context-aware emotion recognition in spoken dialogue. In Proceedings of the 2018 conference on empirical methods in natural language processing (pp. 2562-2572).
Cambria, E. (2017). Affective computing and sentiment analysis. IEEE Intelligent Systems, 31(2), 102-107.
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017). A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion, 37, 98-125.
Akhtar, M. S., Kumar, S., Ekbal, A., & Bhattacharyya, P. (2017). Aspect based sentiment analysis using deep memory networks. Expert Systems with Applications, 89, 155-165.
Yu, L., Jiang, J., Zheng, L., & Luo, B. (2017). Learning context-aware representations for sentiment classification. Knowledge-Based Systems, 128, 10-21.
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research, 15(1), 1929-1958.
Zadeh, A., Liang, P. P., Tiong, R. S. K., Poria, S., Morency, L. P. (2016). Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion framework. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (pp. 2136-2146).*

Download PDF

How to Cite

Akash Verma, (2025-04-28 19:25:57.262). Context-Aware Attentive Deep Learning for Enhanced Sentiment Analysis in Multimodal Social Media Data. JANOLI International Journal of Artificial Intelligence and its Applications, Volume EOCMPeqBj5R9ZDur0Rlk, Issue 2.

ISSN: 3048-6815

JANOLI International Journal of
Artificial Intelligence and its Applications ( JIJAIA )