Sentiment Analysis using DistilBERT

Song Yi Ng; Kian Ming Lim; Chin Poo Lee; Jit Yan Lim

doi:10.1109/ICSPC59664.2023.10420272

Sentiment Analysis using DistilBERT

Song Yi Ng, Kian Ming Lim, Chin Poo Lee, Jit Yan Lim

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

Abstract

Transformers is an architecture that performs well in NLP task. To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model. Sentiment analysis is a process that extracts subjective information from textual data and categorizes them into different classes. The classification classes may include polarity (positive, neutral, negative) or emotions (happy, sad, angry). In addition, multiple techniques such as fine tuning, regularization and hyperparameter tuning are applied to improve the performance of the model. The proposed solution acquired an accuracy score of 85.41% on Internet Movie Database (IMDB) dataset and 86.59% on Customer Reviews (CR) dataset.

Original language	English
Title of host publication	2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	84-89
Number of pages	6
ISBN (Electronic)	9798350340860
DOIs	https://doi.org/10.1109/ICSPC59664.2023.10420272
Publication status	Published - 2023
Externally published	Yes
Event	11th IEEE Conference on Systems, Process and Control, ICSPC 2023 - Malacca, Malaysia Duration: 16 Dec 2023 → …

Publication series

Name	2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings

Conference

Conference	11th IEEE Conference on Systems, Process and Control, ICSPC 2023
Country/Territory	Malaysia
City	Malacca
Period	16/12/23 → …

Keywords

Deep Learning
DistilBERT
Sentiment Analysis
Transformers

ASJC Scopus subject areas

Artificial Intelligence
Computer Science Applications
Information Systems
Information Systems and Management
Safety, Risk, Reliability and Quality
Control and Optimization
Modelling and Simulation
Education

Access to Document

10.1109/ICSPC59664.2023.10420272

Cite this

Ng, S. Y., Lim, K. M., Lee, C. P., & Lim, J. Y. (2023). Sentiment Analysis using DistilBERT. In 2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings (pp. 84-89). (2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSPC59664.2023.10420272

@inproceedings{44ca289f4642400f831e0117eb984455,

title = "Sentiment Analysis using DistilBERT",

abstract = "Transformers is an architecture that performs well in NLP task. To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model. Sentiment analysis is a process that extracts subjective information from textual data and categorizes them into different classes. The classification classes may include polarity (positive, neutral, negative) or emotions (happy, sad, angry). In addition, multiple techniques such as fine tuning, regularization and hyperparameter tuning are applied to improve the performance of the model. The proposed solution acquired an accuracy score of 85.41% on Internet Movie Database (IMDB) dataset and 86.59% on Customer Reviews (CR) dataset.",

keywords = "Deep Learning, DistilBERT, Sentiment Analysis, Transformers",

author = "Ng, {Song Yi} and Lim, {Kian Ming} and Lee, {Chin Poo} and Lim, {Jit Yan}",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 11th IEEE Conference on Systems, Process and Control, ICSPC 2023 ; Conference date: 16-12-2023",

year = "2023",

doi = "10.1109/ICSPC59664.2023.10420272",

language = "English",

series = "2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "84--89",

booktitle = "2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings",

address = "United States",

}

Ng, SY, Lim, KM, Lee, CP & Lim, JY 2023, Sentiment Analysis using DistilBERT. in 2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings. 2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 84-89, 11th IEEE Conference on Systems, Process and Control, ICSPC 2023, Malacca, Malaysia, 16/12/23. https://doi.org/10.1109/ICSPC59664.2023.10420272

Sentiment Analysis using DistilBERT. / Ng, Song Yi; Lim, Kian Ming; Lee, Chin Poo et al.
2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2023. p. 84-89 (2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings).

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Sentiment Analysis using DistilBERT

AU - Ng, Song Yi

AU - Lim, Kian Ming

AU - Lee, Chin Poo

AU - Lim, Jit Yan

PY - 2023

Y1 - 2023

N2 - Transformers is an architecture that performs well in NLP task. To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model. Sentiment analysis is a process that extracts subjective information from textual data and categorizes them into different classes. The classification classes may include polarity (positive, neutral, negative) or emotions (happy, sad, angry). In addition, multiple techniques such as fine tuning, regularization and hyperparameter tuning are applied to improve the performance of the model. The proposed solution acquired an accuracy score of 85.41% on Internet Movie Database (IMDB) dataset and 86.59% on Customer Reviews (CR) dataset.

AB - Transformers is an architecture that performs well in NLP task. To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model. Sentiment analysis is a process that extracts subjective information from textual data and categorizes them into different classes. The classification classes may include polarity (positive, neutral, negative) or emotions (happy, sad, angry). In addition, multiple techniques such as fine tuning, regularization and hyperparameter tuning are applied to improve the performance of the model. The proposed solution acquired an accuracy score of 85.41% on Internet Movie Database (IMDB) dataset and 86.59% on Customer Reviews (CR) dataset.

KW - Deep Learning

KW - DistilBERT

KW - Sentiment Analysis

KW - Transformers

UR - http://www.scopus.com/inward/record.url?scp=85186676842&partnerID=8YFLogxK

U2 - 10.1109/ICSPC59664.2023.10420272

DO - 10.1109/ICSPC59664.2023.10420272

M3 - Conference contribution

AN - SCOPUS:85186676842

T3 - 2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings

SP - 84

EP - 89

BT - 2023 IEEE 11th Conference on Systems, Process and Control, ICSPC 2023 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 11th IEEE Conference on Systems, Process and Control, ICSPC 2023

Y2 - 16 December 2023

ER -

Sentiment Analysis using DistilBERT

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this