Exploring metamorphic testing for fake-news detection software: a case study

Lin Miao, Dave Towey, Yingrui Ma, Tsong Yueh Chen, Zhi Quan Zhou

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

Abstract

Concerns have been growing over fake news and its impact. Software that can automatically detect fake news is becoming more popular. However, the accuracy and reliability of such fake-news detection software remains questionable, partly due to a lack of testing and verification. Testing this kind of software may face the oracle problem, which refers to difficulty (or inability) of identifying the correctness of the software's output in a reasonable amount of time. Metamorphic testing (MT) has a record of effectively alleviating the oracle problem, and has been successfully applied to testing fake-news detection software. This paper reports on a study, extending previous work, exploring the use of MT for fake-news detection software. The study includes new metamorphic relations and additional experimental results and analysis. Some alternative MR-generation approaches are also explored. The study targets software where the output is a real/fake news decision, enhancing the applicability of MT to current fake-news detection software. The paper also explores the impact of the prediction accuracy of the fake-news detection software on the MT process. The study demonstrates the validity and applicability of MT to fake-news detection software. The prediction accuracy of the software has a greater impact on MT experiments with greater changes between the source and follow-up inputs, and less dependence on prediction stability. Some possible factors affecting the experimental results are discussed, and directions for future work are provided.

Original languageEnglish
Title of host publicationProceedings - 2023 IEEE 47th Annual Computers, Software, and Applications Conference, COMPSAC 2023
EditorsHossain Shahriar, Yuuichi Teranishi, Alfredo Cuzzocrea, Moushumi Sharmin, Dave Towey, AKM Jahangir Alam Majumder, Hiroki Kashiwazaki, Ji-Jiang Yang, Michiharu Takemoto, Nazmus Sakib, Ryohei Banno, Sheikh Iqbal Ahamed
PublisherIEEE Computer Society
Pages912-923
Number of pages12
ISBN (Electronic)9798350326970
DOIs
Publication statusPublished - 2023
Event47th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2023 - Hybrid, Torino, Italy
Duration: 26 Jun 202330 Jun 2023

Publication series

NameProceedings - International Computer Software and Applications Conference
Volume2023-June
ISSN (Print)0730-3157

Conference

Conference47th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2023
Country/TerritoryItaly
CityHybrid, Torino
Period26/06/2330/06/23

Keywords

  • fake news
  • fake news detection
  • fake news detection software
  • metamorphic relation
  • Metamorphic testing
  • oracle problem
  • software testing

ASJC Scopus subject areas

  • Software
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Exploring metamorphic testing for fake-news detection software: a case study'. Together they form a unique fingerprint.

Cite this