Metamorphic testing: A review of challenges and opportunities

Tsong Yueh Chen; Fei Ching Kuo; Huai Liu; Pak Lok Poon; Dave Towey; T. H. Tse; Zhi Quan Zhou

doi:10.1145/3143561

Metamorphic testing: A review of challenges and opportunities

Tsong Yueh Chen, Fei Ching Kuo, Huai Liu, Pak Lok Poon, Dave Towey, T. H. Tse, Zhi Quan Zhou

School of Computer Science

Research output: Journal Publication › Article › peer-review

346 Citations (Scopus)

21 Downloads (Pure)

Abstract

Metamorphic testing is an approach to both test case generation and test result verification. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and their expected outputs. Since its first publication, we have witnessed a rapidly increasing body of work examining metamorphic testing from various perspectives, including metamorphic relation identification, test case generation, integration with other software engineering techniques, and the validation and evaluation of software systems. In this article, we review the current research of metamorphic testing and discuss the challenges yet to be addressed. We also present visions for further improvement of metamorphic testing and highlight opportunities for new research.

Original language	English
Article number	4
Journal	ACM Computing Surveys
Volume	51
Issue number	1
DOIs	https://doi.org/10.1145/3143561
Publication status	Published - Jan 2018

Keywords

Metamorphic relation
Metamorphic testing
Oracle problem
Test case generation

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1145/3143561

51607Accepted author manuscript, 739 KBLicence: Unspecified

Cite this

@article{e7fbe2f2212749c5b3b59da6febde3a5,

title = "Metamorphic testing: A review of challenges and opportunities",

abstract = "Metamorphic testing is an approach to both test case generation and test result verification. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and their expected outputs. Since its first publication, we have witnessed a rapidly increasing body of work examining metamorphic testing from various perspectives, including metamorphic relation identification, test case generation, integration with other software engineering techniques, and the validation and evaluation of software systems. In this article, we review the current research of metamorphic testing and discuss the challenges yet to be addressed. We also present visions for further improvement of metamorphic testing and highlight opportunities for new research.",

keywords = "Metamorphic relation, Metamorphic testing, Oracle problem, Test case generation",

author = "Chen, {Tsong Yueh} and Kuo, {Fei Ching} and Huai Liu and Poon, {Pak Lok} and Dave Towey and Tse, {T. H.} and Zhou, {Zhi Quan}",

note = "Publisher Copyright: {\textcopyright} 2018 ACM.",

year = "2018",

month = jan,

doi = "10.1145/3143561",

language = "English",

volume = "51",

journal = "ACM Computing Surveys",

issn = "0360-0300",

publisher = "Association for Computing Machinery (ACM)",

number = "1",

}

TY - JOUR

T1 - Metamorphic testing

T2 - A review of challenges and opportunities

AU - Chen, Tsong Yueh

AU - Kuo, Fei Ching

AU - Liu, Huai

AU - Poon, Pak Lok

AU - Towey, Dave

AU - Tse, T. H.

AU - Zhou, Zhi Quan

PY - 2018/1

Y1 - 2018/1

N2 - Metamorphic testing is an approach to both test case generation and test result verification. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and their expected outputs. Since its first publication, we have witnessed a rapidly increasing body of work examining metamorphic testing from various perspectives, including metamorphic relation identification, test case generation, integration with other software engineering techniques, and the validation and evaluation of software systems. In this article, we review the current research of metamorphic testing and discuss the challenges yet to be addressed. We also present visions for further improvement of metamorphic testing and highlight opportunities for new research.

AB - Metamorphic testing is an approach to both test case generation and test result verification. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and their expected outputs. Since its first publication, we have witnessed a rapidly increasing body of work examining metamorphic testing from various perspectives, including metamorphic relation identification, test case generation, integration with other software engineering techniques, and the validation and evaluation of software systems. In this article, we review the current research of metamorphic testing and discuss the challenges yet to be addressed. We also present visions for further improvement of metamorphic testing and highlight opportunities for new research.

KW - Metamorphic relation

KW - Metamorphic testing

KW - Oracle problem

KW - Test case generation

UR - http://www.scopus.com/inward/record.url?scp=85040704777&partnerID=8YFLogxK

U2 - 10.1145/3143561

DO - 10.1145/3143561

M3 - Article

AN - SCOPUS:85040704777

SN - 0360-0300

VL - 51

JO - ACM Computing Surveys

JF - ACM Computing Surveys

IS - 1

M1 - 4

ER -

Metamorphic testing: A review of challenges and opportunities

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this