Preparing SQA Professionals: Metamorphic Relation Patterns, Exploration, and Testing for Big Data

Purpose – Metamorphic Computer science (CS) has been rapidly growing in popularity, partly due to the great growth of big data (BD) and other new technologies, and the services that support them. BD involves data sets that are too large to be captured, curated, analysed, or processed (for an acceptable time and cost) using traditional CS techniques. Although BD has changed our way of living and working, it may still frequently encounter software quality assurance (SQA) problems — when testing BD systems, it may be too difficult to check whether the system behaves correctly or produces the correct output. This situation is known as the Oracle Problem, and is a major challenge for current and future (student) SQA professionals. Metamorphic testing (MT) has been identified as an effective approach to alleviating the Oracle Problem. MT makes use of metamorphic relations (MRs) across multiple test case executions to help to identify program faults. Metamorphic exploration (ME) is a related approach that can enable a better understanding, and thus testing, of a system. Both MT and ME rely on MRs for their successful implementation. However, identification of MRs is often a manual task requiring creative thinking, and a good understanding of the system. Metamorphic relation patterns (MRPs) are abstractions, or templates, for multiple actual MRs. This paper reports on an experience using MRPs to guide the identification of MRs for the implementation of ME/MT as a step towards training the next generation of SQA professionals.
Design/methodology/approach – The ME/MT experience is examined through reflection and comparison with existing MRP studies reported in the literature. Our case study involved using MRPs to identify MRs for ME/MT of BD systems. In addition to MRP-derived MRs, other MRs were derived directly from the user perspective, or from previous studies in the literature. The future use of MRP as a pedagogical tool to help train CS students and SQA professionals is examined.
Findings –We found that MRPs are useful in guiding, teaching and training the identification of MRs for MT/ME. Testers and students are able to implement MT/ME easily since the identification of MRs is a vital step in conducting MT/ME.
Originality/value/implications – This is, we believe, the first report on using MRPs to support ME/MT as a step towards training and teaching SQA professionals. We highlight the importance of BD SQA, and how MT/ME can support this. We show the usefulness of ME to prepare for MT, and MRPs to prepare good MRs to support MT/ME. We also outline several directions for follow-up action, relating to both MRP-based research and training.