File(s) not publicly available

Using metamorphic testing to evaluate DNN coverage criteria

conference contribution
posted on 08.04.2021, 06:05 by Jinyi Zhou, Kun Qiu, Zheng Zheng, Tsong Yueh Chen, Pak Poon
Generating test cases and further evaluating their "quality" are two critical topics in the area of Deep Neural Networks (DNNs). In this domain, different studies have reported that metamorphic testing (MT) serves as an effective test case generation method, where an initial set of source test cases is augmented with identified metamorphic relations (MRs) to produce the corresponding set of follow-up test cases. As a result, the fault detection effectiveness (and, hence, the "quality") of the resulting test suite T, containing these source and follow-up test cases, will most likely be increased. Recently, we observed that some coverage criteria have been proposed to measure the quality of the test suites in the DNN domain. This observation leads to the following interesting and worth exploring research question (RQ): Do these DNN coverage criteria properly reflect the quality improvement after a test suite has been augmented with MRs? We conducted a preliminary empirical study to answer RQ.

Funding

Category 2 - Other Public Sector Grants Category

History

Editor

Vieira M; Madeira H; Antunes N; Zheng Z

Start Page

147

End Page

148

Number of Pages

2

Start Date

12/10/2020

Finish Date

15/10/2020

ISBN-13

9781728177359

Location

Virtual

Publisher

IEEE

Place of Publication

PIscataway, NJ

Peer Reviewed

Yes

Open Access

No

External Author Affiliations

Swinburne University of Technology; Beihang University, China

Era Eligible

Yes

Name of Conference

31st International Symposium on Software Reliability Engineering Workshops

Parent Title

Proceedings: 2020 IEEE 31st International Symposium on Software Reliability Engineering Workshops