Real Faults in Deep Learning Fault Benchmarks: How Real Are They?

arXiv:2412.16336 · cs.SE, cs.AI, cs.LG · Submitted 2024-12-20 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Real Faults in Deep Learning Fault Benchmarks".

Jane: The gist: A manual analysis of 490 faults from five different deep learning benchmarks reveals that only 18.5% are eligible for study,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Now, let's go over the specific title and who wrote this. The paper is titled "Real Faults in Deep Learning Fault Benchmarks: How Real Are They?" and it was written by Gunel Jahangirova, Nargiz Humbatova, Jinhan Kim, and Paolo Tonella.

Jane: That's a team of researchers from different places—King’s College London, the Software Institute at USI in Switzerland, and KAIST in South Korea. It shows how this kind of testing is becoming a big deal across different research communities.

Tom: Yeah, it highlights that this isn't just one group doing this; it’s a broader conversation happening internationally about making sure our tools for finding deep learning bugs are actually hitting real problems.

Lu: What's interesting is that the motivation for this whole paper comes from seeing that researchers often rely on mutation testing tools to generate faulty deep learning models, and they want to check if those generated faults are actually representative of real-world errors.

Meng: From a production standpoint, I’m always skeptical of benchmarks created purely for academic purposes because they don't reflect the messiness of actual deployment data.

Lalam: Based on the literature I’ve reviewed, this paper is important because it directly addresses that gap by manually analyzing existing fault sets to check their realism.

The paper's summary: Tom: So, what did they actually do? They performed a systematic literature search and then manually analyzed four hundred ninety faults from five different benchmarks to see if they were credible <ref:2412.16336#pg1,490 faults from five different benchmarks>.

Jane: The methodology they used was pretty specific—they checked four conditions to judge realism: the source code for the buggy version had to match the reported source, the fix applied in one benchmark had to match what was reported in another, and so on.

Tom: They looked at correspondence to sources, fault types present, whether faults were reproducible, and whether they could be generated by existing mutation operators.

Lu: The results show that for correspondence to the source code, bugs and fixes matched the source eighty-one point two percent of the time on average, but for two of those benchmarks it dropped down to around fifty-four percent, which is quite low.

Meng: That drop from eighty-one point two to fifty-four percent suggests that a significant chunk of these faults are actually inconsistent with where they were found in the original code.

Lalam: And when they looked at training data, they found it was realistic only in thirty-four point seven percent of cases, and it matched the source data only forty-six point four percent of the time on average.

Jane: It sounds like the realism check is pretty strict for deep learning faults; they aren't just looking for any random code error, they’re looking for errors tied to real training scenarios.

The paper's improvements: Tom: The authors pointed out a few things they think should be done to improve these benchmarks. One big suggestion is that researchers need to apply stricter inclusion criteria when mining deep learning bugs.

Jane: They want people to focus on the whole code and make sure that any source information reported includes the exact training dataset used, plus a precise fix solution if one was proposed.

Lu: They also suggest focusing on making fault types more diverse; they noted that three types were disproportionately common: wrong activation function, which accounted for ninety-five out of three hundred fourteen bugs.

Meng: I agree with Lu on that; if you only have a few kinds of errors dominating the list, you aren't really stress-testing the repair system against a wide range of failure modes.

Lalam: Another point they made is that mutation tools should be enhanced because currently, only twenty-two percent to sixty-three percent of bugs can be simulated with existing mutation operators.

Tom: That means we need better tools for generating those more complex changes, which is important if we want to test advanced repair techniques.

Conclusion: Jane: So, looking at the final results, the paper concludes that only fifty-eight of the faults actually satisfied all their strict realism conditions. That’s a pretty small subset.

Tom: And that ties into their reproducibility findings too; out of one hundred sixty-five bugs they checked, only eighty-six could reproduce and behavior was stable in seventy-nine cases, meaning only about fifty-two percent of the faults are reproducible across multiple runs.

Lu: The main contribution they are making here is proposing a set of characteristics for analyzing bugs specifically to check their realism against real-world scenarios.

Meng: I think the practical implication is that we need to stop blindly using these existing benchmarks and start investing in creating independent ones that are truly robust.

Lalam: It suggests applying this analysis of fault types could help us study which combinations of errors are most frequent when building higher-order mutation generation processes.

Tom: And they also suggested that using containerization platforms like Docker is an effective strategy to handle version problems related to benchmark maintainability, which makes them more reliable.

Jane: So, the overall message of this paper is that we need a much tighter vetting process for deep learning fault benchmarks if we want these testing techniques to be truly effective.

Lu: It’s about moving away from just having a dataset and starting to build systems that ensure those faults actually matter in practice.

Meng: If you’re shipping models, you don't want to waste time on forty-two percent of your test cases that turn out to be unrealistic or non-reproducible.

Lalam: The paper "Real Faults in Deep Learning Fault Benchmarks: How Real Are They?" shows us exactly where the current testing landscape needs refinement.

King’s College London · University of Svizzera Italiana (USI) · University of Svizzera Italiana (USI) · KAIST · University of Svizzera Italiana (USI)

cs.SE, cs.AI, cs.LG

Submitted: 2024-12-20

Updated: 2026-10-19

Code: https://github.com/tensorflow/models

License: http://creativecommons.org/licenses/by-sa/4.0/

Importance score: 73/100

The gist: The gist: A manual analysis of 490 faults from five different deep learning benchmarks reveals that only 18.5% are eligible for study, and successful reproduction was achieved in only 52% of cases.

Key concepts

Realism Analysis Conditions: These are the four strict checks used to determine if a fault benchmark is realistic. They require matching the buggy code, the applied fix, the training data source, and ensuring that training data is either existing or mathematically sound. Passing all checks indicates a high degree of realism.
Fault Correspondence: This measures how well the reported bugs and their corresponding fixes match the original source code they were extracted from. Results showed high correspondence (81.2% on average), but significant inconsistencies in some benchmarks, meaning many faults are not accurately linked to their origin.
Fault Reproducibility: This assesses whether a fault can be reliably reproduced across multiple runs. Only 52% of the considered bugs were reproducible, indicating that many reported issues are unstable or difficult to replicate consistently in practice.

Terminology

Benchmark — 3× in this paper · explained in 2 episodes
Deep Learning — 2× in this paper

Summary

The gist: A manual analysis of 490 faults from five different deep learning benchmarks reveals that only 18.5% are eligible for study, and successful reproduction was achieved in only 52% of cases.

Introduction and Motivation

The growing popularity of deep learning (DL) systems makes ensuring that they are fault-free a task of utmost importance In response to this challenge, in the last few years, research in the Software Engineering (SE) field has strongly focused on proposing approaches for testing, fault localisation, and repair of such systems An obvious requirement for evaluation of these techniques is to demonstrate that they can detect, localise or fix faults in faulty DL systems The need for faulty versions of programs has long been essential for evaluating similar approaches that target traditional software systems

Methodology for Realism Analysis

The researchers performed a manual analysis of 490 faults from five different benchmarks and identified that 314 of them are eligible for our study Our investigation focuses specifically on how well the bugs correspond to the sources they were extracted from, which fault types are represented, and whether the bugs are reproducible To check realism, they performed a systematic literature search and checked four conditions: (1) the source code for the buggy version in the benchmark should match the buggy code reported in its source, (2) the fix applied in the benchmark should match the fix reported in the source, (3) the training data used in the benchmark must match the one mentioned in the source, and (4) realistic training data must refer to well-known existing datasets or be generated using a sound mathematical/statistical procedure. If a bug passes all checks, they run it to check for stable reproduction across multiple runs.

Analysis of Research Questions

The study addressed five research questions to critically evaluate the reproducibility and realism of fault benchmarks

RQ1 [Correspondence to Source]: Do the faulty versions in the benchmarks and their fixes correspond to the source from which the fault was extracted?

The results show that, on average, bugs and fixes in the benchmarks match the sources 81.2% and 78% of the time For two of the benchmarks, these numbers are around 54%, indicating that nearly half of the bugs were extracted inconsistently to their source.

RQ2 [Training Dataset]: Does the training data used in the benchmarks correspond to the source from which it was extracted and is it realistic?

The results reveal that training datasets used in the benchmarks are realistic only in 34.7% of cases Moreover, they correspond to the ones at the source of the bugs only in 46.4% of cases, on average.

RQ3 [Fault Types]: What types of faults do the benchmarks contain and what is the frequency of each fault type?

The benchmarks are also diverse when it comes to fault types, containing on average 23 and all together 52 different fault types However, the level of representation of each fault type is quite disproportionate For instance, out of 592 fault components in 314 bugs, the three most occurring fault types are wrong activation function (95), wrong data preprocessing (93) and wrong number of epochs (74).

RQ4 [Similarity to Artificial Faults]: Can the faulty models in the benchmarks be generated by artificially injecting faults using existing mutation operators?

The results indicate that 22% to 63% of the bugs in the benchmarks analysed can be simulated with the existing mutation operators As such, only 50% of the bugs and 67% of the faults can be generated using existing mutation operators, with these numbers going up to 63% and 86% for the DeepFD dataset.

RQ5 [Fault Reproducibility]: Are the faults in the benchmarks reproducible and stable across multiple runs?

Out of 165 bugs, they were able to successfully reproduce 86 and the behaviour of 79 of them was stable across all the performed runs As results show, only 52% of considered bugs are reproducible.

Conclusion and Implications

The study concludes that only 58 of the faults satisfy the conditions on realism The main contributions include conducting a systematic literature review, proposing a set of characteristics for analyzing bugs to check their realism, sharing a subset of realistic, reproducible, and stable faults, and offering an in-depth discussion on improving bug benchmarks. Researchers should apply stricter inclusion criteria when mining DL bugs and invest on the creation of independent benchmarks to avoid authors’ bias Moreover, their analysis of fault types can be used to study the most frequent combinations that could be integrated in higher-order mutation generation processes. The work also suggests that containerisation using platforms like Docker is an effective strategy to address version problems related to benchmark maintainability.

Data Availability

The experimental data, and the evaluation results that support the findings of this study are available at the following link https://anonymous.4open.science/r/HowRealAreThey-D68E

References

[1] H. Do, S. Elbaum, and G. Rothermel, “Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact,” Empirical Software Engineering, vol. 10, pp. 405–435, 2005

[2] R. Just, D. Jalali, and M. D. Ernst, “Defects4j: A database of existing faults to enable controlled testing studies for java programs,” in Proceedings of the 2014 international symposium on software testing and analysis, 2014, pp. 437–440

[3] P. Gyimesi, B. Vancsics, A. Stocco, D. Mazinanian, A. Beszédes, R. Ferenc, and A. Mesbah, “Bugsjs: a benchmark of javascript bugs,” in 2019 12th IEEE Conference on Software Testing and Analysis (ICST). IEEE, 2019

[4] J. Cao, M. Li, X. Chen, M. Wen, Y. Tian, B. Wu, and S.-C. Cheung, “Deepfd: automated fault diagnosis and localization for deep learning programs,” ser. ICSE ’22

[5] A. Nikanjam, H. B. Braiek, M. M. Morovati, and F. Khomh, “Automatic fault detection for deep learning programs using graph transformations,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 31, no. 1

[6] M. Wardat, B. D. Cruz, W. Le, and H. Rajan, “Deepdiagnosis: automatically diagnosing faults and recommending actionable fixes in deep learning programs,” in Proceedings of the 44th international conference on software engineering

[7] M. M. Morovati, A. Nikanjam, F. Khomh, and Z. M. Jiang, “Bugs in machine learning-based systems: a faultload benchmark,” Empirical Software Engineering, vol. 28, no. 3

[8] Y. Liang, Y. Lin, X. Song, J. Sun, Z. Feng, and J. S.

Improvements for AI systems

Bold header: Stricter Realism Criteria for Benchmark Curation

The paper suggests applying a set of characteristics for the analysis of the bugs to check their realism, specifically requiring that the source code for the buggy version in the benchmark should match the buggy code reported in its source and that the fix applied in the benchmark should match the fix reported in the source. This rigorous enforcement will ensure that evaluation techniques are tested against faults truly representative of human development errors.

Bold header: Enhanced Fault Type Diversity

The analysis reveals a disproportionate representation of fault types, noting the three most occurring fault types are wrong activation function (95), wrong data preprocessing (93) and wrong number of epochs (74). Improving benchmark creation should focus on ensuring that faults are distributed evenly across fault types to provide a more comprehensive stress test for automated repair systems.

Bold header: Integration of Higher-Order Mutants

Since 46% of faults have a higher order, the paper recommends that mutation tools be enhanced to generate these complex changes, as existing tools only apply single-order mutants. This will enable evaluation campaigns on more realistic, composite faults that mimic real-world failures.

Bold header: Development of Version-Agnostic Reproduction Strategies

To address issues where reproduction is hampered by outdated dependencies, the system should implement containerization using platforms like Docker to preserve the environment under which a bug was identified, ensuring legacy software in these bugs remains functional despite updates to the underlying frameworks.

Bold header: Refinement of Source Information Requirements for SO Faults

To mitigate risks associated with vague information in discussion forums, strict criteria must be applied: relevant posts must report the whole code and indicate the training dataset used, as well as contain an accepted answer proposing a precise fix solution. This prevents reliance on low-fidelity data that creates a false promise of real bugs.

Abstract

As the adoption of Deep Learning (DL) systems continues to rise, an increasing number of approaches are being proposed to test these systems, localise faults within them, and repair those faults. The best attestation of effectiveness for such techniques is an evaluation that showcases their capability to detect, localise and fix real faults. To facilitate these evaluations, the research community has collected multiple benchmarks of real faults in DL systems. In this work, we perform a manual analysis of 490 faults from five different benchmarks and identify that 305 of them are eligible for our study. Our investigation focuses specifically on how well the bugs correspond to the sources they were extracted from, which fault types are represented, and whether the bugs are reproducible. Our findings indicate that only 15.4% of the faults satisfy our realism conditions. Our attempts to reproduce these faults were successful only in 54% of cases.

Sources

Towards Enhancing the Reproducibility of Deep Learning Bugs: An Empirical Study