IndusAgent: Agentic Reinforcement Learning for Cross-Category Industrial Anomaly Detection

arXiv:2605.20682 · cs.CV · Submitted 2026-05-20 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "IndusAgent: Agentic Reinforcement Learning for Cross-Category Industrial Anomaly Detection".

Jane: Here's the gist of the paper. The gist The proposed IndusAgent framework introduces an agentic reinforcement learning approach to industrial anomaly detection that overcomes limitations of standard multimodal large language models by integrating structured training, active tool orchestration, and an accuracy-gated reward mechanism to achieve state-of-the-art zero-shot performance across diverse industrial benchmarks.

Tom: How it works IndusAgent is a tool-augmented agentic framework designed to bridge the cognitive gap between Multimodal Large Language Models (MLLMs) and high-precision industrial applications by synergizing domain-specific reasoning with autonomous tool orchestration.

Jane: This framework operates through three tightly coupled stages: first, constructing Indus-CoT, a structured dataset that integrates global visual observations and expert normalcy priors for supervision.

Tom: Second, performing Supervised Fine-Tuning to align the VLM with structured industrial diagnostic trajectories and tool-use syntax.

Paper summary: Tom: So we’re looking at this paper today called "IndusAgent: Agentic Reinforcement Learning for Cross-Category Industrial Anomaly Detection." It seems like the main idea is tackling how models see small defects in totally new product types, which is usually really hard.

Jane: That sounds like it. Basically, they are proposing an agentic reinforcement learning approach to industrial anomaly detection that tries to overcome the limitations of standard multimodal large language models by using structured training and active tool orchestration to get better results across different categories.

Lu: The core contribution seems to be building a framework where the AI doesn't just look at a picture once, but it actively decides which tools to use—like cropping, enhancing, or measuring—to get better evidence before making a final diagnosis.

Meng: But what’s the actual claim? Does this agent actually manage to find defects that standard models miss?

Tom: The paper claims they achieve state-of-the-art zero-shot performance across five industrial anomaly benchmarks. They report an accuracy of eighty-three point five percent mean balanced accuracy on those tests, which is better than the seventy-nine point four percent you get from just using a direct Max inference under the common protocol.

Jane: Eighty-three point five percent is solid, especially when you look at how they train it across different seeds, where they hit eighty point five percent for a weight-matched ungated control and then eighty-three point five percent for the gated policy. That shows the reinforcement learning part really helps condition the final output.

Lalam: From my literature view, what’s interesting is how they structure this training through Indus-CoT, which is a dataset of about three thousand trajectories generated with actual tool feedback and then screened against annotations after teacher generation.

Tom: That structured data construction is key because it provides supervision for multiround diagnostic reasoning where the model learns to acquire and use external evidence when it needs to. It’s not just passive input anymore.

Jane: And the reward mechanism they use, the accuracy-gated objective, optimizes an objective that conditions localization, type, and tool rewards on the final diagnostic correctness while also keeping an independent format reward. That sounds like they’re forcing the reasoning loop to actually be useful for finding defects.

Lu: The agent is based on a Qwen3-VL-8B model interacting with six specific tools: cropping, enhancement, measurement, within-image comparison, texture analysis, and external visual opinions from Qwen3-VL-Max <ref:2605.20682#pg1>.

Meng: So they’re not just one big vision model; they’re an agent that has a toolbox it can pick and choose from based on what the image needs to show. What does that mean for deployment?

Tom: It means it tries to solve the problem by getting all the necessary data, instead of relying on one shot perception which is what we see failing in domain-misaligned reasoning.

Jane: When we look at how they tackle things like structural hallucinations, they use tools like Tprior to retrieve normalcy priors describing defect-free geometry and texture patterns to provide a comparison anchor for distinguishing true defects from acceptable variations.

Paper summary: Lu: That comparison anchor is what really addresses the perceptual dilution problem, which the paper identifies as a major issue with ordinary Chain-of-Thought reasoning because without domain knowledge, the model misjudges subtle defects as normal reflections.

Tom: So they are explicitly building in a way for the AI to compare what it sees against known good patterns before it makes a final call on an anomaly.

Jane: And this whole setup is designed to bridge that gap between what MLLMs can see and the high-precision, physical inspection tasks required in industries. It moves beyond just visual perception into active physical verification.

Meng: From a production standpoint, I’m wondering about the cost of running this agentic loop. Does picking six tools and running those sequential reasoning steps add too much latency compared to a simpler, faster model?

Lalam: The paper focuses on optimizing the GRPO policy using an accuracy-gated objective that conditions tool rewards on final diagnostic correctness while retaining an independent format reward. This suggests they are trying to keep the exploration of visual space organized around actual diagnostic gain, which should manage complexity.

Tom: That gating mechanism is how they ensure that unbounded visual exploration is organically aligned with genuine diagnostic information gain, which is a crucial point for any active learning system.

Jane: So, what does this mean for us who only listen to the show? It means that when you hear about these industrial anomaly detection systems, you should think beyond just the language model answering a question; think about an AI that uses specific tools—like measuring distances or enhancing textures—to physically verify a structural issue.

Lu: The implication is that domain-specific alignment through structured datasets like Indus-CoT is critical because MLLM reasoning alone remains unreliable for complex industrial samples, and this framework provides the necessary grounding.

Meng: I see how the tool orchestration helps with ambiguity, but what happens when the required tool isn't in the set? Does it just fail, or can it adapt?

Tom: The training is designed to align the lightweight base model with domain-specific diagnostic protocols, suggesting that if the protocol is defined well enough, it should perform reliably within that structure.

Jane: Ultimately, this work shows how active inspection paradigms can effectively transcend the resolution and semantic limitations inherent in passive visual perception for industrial anomaly detection. It’s about using an agent to inspect instead of just looking.

Lu: And the future work implied here is continuing to refine this tool-augmented reasoning loop, pushing the boundaries of what these agents can learn from external evidence in complex scenarios.

Tom: So, we've covered how IndusAgent uses agentic reinforcement learning and tool augmentation to achieve state-of-the-art results on industrial anomaly detection across different product categories. That’s what this paper is all about.

Conclusion: Tom: So, this paper is about an agentic reinforcement learning approach called IndusAgent that uses tools to find defects in totally different product categories without needing specific training for each one.

Jane: Right, and the authors are using a Qwen3-VL-8B model as the brain of this system, trained through supervised fine-tuning and tool-augmented reinforcement learning <ref:2605.20682#pg1>.

Lu: What's interesting is that they focus on cross-category industrial anomaly detection, which usually means if you train a model for screws, it won't work well for circuit boards.

Meng: I’m interested in how the agent actually decides which tools to use—things like cropping or measuring—to get the evidence it needs before making a final call.

Lalam: The core idea is that this agent doesn't just look at an image once; it actively selects from six different tools, like texture analysis or external visual opinions from a Max model.

Tom: That level of active tool orchestration is what sets them apart from standard models where they just do a single inference pass.

Jane: They show results across five different industrial benchmarks, and they get an eighty-three point five percent mean balanced accuracy on those tests.

Lu: That’s a solid number, especially compared to the seventy-nine point four percent you get from direct Max inference under the common protocol.

Meng: But what does that actually mean for deployment? It implies we can build systems that are more robust when they encounter new product types they haven't seen before.

Tom: It means we’re moving away from models that only work on narrow, specific datasets toward something more general and adaptable.

Jane: That adaptability is key because industrial inspection is messy, and this framework seems to handle that messiness better by gathering multiple perspectives.

Lalam: They also introduce this accuracy-gated reward mechanism, which conditions the tool rewards based on the final diagnostic correctness, making sure the tools are helping find real defects.

Meng: So it's not just about using tools randomly; the system is penalized if those tool choices don't lead to a correct answer.

Tom: Exactly. It’s about aligning the reasoning policy with actual diagnostic outcomes while still allowing for some exploration of visual space.

Jane: This suggests that future systems in this area won't just be about getting a better image; they’ll be about an AI that knows how to ask the right questions of its own vision capabilities.

Lu: The authors point out that domain-specific alignment through structured data, like their Indus-CoT dataset, is critical because MLLM reasoning alone isn't reliable for complex industrial samples.

Meng: I’m still worried about the cost of running all those tool calls sequentially on a live production line; we need to see how efficient that agentic loop is.

Tom: That efficiency is what they’re optimizing with the GRPO policy, trying to balance accuracy and call costs simultaneously.

Jane: So, the big picture here is that we are seeing agents become better inspectors by learning not just what a defect looks like, but *how* to search for it in unfamiliar environments.

Lalam: It's about moving from passive perception to active verification in complex industrial settings.

Tom: Next up, we’ll look at how they actually built that Indus-CoT dataset and what the specific mechanics of their tool library are.

State Key Lab of Processors, Institute of Computing Technology, CAS · Santa Clara University Team 4 Independent Researcher New York University Sun Yat-sen University Nanyang Technological University Stanford University Chinese Academy of Sciences

cs.CV

Submitted: 2026-05-20

Updated: 2026-10-20

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: The gist The proposed IndusAgent framework introduces an agentic reinforcement learning approach to industrial anomaly detection that overcomes limitations of standard multimodal large language

Key concepts

IndusAgent
An agentic reinforcement learning framework based on Qwen3-VL-8B designed for industrial anomaly detection. It uses a policy that decides which specialized tools to use (like cropping or measurement) to gather evidence and then integrates those observations into a final diagnosis.
Tool-Augmented Reinforcement Learning
A training method where the model learns by interacting with an external set of tools. The agent selects from six specific tools—such as cropping, texture analysis, and geometric measurement—to actively gather evidence to solve a problem, rather than relying solely on its internal visual reasoning.
Accuracy-Gated Reward Mechanism
A specialized reward function that guides the learning process. It multiplies the final diagnostic correctness by other factors like localization and tool usage rewards. This forces the model to prioritize making correct final decisions over simply using tools efficiently.

Terminology

Summary

The gist The proposed IndusAgent framework introduces an agentic reinforcement learning approach to industrial anomaly detection that overcomes limitations of standard multimodal large language models by integrating structured training, active tool orchestration, and an accuracy-gated reward mechanism to achieve state-of-the-art zero-shot performance across diverse industrial benchmarks.

How it works

IndusAgent is a tool-augmented agentic framework designed to bridge the cognitive gap between Multimodal Large Language Models (MLLMs) and high-precision industrial applications by synergizing domain-specific reasoning with autonomous tool orchestration. This framework operates through three tightly coupled stages: first, constructing Indus-CoT, a structured dataset that integrates global visual observations and expert normalcy priors for supervision. Second, performing Supervised Fine-Tuning to align the VLM with structured industrial diagnostic trajectories and tool-use syntax. Third, applying Tool-Augmented Reinforcement Learning with a hierarchical reward that jointly balances tool-usage correctness and structural reasoning coherence.

Agentic Toolkit

The agent is instantiated as an agentic policy πθ based on Qwen3-VL-8B. The model interacts with a customized tool space T =

Tcrop, Tprior, Tenhance, and Tmeasure to actively acquire complementary evidence for diagnosis.

Tprior retrieves normalcy priors describing defect-free geometry, texture, and structural patterns, providing a comparison anchor for distinguishing true defects from acceptable variations.

Tenhance applies lightweight image-processing operations to highlight low-contrast texture changes.

Tmeasure computes geometric relations, such as distances, angles, and relative positions to verify misalignment or deformation.

Indus-CoT Dataset

The Indus-CoT dataset is a tool-integrated reasoning dataset that synthesizes image-query trajectories with predefined prompts to bridge visual perception and tool execution. This corpus is constructed by sampling images from Real-IAD and generating about 3,000 reasoning trajectories. The dataset provides supervision for multiround diagnostic reasoning where the model learns to acquire and use external evidence when necessary. Crucially, it is constructed without paired normal reference images for the teacher model.

Accuracy-Gated Reward Mechanism

The framework introduces a gated reinforcement learning objective that jointly optimizes anomaly classification, localization accuracy, anomaly type reasoning, and efficient tool usage. The overall reward is defined as R(τ) = Racc(τ) ·

1 + αRloc(τ) + βRtype(τ) + γRtool(τ) + Rformat(τ), where Racc denotes binary anomaly classification correctness, and it acts as a multiplicative gate.

State-of-the-Art Performance

Extensive evaluations on five industrial anomaly benchmarks, including MVTec-AD and VisA, demonstrate that IndusAgent achieves state-of-the-art zero-shot performance among all existing methods. The results show that IndusAgent achieves state-of-the-art results across five challenging benchmarks (MVTec-AD [7], VisA [111], DTD [3], MPDD [39], and SDD [80]). Notably, on structurally complex datasets such as VisA and MPDD, IndusAgent achieves impressive scores of 76.8% and 72.7%, respectively.

Key Findings

Finding 1 states that domain-specific alignment is critical because MLLM reasoning alone remains unreliable for complex industrial samples. Finding 2 indicates that active tooling complements passive perception because subtle defects are often diluted by large normal regions or scale ambiguity. The accuracy-gated formulation addresses the limitation of standard RL by aligning the reasoning policy with final diagnostic outcomes.

Tool Library Specifications

The external tools are systematically designed to resolve perceptual dilution and structural hallucinations in complex industrial scenarios. The four tools are:

  1. Tcrop (Dynamic Region Cropping Tool).

  2. Tprior (Normalcy Prior Explanation Tool).

  3. Tenhance (Low-Level Visual Enhancer).

  4. Tmeasure (Geometric Verifier).

Improvements for AI systems

  1. Bold header: Active Inspector Paradigm Shift

This improvement allows MLLMs to move beyond passive visual perception by integrating autonomous, multiround tool orchestration. The system can now dynamically resolve ambiguities by synergizing high-resolution region cropping, low-level texture enhancement, quantitative geometric measurement, and expert semantic priors.

  1. Bold header: Tool-Integrated Reasoning Corpus

The system can be trained on the Indus-CoT dataset to learn a structured diagnostic process that explicitly links visual cues, tool feedback, and diagnostic decisions. This provides supervision for domain-aligned, tool-augmented anomaly reasoning by forcing the model to acquire external evidence when necessary.

  1. Bold header: Accuracy-Gated Reward Mechanism

The agent will be trained with an objective that ensures tool invocation occurs only when beneficial, as formalized by the reward function where task rewards are activated only when the final binary diagnosis is correct. This eradicates stochastic tool abuse and fosters a policy biased toward invoking tools only when evidence improves the final diagnosis, preventing hallucinating defect locations.

  1. Bold header: Robustness to Domain Shifts

By employing a category-disjoint and reference-free problem setting, the improved system can generalize to unseen object categories without memorizing category-specific visual patterns. This ensures that performance is validated against novel products, as the training trajectories are constructed from Real-IAD after removing all categories overlapping with the evaluation benchmarks.

  1. Bold header: Data-Adaptive Tool Selection

The system demonstrates a dataset-adaptive, precision-driven inspection paradigm where tools are selected based on data distribution. For instance, it can prioritize Tcrop for object-centric datasets like MVTec (62.4% invocation rate) and Tenhance for texture-centric benchmarks like DTD (34.7% invocation rate).

Sources

Related papers