Interactive Anomaly Detection for Articulated Objects via Motion Anticipation
Neural Information Processing Systems (NeurIPS 2025)
TL;DR - We introduce InteractiveAD, a novel framework that actively manipulates objects to detect hidden functional anomalies by comparing observed motion with anticipated normal motion.
Abstract
This paper presents a novel problem, interactive anomaly detection (AD) for articulated objects, and introduces a tailored solution that detects functional anomalies by integrating vision, interaction, and anticipation. Unlike traditional AD methods that rely on passive visual observations, our approach actively manipulates objects to reveal anomalies that would otherwise remain hidden. Our method learns to generate a sequence of actions to interact exclusively with normal objects and to anticipate the resulting normal motion. During inference, the model applies predicted actions to the object and compares the observed motion with the anticipated motion to detect anomalies. Additionally, we introduce a new benchmark, PartNet-IAD, for interactive AD, which includes articulated objects with realistic functional anomalies. Experiments show strong generalization to detect anomalies in both seen and unseen object categories.Method
A. Motion Prior Network
Our motion prior $\Psi$ is trained to predict the expected normal rigid motion from a partial point cloud observation $\mathbf{S}$ and an action $\mathbf{a}$ (parameterized by its position $\mathbf{p}$ and direction $\mathbf{u}$). The network is trained using a large set of action-motion interaction pairs, generated in simulation from normal objects. We implement $\Psi$ as a deep neural network comprising a backbone feature encoder and two decoders for motion estimation and segmentation respectively.
At inference time, the model genereates an anticipated motion trajectory by sequentially applying the learned motion prior $\Psi$. The same action sequence is executed on the actual object by an agent (e.g., a robotic arm), and the observed trajectory is recorded. Comparing the anticipated and observed motions enables AD based on motion discrepancies.
Training Data Generation using Physics-Based Simulation
To train the motion prior, we require a large-scale dataset of action–motion interaction pairs capturing the normal motion behavior of common object types. We generate this dataset in simulation using PyBullet, where a 3D position on an object part’s surface and an action direction are randomly sampled from a uniform spherical distribution. The simulated robot arm then executes the sampled action, and we record the resulting motion as a rigid transformation matrix, along with a segmentation mask indicating the moved part. We use 3D objects from PartNet-Mobility for the simulation; however, our framework can extend beyond these, as our motion parametrization is general.
Some examples of the simulation are shown above (in a delayed timescale). In practice, the simulation is parallelized across multiple CPU cores to efficiently generate large-scale simulation data.
Results Analysis & Detailed Visualization
A. Results on the PartNet-IAD
Score Map
Anticipated Motion
Observed Motion
Anomaly detected
Score Map
Anticipated Motion
Observed Motion
Anomaly detected
Score Map
Anticipated Motion
Observed Motion
Anomaly detected
Score Map
Anticipated Motion
Observed Motion
Anomaly detected
Score Map
Anticipated Motion
Observed Motion
Normal
Score Map
Anticipated Motion
Observed Motion
Anomaly detected
B. Motion Estimation Results on the Real-World AKB-48 Dataset
Our method also generalizes to real-world scanned objects from the AKB-48 dataset and can be applied to estimate motion corresponding to any arbitrary action parameter.
Applications and Future Work
This work enables robotic agents to actively detect functional anomalies in
articulated objects, with potential benefits in quality control, assistive robotics, and autonomous
inspection. Those may include furniture (e.g. for cabinets) and consumer electronics (e.g. for laptops)
industry for quality control and durability testing. It may also be used in product development and
prototyping to identify design flaws and improve functionality, and, as well as, in service industry
for inspecting and repairing articulated objects, and warranty assessments.
Cite our work
@inproceedings{bhunia2025interactive,
title={Interactive Anomaly Detection for Articulated Objects via Motion Anticipation},
author={Bhunia, Ankan and Li, Changjian and Bilen, Hakan},
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
year={2025}
}