Volltext-Downloads (blau) und Frontdoor-Views (grau)

Using Visual Anomaly Detection for Task Execution Monitoring

  • Execution monitoring is essential for robots to detect and respond to failures. Since it is impossible to enumerate all failures for a given task, we learn from successful executions of the task to detect visual anomalies during runtime. Our method learns to predict the motions that occur during the nominal execution of a task, including camera and robot body motion. A probabilistic U-Net architecture is used to learn to predict optical flow, and the robot's kinematics and 3D model are used to model camera and body motion. The errors between the observed and predicted motion are used to calculate an anomaly score. We evaluate our method on a dataset of a robot placing a book on a shelf, which includes anomalies such as falling books, camera occlusions, and robot disturbances. We find that modeling camera and body motion, in addition to the learning-based optical flow prediction, results in an improvement of the area under the receiver operating characteristic curve from 0.752 to 0.804, and the area under the precision-recall curve from 0.467 to 0.549.

Export metadata

Additional Services

Search Google Scholar Check availability


Show usage statistics
Document Type:Conference Object
Author:Santosh Thoduka, Juergen Gall, Paul G. Plöger
Parent Title (English):2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 27 Sept.-1 Oct. 2021, Prague, Czech Republic
First Page:4604
Last Page:4610
ArXiv Id:http://arxiv.org/abs/2107.14206
Date of first publication:2021/12/16
Funding Information:This work has been supported by the Bonn-Aachen International Center for Information Technology, a PhD scholarship from the Graduate Institute at Hochschule Bonn-Rhein-Sieg, and the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - GA 1927/6-2 (FOR 2535).
Departments, institutes and facilities:Fachbereich Informatik
Dewey Decimal Classification (DDC):0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik
Entry in this database:2021/08/04