Refine
Departments, institutes and facilities
Document Type
- Conference Object (72)
- Preprint (15)
- Article (11)
- Report (5)
- Part of a Book (3)
- Book (monograph, edited volume) (1)
- Research Data (1)
- Doctoral Thesis (1)
Year of publication
Keywords
- Automatic Short Answer Grading (2)
- Cognitive robot control (2)
- Drosophila (2)
- Explainable robotics (2)
- Learning from experience (2)
- Navigation (2)
- Object detection (2)
- Object recognition (2)
- robot execution failures (2)
- robotics (2)
- Active Learning (1)
- Advanced Driver Assistance Systems (1)
- Autism Spectrum Disorder (1)
- BERT (1)
- Ball Tracking (1)
- Ball tracking (1)
- Benchmarking (1)
- Bioinformatics (1)
- Black-Box Optimization (1)
- Cognitive robotics (1)
- Comparative analysis (1)
- Computer Science - Computer Vision and Pattern Recognition (1)
- Computer Science - Learning (1)
- Continual robot learning (1)
- Dieseleinspritzanlage (1)
- Domestic Robots (1)
- Domestic robotics (1)
- Domestic service robots (1)
- Dynamic motion primitives (1)
- ELMo (1)
- Ego-Motion Estimation (1)
- Eingebettetes System (1)
- Elektrische Simulation (1)
- Entwurf (1)
- Everyday object manipulation (1)
- Experiment design (1)
- External faults (1)
- FOS: Computer and information sciences (1)
- Fault handling (1)
- Feature extraction (1)
- Fusion (1)
- GPT (1)
- GPT-2 (1)
- Hardware (1)
- Hardwareentwurf (1)
- Histograms (1)
- Human robot interaction (1)
- Human-Centered Robotics (1)
- Hyper-parameter Tuning (1)
- Knowledge Graphs (1)
- Learning and Adaptive Systems (1)
- Learning from demonstration (1)
- Machine Learning (1)
- Machine-learning (1)
- Manipulation tasks (1)
- Modalities (1)
- Multimodal (1)
- Naive physics (1)
- Natural Language Processing (1)
- Natural language understanding (1)
- Optical Flow (1)
- Optical flow (1)
- Quality control (1)
- Rapid prototyping (1)
- Real-Time Image Processing (1)
- Real-time image processing (1)
- Robot commands (1)
- Robot failure diagnosis (1)
- Robot learning (1)
- Robot-Assisted Therapy (1)
- Robotics (1)
- Robotics (cs.RO) (1)
- Robustness (1)
- SISAL (1)
- SPICE (1)
- Scale Tuning (1)
- Scene text recognition, active vision, domestic robot, pantilt, auto-zoom, auto-focus, adaptive aperture control (1)
- Semantic scene understanding (1)
- SensorFusion (1)
- Simulation (1)
- Software (1)
- Softwareentwicklung (1)
- Spherical Treadmill (1)
- Spherical treadmill (1)
- State machines (1)
- Text detection (1)
- Text recognition (1)
- ToF Camera (1)
- Toyota HSR (1)
- Transfer Learning (1)
- Transformers (1)
- Transparency (1)
- Unidirectional thermoplastic composites (1)
- Virtual Reality (1)
- Virtual reality (1)
- Whole body motion (1)
- anomaly detection (1)
- assistive robotics (1)
- dependable robots (1)
- faults in robotics (1)
- force sensing (1)
- learning-based fault detection and diagnosis (1)
- naive physics (1)
- object detection (1)
- personalized behaviour model (1)
- property-based testing for robots (1)
- reinforcement learning (1)
- remote diagnosis (1)
- robot action diagnosability (1)
- robot behaviour model (1)
- robot component monitoring (1)
- robot introspection (1)
- robot personalisation (1)
- robotic black box (1)
- run-time adaptation (1)
- sensor fusion (1)
- sensor-based fault detection and diagnosis (1)
- simulation-based robot testing (1)
- skill execution models (1)
- slip detection (1)
- tactile sensing (1)
- unexpected situations (1)
- user modelling (1)
- verification and validation of robot action execution (1)
Robot Action Diagnosis and Experience Correction by Falsifying Parameterised Execution Models
(2021)
When faced with an execution failure, an intelligent robot should be able to identify the likely reasons for the failure and adapt its execution policy accordingly. This paper addresses the question of how to utilise knowledge about the execution process, expressed in terms of learned constraints, in order to direct the diagnosis and experience acquisition process. In particular, we present two methods for creating a synergy between failure diagnosis and execution model learning. We first propose a method for diagnosing execution failures of parameterised action execution models, which searches for action parameters that violate a learned precondition model. We then develop a strategy that uses the results of the diagnosis process for generating synthetic data that are more likely to lead to successful execution, thereby increasing the set of available experiences to learn from. The diagnosis and experience correction methods are evaluated for the problem of handle grasping, such that we experimentally demonstrate the effectiveness of the diagnosis algorithm and show that corrected failed experiences can contribute towards improving the execution success of a robot.
The majority of biomedical knowledge is stored in structured databases or as unstructured text in scientific publications. This vast amount of information has led to numerous machine learning-based biological applications using either text through natural language processing (NLP) or structured data through knowledge graph embedding models (KGEMs). However, representations based on a single modality are inherently limited. To generate better representations of biological knowledge, we propose STonKGs, a Sophisticated Transformer trained on biomedical text and Knowledge Graphs. This multimodal Transformer uses combined input sequences of structured information from KGs and unstructured text data from biomedical literature to learn joint representations. First, we pre-trained STonKGs on a knowledge base assembled by the Integrated Network and Dynamical Reasoning Assembler (INDRA) consisting of millions of text-triple pairs extracted from biomedical literature by multiple NLP systems. Then, we benchmarked STonKGs against two baseline models trained on either one of the modalities (i.e., text or KG) across eight different classification tasks, each corresponding to a different biological application. Our results demonstrate that STonKGs outperforms both baselines, especially on the more challenging tasks with respect to the number of classes, improving upon the F1-score of the best baseline by up to 0.083. Additionally, our pre-trained model as well as the model architecture can be adapted to various other transfer learning applications. Finally, the source code and pre-trained STonKGs models are available at https://github.com/stonkgs/stonkgs and https://huggingface.co/stonkgs/stonkgs-150k.
When an autonomous robot learns how to execute actions, it is of interest to know if and when the execution policy can be generalised to variations of the learning scenarios. This can inform the robot about the necessity of additional learning, as using incomplete or unsuitable policies can lead to execution failures. Generalisation is particularly relevant when a robot has to deal with a large variety of objects and in different contexts. In this paper, we propose and analyse a strategy for generalising parameterised execution models of manipulation actions over different objects based on an object ontology. In particular, a robot transfers a known execution model to objects of related classes according to the ontology, but only if there is no other evidence that the model may be unsuitable. This allows using ontological knowledge as prior information that is then refined by the robot’s own experiences. We verify our algorithm for two actions – grasping and stowing everyday objects – such that we show that the robot can deduce cases in which an existing policy can generalise to other objects and when additional execution knowledge has to be acquired.
Swedish wheeled mobile robots have remarkable mobility properties allowing them to rotate and translate at the same time. Being holonomic systems, their kinematics model results in the possibility of designing separate and independent position and heading trajectory tracking control laws. Nevertheless, if these control laws should be implemented in the presence of unaccounted actuator saturation, the resulting saturated linear and angular velocity commands could interfere with each other thus dramatically affecting the overall expected performance. Based on Lyapunov’s direct method, a position and heading trajectory tracking control law for Swedish wheeled robots is developed. It explicitly accounts for actuator saturation by using ideas from a prioritized task based control framework.
Co-design is concerned with the joint design of hardware and software making up an embedded computer system [Wol94]. A top down design flow for an embedded system begins with a system specification. If it is executable, it may be used for simulation, system verification or to identify algorithmical bottlenecks. In contrast to other chapters of this book, the specification is not developed in this case study, rather it is given from the beginning. Furthermore we are not concerned with partitioning or synthesis of dedicated HW. Instead we focus on the problem how to find an off-the-shelf micro-controller which implements the desired functionality and meets all specification constraints. If feasible, this is usually much cheaper then using dedicated hardware. This chapter will answer the question of feasibility for a real life problem from automobile industry.
Background: Virtual reality combined with spherical treadmills is used across species for studying neural circuits underlying navigation.
New Method: We developed an optical flow-based method for tracking treadmil ball motion in real-time using a single high-resolution camera.
Results: Tracking accuracy and timing were determined using calibration data. Ball tracking was performed at 500 Hz and integrated with an open source game engine for virtual reality projection. The projection was updated at 120 Hz with a latency with respect to ball motion of 30 ± 8 ms.
Comparison: with Existing Method(s) Optical flow based tracking of treadmill motion is typically achieved using optical mice. The camera-based optical flow tracking system developed here is based on off-the-shelf components and offers control over the image acquisition and processing parameters. This results in flexibility with respect to tracking conditions – such as ball surface texture, lighting conditions, or ball size – as well as camera alignment and calibration.
Conclusions: A fast system for rotational ball motion tracking suitable for virtual reality animal behavior across different scales was developed and characterized.
With regard to performance well established SW-only design methodologies proceed by making the initial specification run first, then by enhancing its functionality and finally by optimizing it. When designing Embedded Systems (EbS) this approach is not viable since decisive design decisions like e.g. the estimation of required processing power or the identification of those parts of the specification which need to be delegated to dedicated HW depend on the fastness and fairness of the initial specification. We here propose a sequence of optimization steps embedded into the design flow, which enables a structured way to accelerate a given working EbS specification at different layers. This sequence of accelerations comprises algorithm selection, algorithm transformation, data transformation, implementation optimization and finally HW acceleration. It is analyzed how all acceleration steps are influenced by the specific attributes of the underlying EbS. The overall acceleration procedure is explained and quantified at hand of a real-life industrial example.
SISAL: User manual
(1990)
Cognitive robotics aims at understanding biological processes, though it has also the potential to improve future robotics systems. Here we show how a biologically inspired model of motor control with neural fields can be augmented with additional components such that it is able to solve a basic robotics task, that of obstacle avoidance. While obstacle avoidance is a well researched area, the focus here is on the extensibility of a biologically inspired framework. This work demonstrates how easily the biological inspired system can be used to adapt to new tasks. This flexibility is thought to be a major hallmark of biological agents.
Application of underwater robots are on the rise, most of them are dependent on sonar for underwater vision, but the lack of strong perception capabilities limits them in this task. An important issue in sonar perception is matching image patches, which can enable other techniques like localization, change detection, and mapping. There is a rich literature for this problem in color images, but for acoustic images, it is lacking, due to the physics that produce these images. In this paper we improve on our previous results for this problem (Valdenegro-Toro et al, 2017), instead of modeling features manually, a Convolutional Neural Network (CNN) learns a similarity function and predicts if two input sonar images are similar or not. With the objective of improving the sonar image matching problem further, three state of the art CNN architectures are evaluated on the Marine Debris dataset, namely DenseNet, and VGG, with a siamese or two-channel architecture, and contrastive loss. To ensure a fair evaluation of each network, thorough hyper-parameter optimization is executed. We find that the best performing models are DenseNet Two-Channel network with 0.955 AUC, VGG-Siamese with contrastive loss at 0.949 AUC and DenseNet Siamese with 0.921 AUC. By ensembling the top performing DenseNet two-channel and DenseNet-Siamese models overall highest prediction accuracy obtained is 0.978 AUC, showing a large improvement over the 0.91 AUC in the state of the art.
CASTLE is a co-design platform developed at GMD SET institute. It provides a number of design tools for configuring application specific design flows. This paper presents a walk through the CASTLE co-design environment, following the design flow of a video processing system. The design methodology and the tool usage for this real life example are described, as seen from a designers point of view. The design flow starts with a C/C++ program and gradually derives a register-transfer level description of a processor hardware, as well as the corresponding compiler for generating the processor opcode. The main results of each design step are presented and the usage of the CASTLE tools at each step is explained.
Loading of shipping containers for dairy products often includes a press-fit task, which involves manually stacking milk cartons in a container without using pallets or packaging. Automating this task with a mobile manipulator can reduce worker strain, and also enhance the efficiency and safety of the container loading process. This paper proposes an approach called Adaptive Compliant Control with Integrated Failure Recovery (ACCIFR), which enables a mobile manipulator to reliably perform the press-fit task. We base the approach on a demonstration learning-based compliant control framework, such that we integrate a monitoring and failure recovery mechanism for successful task execution. Concretely, we monitor the execution through distance and force feedback, detect collisions while the robot is performing the press-fit task, and use wrench measurements to classify the direction of collision; this information informs the subsequent recovery process. We evaluate the method on a miniature container setup, considering variations in the (i) starting position of the end effector, (ii) goal configuration, and (iii) object grasping position. The results demonstrate that the proposed approach outperforms the baseline demonstration-based learning framework regarding adaptability to environmental variations and the ability to recover from collision failures, making it a promising solution for practical press-fit applications.
In the design of robot skills, the focus generally lies on increasing the flexibility and reliability of the robot execution process; however, typical skill representations are not designed for analysing execution failures if they occur or for explicitly learning from failures. In this paper, we describe a learning-based hybrid representation for skill parameterisation called an execution model, which considers execution failures to be a natural part of the execution process. We then (i) demonstrate how execution contexts can be included in execution models, (ii) introduce a technique for generalising models between object categories by combining generalisation attempts performed by a robot with knowledge about object similarities represented in an ontology, and (iii) describe a procedure that uses an execution model for identifying a likely hypothesis of a parameterisation failure. The feasibility of the proposed methods is evaluated in multiple experiments performed with a physical robot in the context of handle grasping, object grasping, and object pulling. The experimental results suggest that execution models contribute towards avoiding execution failures, but also represent a first step towards more introspective robots that are able to analyse some of their execution failures in an explicit manner.
Saliency methods are frequently used to explain Deep Neural Network-based models. Adebayo et al.'s work on evaluating saliency methods for classification models illustrate certain explanation methods fail the model and data randomization tests. However, on extending the tests for various state of the art object detectors we illustrate that the ability to explain a model is more dependent on the model itself than the explanation method. We perform sanity checks for object detection and define new qualitative criteria to evaluate the saliency explanations, both for object classification and bounding box decisions, using Guided Backpropagation, Integrated Gradients, and their Smoothgrad versions, together with Faster R-CNN, SSD, and EfficientDet-D0, trained on COCO. In addition, the sensitivity of the explanation method to model parameters and data labels varies class-wise motivating to perform the sanity checks for each class. We find that EfficientDet-D0 is the most interpretable method independent of the saliency method, which passes the sanity checks with little problems.
The dataset contains the following data from successful and failed executions of the Toyota HSR robot placing a book on a shelf.
RGB images from the robot's head camera
Depth images from the robot's head camera
Rendered images of the robot's 3D model from the point of view of the robot's head camera
Force-torque readings from a wrist-mounted force-torque sensor
Joint efforts, velocities and positions
extrinsic and intrinsic camera calibration parameters
frame-level anomaly annotations
The anomalies that occur during execution include:
the manipulated book falling down
books on the shelf being disturbed significantly
camera occlusions
robot being disturbed by an external collision
The dataset is split into a train, validation and test set with the following number of trials:
Train: 48 successful trials
Validation: 6 successful trials
Test: 60 anomalous trials and 7 successful trials