A Neuromorphic Approach to Obstacle Avoidance in Robot Manipulation
- Neuromorphic computing aims to mimic the computational principles of the brain in silico and has motivated research into event-based vision and spiking neural networks (SNNs). Event cameras (ECs) capture local, independent changes in brightness, and offer superior power consumption, response latencies, and dynamic ranges compared to frame-based cameras. SNNs replicate neuronal dynamics observed in biological neurons and propagate information in sparse sequences of ”spikes”. Apart from biological fidelity, SNNs have demonstrated potential as an alternative to conventional artificial neural networks (ANNs), such as in reducing energy expenditure and inference time in visual classification. Although potentially beneficial for robotics, the novel event-driven and spike-based paradigms remain scarcely explored outside the domain of aerial robots. To investigate the utility of brain-inspired sensing and data processing in a robotics application, we developed a neuromorphic approach to real-time, online obstacle avoidance on a manipulator with an onboard camera. Our approach adapts high-level trajectory plans with reactive maneuvers by processing emulated event data in a convolutional SNN, decoding neural activations into avoidance motions, and adjusting plans in a dynamic motion primitive formulation. We conducted simulated and real experiments with a Kinova Gen3 arm performing simple reaching tasks involving static and dynamic obstacles. Our implementation was systematically tuned, validated, and tested in sets of distinct task scenarios, and compared to a non-adaptive baseline through formalized quantitative metrics and qualitative criteria. The neuromorphic implementation facilitated reliable avoidance of imminent collisions in most scenarios, with 84% and 92% median success rates in simulated and real experiments, where the baseline consistently failed. Adapted trajectories were qualitatively similar to baseline trajectories, indicating low impacts on safety, predictability and smoothness criteria. Among notable properties of the SNN were the correlation of processing time with the magnitude of perceived motions (captured in events) and robustness to different event emulation methods. Preliminary tests with a DAVIS346 EC showed similar performance, validating our experimental event emulation method. These results motivate future efforts to incorporate SNN learning, utilize neuromorphic processors, and target other robot tasks to further explore this approach.
Document Type: | Report |
---|---|
Language: | English |
Author: | Ahmed Faisal Abdelrahman |
Number of pages: | x, 173 |
ISBN: | 978-3-96043-111-4 |
ISSN: | 1869-5272 |
URN: | urn:nbn:de:hbz:1044-opus-76772 |
DOI: | https://doi.org/10.18418/978-3-96043-111-4 |
Supervisor: | Paul G. Plöger, Maren Bennewitz, Matias Valdenegro-Toro |
Publishing Institution: | Hochschule Bonn-Rhein-Sieg |
Date of first publication: | 2023/11/30 |
Award: | AFCEA-Studienpreis 2023, 1. Platz |
Award: | VDI-Förderpreis 2024, 1.Platz |
Funding: | We gratefully acknowledge the support by the b-it International Center for Information Technology. |
Series (Volume): | Technical Report / Hochschule Bonn-Rhein-Sieg University of Applied Sciences. Department of Computer Science (02-2023) |
Departments, institutes and facilities: | Fachbereich Informatik |
Dewey Decimal Classification (DDC): | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik |
Series: | Technical Report / University of Applied Sciences Bonn-Rhein-Sieg. Department of Computer Science |
Entry in this database: | 2023/11/30 |
Licence (Multiple languages): | In Copyright - Educational Use Permitted (Urheberrechtsschutz - Nutzung zu Bildungszwecken erlaubt) |