448 pubs · 6,916 cites · h-index 39
Wear and tear detection in fleet and shared vehicle systems is a critical challenge, particularly in rental and car-sharing services, where minor damage, such as dents, scratches, and underbody impacts, often goes unnoticed or is detected too late. Currently, manual inspection methods are the default approach, but are labour-intensive and prone to human error. In contrast, state-of-the-art image-based methods are less reliable when the vehicle is moving, and they cannot effectively capture underbody damage due to limited visual access and spatial coverage. This work introduces a novel multi-modal architecture based on anomaly detection to address these issues. Sensors such as Inertial Measurement Units (IMUs) and microphones are integrated into a compact device mounted on the vehicle’s windshield. This approach supports real-time damage detection while avoiding the need for highly resource-intensive sensors. We developed multiple variants of multi-modal autoencoder-based architectures and evaluated them against unimodal and state-of-the-art methods. Our multi-modal ensemble model with pooling achieved the highest performance, with a Receiver Operating Characteristic-Area Under Curve (ROC-AUC) of 92%, demonstrating its effectiveness in real-world applications. This approach can also be extended to other applications, such as improving automotive safety. It can integrate with airbag systems for efficient deployment and help autonomous vehicles by complementing other sensors in collision detection.
Competition assessed various reinforcement learning methods for dynamic control in robotics, suggesting benchmarks for real-world applications.
Exploration shows potential for finding exobiological signatures in lava caves on planetary bodies, suggesting robotic teams could enhance research efficiency.
Variational quantum algorithms (VQAs) and their applications in the field of quantum machine learning through parametrized quantum circuits (PQCs) are thought to be one major way of leveraging noisy intermediate-scale quantum computing devices. However, differences in the performance of certain VQA architectures are often unclear since established best practices, as well as detailed studies, are missing. In this paper, we build upon the work by Schuld [] and Vidal [] by comparing how well popular ansatz for PQCs learn different one-dimensional truncated Fourier series. We also examine dissipative quantum neural networks (dQNN) as introduced by Beer [] and propose a data reupload structure for dQNNs to increase their capability for this regression task. By comparing the results for different PQC architectures, we can provide guidelines for designing efficient PQCs.
Future space missions will include multi-robot systems, with greater autonomy and a large degree of heterogeneity for a wider range of task capabilities and redundancy. It is imperative that both software (learning models, parallelizing capabilities, resource distribution, etc.) and hardware factors must be considered during decentralized task negotiation to lead to better performance of the team. By utilizing the formalism of contextual Markov decision processes, team composition can be incorporated into the learning process and used for more meaningful and reliable evaluation using measures such as total time, overall consumed energy, performance feedback from tasks, or damage incurred. Improved team performance will in turn enhance the overall results of the mission. Planetary exploration tasks often involve time, communication and energy constraints. Such missions are also prone to noisy sensor data (e.g., camera images distorted by dust), as well as wear and tear on hardware (e.g., wheels, manipulators). To ensure that such factors do not jeopardize the mission, they must be taken into account. Therefore, this paper describes a software framework for the reliable execution of tasks in constrained and dynamic environments. Our work leverages the advantages of heterogeneity for more resilient planetary missions by addressing two aspects—first, the integration of hardware parameters into the negotiation process, and second the analysis of how the integration of team performance metrics, particularly adaptability and mutual support, in task negotiation plays a role in the overall mission success.
• We train a transformation-equivariant 3D detection model on low-resolution pointcloud maps collected using a MBES. • We perform 3D detection of objects in pointclouds, predicting their positions and orientations. • We generate synthetic and real datasets which include a variety of geometric objects to investigate the effect of pointcloud sparsity and the addition of synthetic noise on the detection performance. • We show the generalizability of the proposed method to detect objects in real MBES maps while being trained on synthetic data only. Exploration of the underwater domain has always been a challenging task for researchers to tackle. Perception being a major part of this exploration, requires robust systems and sensors to perform accurate mapping, detection, tracking, and 3d-reconstruction of the underwater medium. Although many use optical sensors for such tasks, they get highly affected by the reduced visibility and high turbidity causing the loss of essential features in captured scenes. This work proposes the use of a deep-learning model for acoustic-based 3D object detection. The idea is to detect regions of interest/objects in sparse multibeam echo sounder 3D maps which are low in resolution and consist of minimum amount of features. Since acquiring data from a real environment is hard, the model was trained on a synthetically generated datasets consisting of several objects. Simulation experiments showed promising results performing successful 3D object detection, which is then further tested in a real experiment on seven different objects. The system performed accurate 3D detection achieving an mAP of 0.75.
This study presents a deep learning-based prediction system with an elevated approach to prevent tip-over incidents on planetary exploration rovers, enhancing their operational safety and reliability. Planetary rovers, critical for space exploration missions, must navigate through uneven surfaces and terrains with undefined interaction properties. Future planetary rovers must navigate harsher terrains, like steep craters and caves, to access critical scientific data, significantly risking tip-over in any state of operational control. The proposed system employs linear accelerations and angular velocities measured by the accelerometer and the gyroscope of the Inertial Measurements Unit (IMU) to monitor the rover’s dynamic behavior and stability while navigating the environment. By leveraging deep learning algorithms, the system evaluates predictions and true measurements in real time to identify potential tip-overs. Additionally, the system provides the possibility to adjust the rover’s motion to prevent failure. The efficacy of this prediction-based approach is validated through simulations and field tests on two robotic platforms, the Asguard v4 and Coyote 3 rovers, demonstrating its capability to reduce the incidence of tip-overs under various challenging conditions. The integration of this system aims to extend the operational lifespan of rovers, optimize mission outcomes, and enhance the overall safety of planetary exploration missions.
Shor's algorithm is one of the most prominent quantum algorithms, yet finding efficient implementations remains an active research challenge. While many approaches focus on low-level modular arithmetic optimizations, a broader perspective can provide additional opportunities for improvement. By adopting a mid-level abstraction, we analyze the algorithm as a sequence of computational tasks, enabling systematic identification of idle time and optimization of execution flow. Building on this perspective, we first introduce an alternating design approach to minimize idle time while preserving qubit efficiency in Shor's algorithm. By strategically reordering tasks for simultaneous execution, we achieve a substantial reduction in overall execution time. Extending this approach to distributed implementations, we demonstrate how task rearrangement enhances execution efficiency in the presence of multiple distribution channels. Furthermore, to effectively evaluate the impact of design choices, we employ static timing analysis (STA)—a technique from classical circuit design—to analyze circuit delays while accounting for hardware-specific execution characteristics, such as measurement and reset delays in monolithic architectures and ebit generation time in distributed settings. Finally, we validate our approach by integrating modular exponentiation circuits from QRISP and constructing circuits for factoring numbers up to 64 bits. Through an extensive study across neutral atom, superconducting, and ion trap quantum computing platforms, we analyze circuit delays, highlighting trade-offs between qubit efficiency and execution time. Our findings provide a structured framework for optimizing compiled quantum circuits for Shor's algorithm, tailored to specific hardware constraints.
ARC-OPT (Adaptive Robot Control using OPTimization) is a C++ library for Whole-Body Control (WBC) (Sentis & Khatib, 2006) of complex robotic systems, such as humanoids, quadrupedal robots, or mobile manipulators.In general, WBC describes a robot control problem in terms of costs and constraints of a quadratic program (QP).The cost function thereby minimizes the residuals of multiple feedback controllers, each dedicated to a specific robot task, along with further objectives.In each control cycle, the QP is solved and the solution, which should fulfill all objectives if possible, is sent to the robot's actuators.WBC is a reactive control approach, which targets redundant robots and is able to control multiple tasks simultaneously, like, e.g., grasping and balancing on a humanoid robot.
A great advantage of legged robots is their ability to operate on particularly difficult and obstructed terrain, which demands dynamic, robust, and precise movements. The study of obstacle courses provides invaluable insights into the challenges legged robots face, offering a controlled environment to assess and enhance their capabilities. Traversing it with a one-legged hopper introduces intricate challenges, such as planning over contacts and dealing with flight phases, which necessitates a sophisticated controller. A novel model predictive parkour controller is introduced, that finds an optimal path through a real-time changing obstacle course with mixed integer motion planning. The execution of this optimized path is then achieved through a state machine employing a PD control scheme with feedforward torques, ensuring robust and accurate performance.
As artificial intelligence gains new capabilities, it becomes important to evaluate it on real-world tasks. In particular, the fields of robotics and reinforcement learning (RL) are lacking in standardized benchmarking tasks on real hardware. To facilitate reproducibility and stimulate algorithmic advancements, we held an AI Olympics competition at IJCAI 2023 conference based on the double pendulum system in the RealAIGym project where the participants were asked to develop a controller for the swing up and stabilization task. This paper presents the methods and results from the top participating teams and provides insights into the real-world performance of RL algorithms with respect to a baseline time-varying LQR controller.
Reliable detection of human intentions from electroencephalogram (EEG) to improve human-robot interaction (HRI) has recently gained significant importance. To ensure safe and satisfactory interactions, implicit detection of erroneous behavior of robotic systems, particularly assistive devices, is essential. This can be achieved by detecting error-related potentials (ErrPs) in EEG, evoked by visual, tactile, or visuo-tactile stimuli. Of these, the ErrPs evoked tactilely with the help of a robot remains unexplored and has been the main focus of this competition. The task for participating teams was to develop robust AI models for continuous real-time classification of erroneous behavior of assistive robotic devices from the human EEG. Even though the competition results prove its feasibility, a performance gap (balanced accuracy and computation time) of more than 10% was observed between the offline and online classification of errors in real-world scenarios. In addition to the competitive AI models developed by the participating teams, this competition also contributed towards a one-of-its-kind open-access EEG and EMG dataset, a lossless live streaming solution for EEG data, and a novel quantitative metric for benchmarking online asynchronous EEG detection solutions.
Exoskeleton-based support for patients requires the learning of individual machine-learning models to recognize movement intentions of patients based on the electroencephalogram (EEG). A major issue in EEG-based movement intention recognition is the long calibration time required to train a model. In this paper, we propose a transfer learning approach that eliminates the need for a calibration session. This approach is validated on healthy subjects in this study. We will use the proposed approach in our future rehabilitation application, where the movement intention of the affected arm of a patient can be inferred from the EEG data recorded during bilateral arm movements enabled by the exoskeleton mirroring arm movements from the unaffected to the affected arm. For the initial evaluation, we compared two trained models for predicting unilateral and bilateral movement intentions without applying a classifier transfer. For the main evaluation, we predicted unilateral movement intentions without a calibration session by transferring the classifier trained on data from bilateral movement intentions. Our results showed that the classification performance for the transfer case was comparable to that in the non-transfer case, even with only 4 or 8 EEG channels. Our results contribute to robotic rehabilitation by eliminating the need for a calibration session, since EEG data for training is recorded during the rehabilitation session, and only a small number of EEG channels are required for model training.
Anomaly detection deals with detecting deviations from established patterns within data. It has various applications like autonomous driving, predictive maintenance, and medical diagnosis. To improve anomaly detection accuracy, transfer learning can be applied to large, pre-trained models and adapt them to the specific application context. In this paper, we propose a novel framework for online-adaptive anomaly detection using transfer learning. The approach adapts to different environments by selecting visually similar training images and online fitting a normality model to EfficientNet features extracted from the training subset. Anomaly detection is then performed by computing the Mahalanobis distance between the normality model and the test image features. Different similarity measures (SIFT/FLANN, Cosine) and normality models (MVG, OCSVM) are employed and compared with each other. We evaluate the approach on different anomaly detection benchmarks and data collected in controlled laboratory settings. Experimental results showcase a detection accuracy exceeding 0.975, outperforming the state-of-the-art ET-NET approach.
Optimal behaviours of a system to perform a specific task can be achieved by leveraging the coupling between trajectory optimization, stabilization, and design optimization.This approach is particularly advantageous for underactuated systems, which are systems that have fewer actuators than degrees of freedom and thus require for more elaborate control systems.This paper proposes a novel co-design algorithm, namely Robust Trajectory Control with Design optimization (RTC-D).An inner optimization layer (RTC) simultaneously performs direct transcription (DIRTRAN) to find a nominal trajectory while computing optimal hyperparameters for a stabilizing time-varying linear quadratic regulator (TVLQR).RTC-D augments RTC with a design optimization layer, maximizing the system's robustness through a time-varying Lyapunov-based region of attraction (ROA) analysis.This analysis provides a formal guarantee of stability for a set of off-nominal states.The proposed algorithm has been tested on two different underactuated systems: the torque-limited simple pendulum and the cart-pole.Extensive simulations of off-nominal initial conditions demonstrate improved robustness, while real-system experiments show increased insensitivity to torque disturbances.
This paper presents the design, analysis, and performance evaluation of RicMonk, a novel three-link brachiation robot equipped with passive hook-shaped grippers. Brachiation, an agile and energy-efficient mode of locomotion observed in primates, has inspired the development of RicMonk to explore versatile locomotion and maneuvers on ladder-like structures. The robot’s anatomical resemblance to gibbons and the integration of a tail mechanism for energy injection contribute to its unique capabilities. The paper discusses the use of the Direct Collocation methodology for optimizing trajectories for the robot’s dynamic behaviors and stabilization of these trajectories using a Time-varying Linear Quadratic Regulator. With RicMonk we demonstrate bidirectional brachiation, and provide comparative analysis with its predecessor, AcroMonk - a two-link brachiation robot, to demonstrate that the presence of a passive tail helps improve energy efficiency. The system design, controllers, and software implementation are publicly available on GitHub at https://github.com/dfki-ric-underactuated-lab/ricmonk and the video demonstration of the experiments can be viewed at https://youtu.be/hOuDQI7CD8w.
This paper investigates four different mobile robots with respect to their driving characteristics and soil preservation properties in an agricultural environment. Thereby, robots of classical design from agriculture as well as systems from space robotics with advanced locomotion concepts are considered to determine the individual advantages of each rover concept with respect to the application domain. Locomotion experiments were conducted to analyze the general driving behavior, tensile force, and obstacle‐surmounting capability and ground interaction of each robot. Various soil conditions typical for the area of application are taken into account, which are varied in terms of moisture and density. The presented work covers the specification of the conducted experiments, documentation of the implementation as well as analysis and evaluation of the collected data. In the evaluation, particular attention is paid to the change in driving characteristics under different soil conditions, as well as to the soil stress caused by driving, since soil quality is of critical importance for agricultural applications. The analysis shows that the advanced locomotion concepts, as used in space robotics, also have positive implications for certain requirements in agricultural applications, such as maneuverability in wet conditions and soil conservation. The results show potential for design innovations in agricultural robotics that can be used, to open up new fields of application for instance in the context of precision farming.
This paper presents the design, analysis, and performance evaluation of RicMonk, a novel three-link brachiation robot equipped with passive hook-shaped grippers. Brachiation, an agile and energy-efficient mode of locomotion observed in primates, has inspired the development of RicMonk to explore versatile locomotion and maneuvers on ladder-like structures. The robot's anatomical resemblance to gibbons and the integration of a tail mechanism for energy injection contribute to its unique capabilities. The paper discusses the use of the Direct Collocation methodology for optimizing trajectories for the robot's dynamic behaviors and stabilization of these trajectories using a Time-varying Linear Quadratic Regulator. With RicMonk we demonstrate bidirectional brachiation, and provide comparative analysis with its predecessor, AcroMonk - a two-link brachiation robot, to demonstrate that the presence of a passive tail helps improve energy efficiency. The system design, controllers, and software implementation are publicly available on GitHub and the video demonstration of the experiments can be viewed YouTube.
Motion planning for robot manipulators in a cluttered environment is one of the most researched areas in the field of robotics.For high degrees of freedom (DOF) robotic systems, optimization-based motion planners are often preferred for trajectory planning, as they are computationally efficient and provide a smooth, locally optimal solution.To converge to an optimal solution, an optimization-based motion planner needs a good initial trajectory. Finding an initial trajectory, which is not far away from the basin of attraction of the optimum is not a trivial task. In this work, we propose an Initial Trajectory Prediction Network (ITPNet), a deep neural network framework for predicting an initial trajectory to warm start optimization-based motion planners.Given a planning task in the form of task and environment features, the ITPNet predicts the best initial trajectory for warm starting an optimization-based motion planner. Two different task features: joint and Cartesian features, and three types of environment features extracted using: Principal Component Analysis (PCA), Variational Autoencoder (VAE), and Signed Spatial Distance (SSD) techniques are compared.The learned models are evaluated on an upper-torso humanoid system in two different scenarios. The results show that the model, using the Cartesian task and the SSD-based environment features, efficiently learns the mapping between the planning tasks and the optimal trajectories. Warm-starting the planner with the predicted initial trajectory, even for an unseen environment results in a higher success rate and requires fewer iterations.