Robotics (cs.RO)

Abstract: Soft object manipulation tasks in domestic scenes pose a significant challenge for existing robotic skill learning techniques due to their complex dynamics and variable shape characteristics. Since learning new manipulation skills from human demonstration is an effective way for robot applications, developing prior knowledge of the representation and dynamics of soft objects is necessary. In this regard, we propose a pre-trained soft object manipulation skill learning model, namely SoftGPT, that is trained using large amounts of exploration data, consisting of a three-dimensional heterogeneous graph representation and a GPT-based dynamics model. For each downstream task, a goal-oriented policy agent is trained to predict the subsequent actions, and SoftGPT generates the consequences of these actions. Integrating these two approaches establishes a thinking process in the robot's mind that provides rollout for facilitating policy learning. Our results demonstrate that leveraging prior knowledge through this thinking process can efficiently learn various soft object manipulation skills, with the potential for direct learning from human demonstrations.

Abstract: Tactile sensing plays an irreplaceable role in robotic material recognition. It enables robots to distinguish material properties such as their local geometry and textures, especially for materials like textiles. However, most tactile recognition methods can only classify known materials that have been touched and trained with tactile data, yet cannot classify unknown materials that are not trained with tactile data. To solve this problem, we propose a tactile zero-shot learning framework to recognise unknown materials when they are touched for the first time without requiring training tactile samples. The visual modality, providing tactile cues from sight, and semantic attributes, giving high-level characteristics, are combined together to bridge the gap between touched classes and untouched classes. A generative model is learnt to synthesise tactile features according to corresponding visual images and semantic embeddings, and then a classifier can be trained using the synthesised tactile features of untouched materials for zero-shot recognition. Extensive experiments demonstrate that our proposed multimodal generative model can achieve a high recognition accuracy of 83.06% in classifying materials that were not touched before. The robotic experiment demo and the dataset are available at https://sites.google.com/view/multimodalzsl.

Abstract: Quadrupedal robots have emerged as a cutting-edge platform for assisting humans, finding applications in tasks related to inspection and exploration in remote areas. Nevertheless, their floating base structure renders them susceptible to fall in cluttered environments, where manual recovery by a human operator may not always be feasible. Several recent studies have presented recovery controllers employing deep reinforcement learning algorithms. However, these controllers are not specifically designed to operate effectively in cluttered environments, such as stairs and slopes, which restricts their applicability. In this study, we propose a robust all-terrain recovery policy to facilitate rapid and secure recovery in cluttered environments. We substantiate the superiority of our proposed approach through simulations and real-world tests encompassing various terrain types.

Abstract: Multiple solutions mainly originate from the existence of redundant degrees of freedom in the robot arm, which may cause difficulties in inverse model learning but they can also bring many benefits, such as higher flexibility and robustness. Current multi-solution inverse model learning methods rely on conditional deep generative models, yet they often fail to achieve sufficient precision when learning multiple solutions. In this paper, we propose Conditional Embodied Self-Supervised Learning (CEMSSL) for robot arm multi-solution inverse model learning, and present a unified framework for high-precision multi-solution inverse model learning that is applicable to other conditional deep generative models. Our experimental results demonstrate that our framework can achieve a significant improvement in precision (up to 2 orders of magnitude) while preserving the properties of the original method. The related code will be available soon.

Abstract: With the implementation of the new EU regulation 2022/1426 regarding the type-approval of the automated driving system (ADS) of fully automated vehicles, scenario-based testing has gained significant importance in evaluating the performance and safety of advanced driver assistance systems and automated driving systems. However, the exploration and generation of concrete scenarios from a single logical scenario can often lead to a number of similar or redundant scenarios, which may not contribute to the testing goals. This paper focuses on the the goal to reduce the scenario set by clustering concrete scenarios from a single logical scenario. By employing clustering techniques, redundant and uninteresting scenarios can be identified and eliminated, resulting in a representative scenario set. This reduction allows for a more focused and efficient testing process, enabling the allocation of resources to the most relevant and critical scenarios. Furthermore, the identified clusters can provide valuable insights into the scenario space, revealing patterns and potential problems with the system's behavior.

Abstract: The robot operating system is the de-facto standard for designing and implementing robotics applications. Several previous works deal with the integration of heterogeneous accelerators into ROS-based applications. One of these approaches is ReconROS, which enables nodes to be completely mapped to hardware. The follow-up work fpgaDDS extends ReconROS by an intra-FPGA data distribution service to process topic-based communication between nodes entirely in hardware. However, the application of this approach is strictly limited to communication between nodes implemented in hardware only. This paper introduces gateways to close the gap between topic communication in hardware and software. Gateways aim to reduce data transfers between hardware and software by synchronizing a hardware-and software-mapped topic. As a result, data must be transferred only once compared to a separate data transmission for each subscribing hardware node in the baseline. Our measurements show significant speedups in multi-subscriber scenarios with large message sizes. From the conclusions of these measurements, we present a methodology for the communication mapping of ROS 2 computation graphs. In the evaluation, an autonomous driving real-world example benefits from the gateway and achieves a speedup of 1.4.

Abstract: In this paper, we address the problem of autonomous search and vessel detection in an unknown GNSS-denied maritime environment with fixed-wing UAVs. The main challenge in such environments with limited localization, communication range, and the total number of UAVs and sensors is to implement an appropriate search strategy so that a target vessel can be detected as soon as possible. Thus we present informed and non-informed methods used to search the environment. The informed method relies on an obtained probabilistic map, while the non-informed method navigates the UAVs along predefined paths computed with respect to the environment. The vessel detection method is trained on synthetic data collected in the simulator with data annotation tools. Comparative experiments in simulation have shown that our combination of sensors, search methods and a vessel detection algorithm leads to a successful search for the target vessel in such challenging environments.

Abstract: Differential GPS, commonly referred as DGPS, is a well-known and very accurate localization system for many outdoor applications in particular for mobile outdoor robotics. The most common drawback of DGPS systems are the high costs for both base station and receivers. In this paper, we present a setup that uses third-party open-source software and a Ublox ZED-F9P chip to build a ROS-enabled low-cost DGPS setup that is ready to use in a few hours. The main goal of this paper is to analyze and evaluate the repetitive and absolute accuracy of the system. The first measurement also examines the differences between a SAPOS base station and a locally installed one consisting of low-cost components. During the evaluation process of the absolute accuracy, a moving mobile robot is used on the receiver side. It is tracked through a highly accurate VICON motion capture system.

Abstract: This article presents a new three-degree-of-freedom (3-DOF) parallel mechanism (PM) with two translations and one rotation (2T1R), designed based on the topological design theory of the parallel mechanism using position and orientation characteristics (POC). The PM is primarily intended for use in package sorting and delivery. The mobile platform of the PM moves along a translation axis, picks up objects from a conveyor belt, and tilts them to either side of the axis. We first calculate the PM's topological characteristics, such as the degree of freedom (DOF) and the degree of coupling, and provide its topological analytical formula to represent the topological information of the PM. Next, we solve the direct and inverse kinematic models based on the kinematic modelling principle using the topological features. The models are purely analytic and are broken down into a series of quadratic equations, making them suitable for use in an industrial robot. We also study the singular configurations to identify the serial and parallel singularities. Using the decoupling properties, we size the mechanism to address the package sorting and depositing problem using an algebraic approach. To determine the smallest segment lengths, we use a cylindrical algebraic decomposition to solve a system with inequalities.

Abstract: Understanding and manipulating articulated objects, such as doors and drawers, is crucial for robots operating in human environments. We wish to develop a system that can learn to articulate novel objects with no prior interaction, after training on other articulated objects. Previous approaches for articulated object manipulation rely on either modular methods which are brittle or end-to-end methods, which lack generalizability. This paper presents FlowBot++, a deep 3D vision-based robotic system that predicts dense per-point motion and dense articulation parameters of articulated objects to assist in downstream manipulation tasks. FlowBot++ introduces a novel per-point representation of the articulated motion and articulation parameters that are combined to produce a more accurate estimate than either method on their own. Simulated experiments on the PartNet-Mobility dataset validate the performance of our system in articulating a wide range of objects, while real-world experiments on real objects' point clouds and a Sawyer robot demonstrate the generalizability and feasibility of our system in real-world scenarios.

Abstract: Simultaneous localisation and mapping (SLAM) play a vital role in autonomous robotics. Robotic platforms are often resource-constrained, and this limitation motivates resource-efficient SLAM implementations. While sparse visual SLAM algorithms offer good accuracy for modest hardware requirements, even these more scalable sparse approaches face limitations when applied to large-scale and long-term scenarios. A contributing factor is that the point clouds resulting from SLAM are inefficient to use and contain significant redundancy. This paper proposes the use of subset selection algorithms to reduce the map produced by sparse visual SLAM algorithms. Information-theoretic techniques have been applied to simpler related problems before, but they do not scale if applied to the full visual SLAM problem. This paper proposes a number of novel information\hyp{}theoretic utility functions for map point selection and optimises these functions using greedy algorithms. The reduced maps are evaluated using practical data alongside an existing visual SLAM implementation (ORB-SLAM 2). Approximate selection techniques proposed in this paper achieve trajectory accuracy comparable to an offline baseline while being suitable for online use. These techniques enable the practical reduction of maps for visual SLAM with competitive trajectory accuracy. Results also demonstrate that SLAM front-end performance can significantly impact the performance of map point selection. This shows the importance of testing map point selection with a front-end implementation. To exploit this, this paper proposes an approach that includes a model of the front-end in the utility function when additional information is available. This approach outperforms alternatives on applicable datasets and highlights future research directions.

Abstract: Deep reinforcement learning (DRL) has seen remarkable success in the control of single robots. However, applying DRL to robot swarms presents significant challenges. A critical challenge is non-stationarity, which occurs when two or more robots update individual or shared policies concurrently, thereby engaging in an interdependent training process with no guarantees of convergence. Circumventing non-stationarity typically involves training the robots with global information about other agents' states and/or actions. In contrast, in this paper we explore how to remove the need for global information. We pose our problem as a Partially Observable Markov Decision Process, due to the absence of global knowledge on other agents. Using collective transport as a testbed scenario, we study two approaches to multi-agent training. In the first, the robots exchange no messages, and are trained to rely on implicit communication through push-and-pull on the object to transport. In the second approach, we introduce Global State Prediction (GSP), a network trained to forma a belief over the swarm as a whole and predict its future states. We provide a comprehensive study over four well-known deep reinforcement learning algorithms in environments with obstacles, measuring performance as the successful transport of the object to the goal within a desired time-frame. Through an ablation study, we show that including GSP boosts performance and increases robustness when compared with methods that use global knowledge.

Abstract: Collaborative robots must simultaneously be safe enough to operate in close proximity to human operators and powerful enough to assist users in industrial tasks such as lifting heavy equipment. The requirement for safety necessitates that collaborative robots are designed with low-powered actuators. However, some industrial tasks may require the robot to have high payload capacity and/or long reach. For collaborative robot designs to be successful, they must find ways of addressing these conflicting design requirements. One promising strategy for navigating this tradeoff is through the use of static balancing mechanisms to offset the robot's self weight, thus enabling the selection of lower-powered actuators. In this paper, we introduce a novel, 2 degree of freedom static balancing mechanism based on spring-loaded, wire-wrapped cams. We also present an optimization-based cam design method that guarantees the cams stay convex, ensures the springs stay below their extensions limits, and minimizes sensitivity to unmodeled deviations from the nominal spring constant. Additionally, we present a model of the effect of friction between the wire and the cam. Lastly, we show experimentally that the torque generated by the cam mechanism matches the torque predicted in our modeling approach. Our results also suggest that the effects of wire-cam friction are significant for non-circular cams.

Abstract: Long-term visual localization is an essential problem in robotics and computer vision, but remains challenging due to the environmental appearance changes caused by lighting and seasons. While many existing works have attempted to solve it by directly learning invariant sparse keypoints and descriptors to match scenes, these approaches still struggle with adverse appearance changes. Recent developments in image transformations such as neural style transfer have emerged as an alternative to address such appearance gaps. In this work, we propose to combine an image transformation network and a feature-learning network to improve long-term localization performance. Given night-to-day image pairs, the image transformation network transforms the night images into day-like conditions prior to feature matching; the feature network learns to detect keypoint locations with their associated descriptor values, which can be passed to a classical pose estimator to compute the relative poses. We conducted various experiments to examine the effectiveness of combining style transfer and feature learning and its training strategy, showing that such a combination greatly improves long-term localization performance.

Abstract: This paper proposes a novel operation for controlling a mobile robot using a head-mounted device. Conventionally, robots are operated using computers or a joystick, which creates limitations in usability and flexibility because control equipment has to be carried by hand. This lack of flexibility may prevent workers from multitasking or carrying objects while operating the robot. To address this limitation, we propose a hands-free method to operate the mobile robot with a human gaze in an Augmented Reality (AR) environment. The proposed work is demonstrated using the HoloLens 2 to control the mobile robot, Robotnik Summit-XL, through the eye-gaze in AR. Stable speed control and navigation of the mobile robot were achieved through admittance control which was calculated using the gaze position. The experiment was conducted to compare the usability between the joystick and the proposed operation, and the results were validated through surveys (i.e., SUS, SEQ). The survey results from the participants after the experiments showed that the wearer of the HoloLens accurately operated the mobile robot in a collaborative manner. The results for both the joystick and the HoloLens were marked as easy to use with above-average usability. This suggests that the HoloLens can be used as a replacement for the joystick to allow hands-free robot operation and has the potential to increase the efficiency of human-robot collaboration in situations when hands-free controls are needed.