The internet of things (IoT) and autonomous robotics are transforming multiple industries, and the market for IoT and robotics in mining has witnessed rapid growth.
From performing monotonous and repetitive tasks in daily operations, to the fully automated unmanned mines of the future, the potential applications of autonomous robotic systems in underground mining are plenty. These innovations promise cost-effectiveness and increased safety by reducing the human factor on-site.
In the context of the digitalisation of the mining industry, IoT refers to technologies that enable real-time data collection from in-pit equipment and environmental sensors for the purposes of mine operation planning, among other uses.
The research project Autonome Roboter und Internet der Dinge in untertägigen Anlagen (ARIDuA), instigated by the Technical University Bergakademie Freiberg (TUBF) in Germany, has been exploring the synergies between these two disruptive technologies in underground mining scenarios. On the one hand, robots can benefit from instrumentation of mines with IoT technologies, such as data communication networks and indoor-GPS systems. Robots, could, alternatively, find themselves being used to install and maintain such IoT infrastructures in a mine.
The expected skills of robots
In a test setting of the ARIDuA project, the mobile robot Julius has the task of instrumenting an old ore mine. These boxes contain various sensors to monitor environmental conditions, including gases, temperature, radioactivity and humidity. When distributing the sensor boxes throughout the mine, due to its self-localisation capabilities, the robot is able to provide georeferenced positions of the boxes.
Furthermore, the boxes may contain WLAN modules that have been configured to create data communication network with other sensor boxes installed nearby. The robot can use the network – which it installed – to communicate with a control room outside of the mine. In order to provide such functionalities, an autonomous robot must have a certain skill set that includes:
- localisation and mapping
- collision detection, obstacle avoidance, navigation and path planning
- environmental sensing and interaction with sensor networks
- tool handling and object manipulation
- teleoperation and human-robot interaction capabilities.
To access an area of a mine that is inaccessible for humans, a robot must be equipped with a variety of sensors, a manipulator and other supporting technology.
Julius’s base is an upgraded commercially available robotic platform powered by electric engines with IP67 protection. It has a footprint of 65×105cm and a ground clearance of 14cm to allow driving across rail tracks. Two powerful computers are also on board, and one of these is used for basic processes like drive control, acquiring sensor data,and controlling the robotic arm and hand. The other performs high-level AI processes. These include path planning, image processing, localisation and automatic mapping.
Localisation and mapping
In robotics, mapping is done by localisation and registration of sensor readings over time using simultaneous localisation and mapping (SLAM) algorithms. The location is understood to be a position inside of the generated navigation map, and not so much as a georeferenced position in a mine. This gap is overcome by providing digital plans of the mine.
For self-localisation within a given global map, Julius uses a front-facing RGB-D camera that combines colour and depth information. When travelling in a mine, the robot constructs a further local navigation map using two laser scanners positioned at its front and rear. For this, it uses a SLAM algorithm called real-time appearance-based mapping. In post-processing, the recorded camera and light detection and ranging (LIDAR) data can also be used to create detailed 3D models of the mine.
Navigation, obstacle avoidance and path planning
Based on the navigation map, and using its various laser scanners and cameras, Julius is able to navigate autonomously in a mine that is totally dark, except for the lighting provided by the robot itself. Before the robot is deployed, a human supervisor defines the waypoints the robot has to travel within a digital map. Using a timed-elastic-bands approach, the robot plans its path from waypoint to waypoint.
In addition to having two 2D laser scanners, a 3D laser scanner that is mounted on top of the robot provides 360° coverage. It constantly updates a local-3D navigation map in a perimeter of approximately 5m. If unexpected obstacles are detected in this area, Julius dynamically plans a local path around them, while heading towards the next waypoint along a global path.
Smart-sensor boxes
In the ARIDuA setting, the mobile robot Julius installs a wireless sensor network by distributing smart sensor boxes in a part of the mine that is assumed to be inaccessible for humans. In addition to containing various sensors that measure environmental conditions,they act as Wi-Fi routers.
Multiple sensor boxes span a wireless network, where they communicate with each other and transmit data to the outside control room. In the mesh topology, each box forms a node. As long as a data path from the box to a base station exists, communication to the outside is possible. A prototype of the smart sensor box has been developed.
Object manipulation and tool handling
Julius is designed to perform versatile object manipulation tasks in the mine. Besides placing and picking up smart sensor boxes, Julius should, for example, be able to operate handheld scanners or take water samples using devices made for human operation. Julius’s three-finger gripper – which resembles a human hand – also allows different grasp types. Its arm comprises six joints that can rotate 360°. A rubber glove and custom sleeve protect the gripper and the arm against the harsh environmental conditions in the mine. To provide an image of grasping operations, a dedicated RGB-D camera is mounted on the wrist joint. With all this, Julius is ready to place a smart sensor box in the mine.
Teleoperation and human-robot interaction
While Julius is mostly operated in autonomous mode, human controlcan be a better option in non-routine situations that require complex decision making. Teleoperation is possible when in range of the wireless communication network. Video feeds and relevant sensor readings are sent to the teleoperator, while control commands are sent to the robot.
Julius’s ability to bring his own wireless communication network to the mine helps to ensure the necessary data transmission bandwidth. In the mine, the robot may also interact with mine workers; in fact, it can detect the presence of people in its vicinity through the 3D laser scanner, which helps it keep at a safe distance. On-site, workers can operate Julius via a handheld controller, and specify procedures using a laptop computer carried by it.
Synergy between IoT and robotics in mining
The ARIDuA project is a study in how autonomous robots with versatile information gathering, and object manipulation skills, can be used to install and maintain smart IoT equipment in underground mining environments. Once installed, this IoT equipment not only provides valuable information about the mining environment, but also benefits the operation of the robots themselves.
As well as establishing a wireless sensor and data communication network by placing smart sensor boxes in pre-planned locations, Julius monitors the signal strength of the wireless data connection, and may decide where to place further Wi-Fi access points. By establishing a reliable data communication infrastructure, Julius ensures that it can be teleoperated from a remote control room.
Synergy is also achieved by the robot’s self-localisation capabilities. Under regular conditions, its visual sensors provide accurate position estimates to allow exact georeferencing of the placed sensor boxes. Once installed, the sensor boxes act as beacons to make the self-localisation of Julius robust under poor visual conditions, such as dust from blasting.
With the robot, IoT devices support easy pick-and-place operations; for example, the smart box handle fits Julius’s hand well for better grip. In addition, the sensor boxes contain a Bluetooth module for short-range communication with the robot. An integrated inertial measurement unit (IMU) senses the boxes’ orientation to notify the robot if the box is not positioned upright. Visual AR markers on the boxes also improve the robot’s detection of the sensors.
While current state-of-the art robotic systems do not match the capabilities of skilled human workers, near-term applications mean operating in restricted areas that are too dangerous for human access. In the long term, robots that are capable of installing and maintaining IoT infrastructures will likely play an integral part in the operation and excavation of future unmanned mines.
This article originally appeared in World Mining Frontiers magazine.
