Offering improved speed and precision while reducing the health and safety risk to personnel, particularly when high terrain or cliffs are involved, many mining sites are turning to drones when surveying and mapping mineral landscapes. Andrew Barnett speaks to Dr Stefan Hrabar, CEO and co-founder of Emesent, about this technology.
Transport automation and the augmentation of mobile phones are driving a new generation of technology that’s poised to transform mining operations and safety conditions, both above the surface and deep underground.
A cornerstone of these developments is lighter and more capable light detection and ranging (LIDAR) sensors – devices that can calculate the distance to a surface or object using packets of reflected light.
When coupled with drones, they provide impressive three-dimensional mapping capabilities in difficult terrain. Add autonomy into the mix and LIDAR-equipped drones have the potential to drastically improve safety and contribute to detailed geotechnical surveying across the mining sector.
Dr Stefan Hrabar is CEO and co-founder of Emesent, the Australian tech company behind Hovermap, a product that integrates LIDAR sensors with mapping and autonomous technology that interfaces with drones.
The company began life as a spin-off from Data61, the digital and data arm of the Commonwealth Scientific and Industrial Research Organisation (CSIRO), an Australian governmental agency responsible for scientific research.
“We had been developing autonomy for drones for a long time at CSIRO,” explains Hrabar, adding that preliminary work concentrated on adding sensors and computing power to drones to allow them to survey their environments without flying into obstacles.
Work then progressed to develop the ability for the LIDAR sensor, to not only sense the environment but also to estimate how the drone was moving without using GPS – a capability Hrabar describes as extremely powerful.
“We then started exploring different use cases and getting feedback from industry. It didn’t take long for miners to find out about what we were doing and said, ‘Hey, have you heard about a stope?’”
A stope is created to access ore underground, with the excavation and extraction process often resulting in large unsupported spaces.
However, stoping is a broad term and techniques vary depending on the ore, the type of mine and the country rock. Regardless, stopes can be dangerous places given their inherent risk of collapse, making them an ideal testing ground for this new generation of drones.
The potential benefits of integrating LIDAR-equipped drones into a mine’s workflow are clear. “Mapping a stope with a drone takes only a few minutes, depending on the size,” says Hrabar. “With a typical stope, you could fly in, fly down, fly up and out again in five minutes or less.”
That offers significant time advantages over traditional surveying techniques, such as the Cavity Monitoring System (CMS), which involves an operator manually manoeuvring a boom-based laser scanning head to peer into voids and cavities.
“With a CMS-type boom, getting access to the area to scan is very difficult because it might be closed off with mesh or they might not have ventilation in that part of the mine,” says Hrabar, adding that ventilation work is usually required to provide access.
“You can do away with all those problems with the drone because you launch from a safe location and then fly around or over other obstacles that would get in the way of a CMS boom.” The primary benefit, therefore, is keeping people safe.
A secondary safety benefit is the new insights that can be derived. “You can have a better understanding of what the conditions are underground and see issues such as over breaks that can cause problems,” says Hrabar.
“With a typical stope, you could fly in, fly down, fly up and out again in five minutes or less.” Dr Stefan Hrabar, Emesent
Introducing autonomy
The recent game-changer for LIDAR drones and other robotic plant equipment has been improved levels of autonomy. Firstly, traditional line-of-sight control – based on humans on the joysticks watching the drone – has been augmented to ease the burden on pilots.
“Hovermap forms a bubble around the drone to keep it safe and provide stability, even without GPS,” says Hrabar. “If you take your hands off the controls – even if you’re underground – it stays very stable.
“And that makes a huge difference because, without that, you have to have expert piloting skills – you’re adjusting your controls all the time so it can be very stressful.”
This capability, which Hovermap had at launch, proved valuable underground in terms of flying line-of-sight in a safe, controlled and stress-free way, but the distance it could fly into a stope or down a drive was limited.
To bridge that gap, Emesent extended its research to develop fully autonomous technology capable of flying drones well beyond an operator’s vision. For this beyond-line-of-sight capability, self-navigation is key.
To achieve that, Hovermap streams a low-level resolution point cloud, based on a subset of the LIDAR reflections generated as the drone flies, on to a tablet. This allows the operator to see a 3D map being created in real time.
“By tapping on the map, you can place a waypoint and the drone self-navigates to try and reach that waypoint. Importantly – because if you’re at the beginning of a stope or a drive, you don’t know what’s happening around the corner – you can place waypoints that are way outside the current bounds of the map.”
Those waypoints help guide the drone so that it self-navigates to where it needs to go – and back again – safely and within its battery capacity.
“The beyond-line-of-sight capability means you can get into areas that are otherwise impossible to reach,” says Hrabar. “It makes flying easier because it’s just a tablet – we joke that even our grandmothers could fly the drone.”
Hrabar says automation has the potential to remove people from mines completely, even with the latency times of interfaces such as 4G and Wi-Fi commonly available today.
“We did a proof of concept where I flew a drone underground in Canada from my office in Brisbane,” he says. “We just used remote standard net collaboration tools to control the drone because you don’t need a low-latency link.”
Use data effectively
Flying in a mine environment, even for a few minutes, can generate huge amounts of data. It’s important to find a balance between what’s needed as an immediate priority and what can be saved on-board for analysis at a later time.
“When we fly in these environments, we typically get very dense data and that can be sub-sampled,” Hrabar says, citing an example of a surveyor who is trying to calculate the volume of a stope and doesn’t immediately need access to full high-resolution data to do this.
“Maybe a point every 10cm is sufficient, but if engineers are trying to map geotechnical structures, they need as much detail as possible, so the full resolution data is always available.” The resolution is reduced so that it’s usable and the full resolution data is then stored on-board automatically.
Products like Hovermap are standalone devices that can interface with a variety of drones and robots, but what obstacles exist in a world of different standards? “It’s definitely a challenge,” admits Hrabar.
“We started off interfacing with Da-Jiang Innovations (DJI) drones because they were the market leader and they had a software development kit (SDK), which basically allowed you to tap into their flight controller, to read from it and write to it.”
However, that system is not open source and replies on documentation from the supplier. “It’s also a Chinese-made system and many countries and industries don’t want to use Chinese drones,” adds Hrabar, citing potential security concerns.
As a result, the team at Emesent adapted their system’s autonomy to an open-source flight controller called Pixhawk – a system used in many drones, including the Zoe, a quadcopter manufactured by Acecore Technologies that Emesent is exploiting. “Because it’s open source, you get full access to what’s under the hood and you can adapt things,” he explains.
Look to the future
Hrabar sees huge growth potential in terms of further adapting this technology for mining landscapes. LIDAR sensors, combined with drones, he says, can be used for more than just safety and mapping.
“Our vision of the future is a mine teeming with drones and robotic systems all working in unison to continuously capture and monitor,” he says, adding that airborne drones can be used in combination with ground robots and autonomous vehicles to collectively replace frontline workforces.
“Our role is to bring that all together – teams of robots flying, driving, scanning, capturing data and bringing it all back to a central data points to be managed, then automating the insights from the data as well,” says Hrabar. The ambition is to remove humans from the actual mining loop altogether and instead use human talent and capabilities in the analysis of the data.
“We’re already working in that space,” he adds, citing the 2021 Defence Advanced Research Projects Agency (DARPA) subterranean challenge, a global robotics initiative that encourages new approaches to map, navigate and search underground environments, namely in combat operations or disaster response situations.
For the challenge – which pits the world’s top robotics companies against each other – Emesent came full circle and teamed up with the CSIRO, which provided a tank-top robot to carry Emesent’s drone to its launch site.
“From there, we would fly around and share maps in real time between the drone and ground robots,” says Hrabar, adding that the exercise also featured robotic ‘walking dogs’ capable of carrying sensors, building maps and sharing data too. “So that’s definitely a little vision of the future.”
This article originally appeared in World Mining Frontiers winter 2021.
