AR games

Augmented reality (AR) is the future and it’s going to come quite soon,” Peter W Singer says. It’s an opinion that is not far from the truth, if you believe recent publicity. In late 2018, parts of the press were awash with stories about AR and its future potential use within the defence sphere. In particular, stories opined the benefits it could offer soldiers in ‘boots on the ground’ scenarios.

Singer is a strategist at New America and awardwinning author of numerous defence titles, including Wired for War: The Robotics Revolution and 21st Century Conflict and most recently LikeWar: The Weaponization of Social Media. His fictional Ghost Fleet: A Novel of the Next World War considered how AR technology – from commercially available products such as Google Glass to a system the Israeli military was, at the time, developing for its infantry – might be used in the future battlespace.

Today, we are familiar with commercial products like Google Glass and even Pokémon Go, where the real world is overlaid with images and data only visible using the device. However, these devices continue to advance as commercial players become more accustomed to what they can do and learn how to further exploit their capability. Soon, instead of users having to retrieve information, it could be given to them without the need to look for it.

“You could be out for a walk, and as you’re out things might be pushed to your lens – advertisements, perhaps. We know their [lenses’] utility, providing a more effective way for people to receive and digest information than having to look down at their tablet or smartphone.”

Such possibilities could significantly alter the way soldiers operate. “Augmented reality presents the potential for individuals and teams to be engaged, simultaneously, in a physical and an information space,” says Professor Anthony Finkelstein. “It has transformative potential for applications in defence and security.” Finkelstein is the UK’s chief scientific adviser for national security, as well as being professor of software systems engineering at University College London and working with The Alan Turing Institute as part of its defence and security research programme.

The art of AR

The development of such technologies has long been a challenge, as Finkelstein highlights. “First, how do you construct devices that are sufficiently light, energy efficient, robust, accurate and so forth; and second, how to build user interactions that make the most of the technology but are also ergonomic, physically and cognitively?”

The arguably controversial US Land Warrior programme has been one of the most publicised in future soldier development. Started in the 1990s, it aimed to develop an integrated fighting system that comprised a heads-up display (HUD) helmet, communications, and a power supply.

Despite funding issues, the programme did result in some technology being deployed in Iraq in 2007, and in Afghanistan in 2009. Unfortunately, much of the feedback was critical, with soldiers suggesting the equipment was prone to failure and that even when it did work, it was cumbersome. The programme was later superseded by the Ground Soldier System.

“There remains a journey to full practicality. Some of this journey will entail working out how we design and develop software for such devices, but mostly it will be about the slow process of addressing the engineering and interaction issues,” says Finkelstein.

The controversy surrounding the Land Warrior system has not, however, deterred the US Army from its aims. In early 2018, it embarked on the next phase of that journey, saying it was just 18 months away from testing the HUD 3.0, the latest innovation in visual devices for combat and training. Speaking at the Association of the United States (AUSA) Global Symposium 2018 in Alabama, US, Brigadier General Christopher Donahue said the headset was already proving its potential, improving marksmanship thanks to the advanced crosshairs linked to the weapon being used. This allowed the soldier to see where their bullet would land even before they had fired it.

As the year drew to a close the US Army announced it had agreed a $480 million contract with Microsoft to develop AR system prototypes with the hope of ultimately supplying 100,000 units. In a press statement the company said, “Augmented reality technology will provide troops with more and better information to make decisions. This new work extends our long-standing, trusted relationship with the [US] Department of Defense to this new area.”

1990s
Land Warrior programme is started by the US Army.

The news was met with stinging criticism from some employees of the company, who demanded it withdraw from the contract. Writing to Microsoft’s CEO and president, the group said the move would further distance soldiers from “the grim stakes of war and the reality of bloodshed”. Such moral concerns are valid, says Singer, but they have always been there, right from the introduction of the rifle, which was better equipped to shoot at distance than the musket.

2007
Despite funding issues, some Land Warrior equipment is deployed in Iraq.

“I wrote a book on drone warfare and its impact,” he adds. “There is a similar parallel of watching a screen – does that make it easier to take that decision to shoot? Data showed such things still have psychological consequences. Yes, there is that video game parallel, but we were seeing the same levels of combat stress as if operative were in the field.” He argues the risk of lessening the impact of war is something policymakers, rather than those on active service, should consider. “I feel that they treat these decisions differently,” he says.

2009
A US Army brigade outfitted with Land Warrior equipment is deployed in Afghanistan.

Challenges and limitations

Nevertheless, AR presents a raft of challenges for those using it in combat situations, one being what Singer calls the “data overload factor”. He says, “This was the challenge with the very early uses of AR in military tests. They found people would focus too much on the projected data and not what they were actually seeing. It was a foreground issue, your eyes drawn to that data when you should have been looking at the wider world.” Aside from missing potential threats, it was causing physical effects such as stomach upsets and car sickness-like symptoms.

2018
US Army begin investing in HUD 3.0, and agree deal with Microsoft to develop AR system prototypes.

The Microsoft project, known as Integrated Visual Augmentation System (IVAS), is the latest iteration of the HUD 3.0, and was officially unveiled in April 2019. The prototype will provide soldiers with digital cameras, integrated 3D displays, a ballistic laser and hearing protection. The system – which, it is believed, will not be mounted – will provide a plethora of data to the user.

2019
Microsoft unveils the Integrated Visual Augmentation System (IVAS), with the intention to supply over 100,000 of them.

Speaking more generally of these types of systems, Singer said, “We have to think about all the different forms of data you get and how that changes not just the way you see the world, but how you interact with the people in it.” He painted a mental image of a soldier in an Afghan village and how such data might help them. “It might be that I’m in a village and somebody speaks to me in their language. That could feed into an auto translate with 90% or so accuracy. It might be that type of thing that changes our interactions. Then you move to facial recognition; a person comes into your vision and it might provide a label as to who they are.”

However, Singer warns that this is where having access to a lot of data brings certain challenges with it. “You and I might meet, and it labels me ‘Peter Singer’. You’ll accept that I am him when maybe I’m not,” Singer continues. “When it goes wrong you are in a position where the machine has told you something and you’ve trusted, and maybe acted, on it.”

A further concern is the risk of security breaches. Given the level of information these devices might provide troops on operations, the threat of having that data infiltrated, tampered with, distorted, blocked, or even used against you is very real. “With blocking, for example, if you’ve got people that have become reliant on these devices in order to operate and they lose it, that could have a kind of paralysing effect. As for stealing these devices, if someone else can see what you’re seeing – well, you can already think of all the tactical and strategic threats that can come out of that. And if someone is able to penetrate the networks, they can inject false information – anything from believing you’re in one place when you’re actually in another.”

Due to these concerns, and the risk of troops suffering data overload that may make it difficult to operate or alter their judgement, it is imperative training is adapted. Singer believes in order to avoid soldiers having a “warped view of the world” they will need to be trained to properly process the data they receive to use it to their advantage.

Potential in its future

Despite this, he is encouraged by what this technology could offer. “Think about the benefits of knowing everyone’s face, who they’re related to, their network. Then when they talk you can understand what they’re saying. Now we can even start to know if someone is telling the truth by their micro expressions. This type of ability might offer superior decision-making.”

That is for the future, says Finkelstein, perhaps three to five years away. He sees AR offering different benefits right now, “Combat may well be a key application but the immediate promise is for tasks such as the maintenance of complex systems and, fairly obviously, training.”