Apple Banks on Augmented Reality To Reignite Smartphone Market
Smartphone makers have been getting anxious in the past couple of years. People have been hanging on to their phones for longer, meaning slower sales in China, the US and Europe. They need something new to reignite the market, and now Apple thinks it has found the elixir to persuade everyone to buy new phones again.
The company is pinning its hopes on "augmented reality" or AR, which uses a combination of the phone's camera and other sensors to enhance what it shows on the screen -- to show, on your phone or tablet's screen, things that aren't there but which you might like to see.
It might be a flying Harry Potter as you cross a London bridge or, at home, a new kitchen design complete with virtual lighting and curtains; or a highlight showing which screw to use as you assemble that new Ikea wardrobe. A forecast from Digi-Capital, a mergers and acquisitions adviser working in the sector, predicts a billion people will be using AR by 2021, and that the global market will be worth $60bn (£47.4bn).
Before you have a market, however, you need someone to write the software, and the big tech companies are aware of that.
Apple last week wowed its annual conference for developers -- the thousands of people who write apps for the iPhone and iPad -- with new software frameworks to build AR apps.
Craig Federighi, the company's head of software, showed off a pre-release app. He pointed an iPhone at a table and positioned a virtual cup of coffee with moving steam on it, then added a lit lamp and a vase, and moved them around to show the movement of virtual shadows cast by the nonexistent lamp. As he did so he moved around the table. The camera showed the live audience in the background, and the virtual objects on the table, all moving smoothly.
In another demonstration, he showed a Lego app that offered an exploded view of a model, and a virtual battle played out on a table. Ikea is already working on apps using the software, so the end of the lost screw may be in sight.
Federighi boasted that Apple's next big software update this autumn, which will incorporate AR functions, will mean that the company will become "the largest AR platform in the world" overnight, able to run on "hundreds of millions of iPhones and iPads."
In fact, AR is already big business, and mostly on mobile. Millions of people brandished their smartphones at the empty air last summer to play Pokemon Go, a mobile-only app that was downloaded more than 100m times and generated more than $500m in revenues in just two months, according to the research firm App Annie.
Pokemon Go was about "seeing" virtual animals on a screen and Nintendo turned it into a moneyspinner with in-app purchases.
Lego also already uses AR in stores. Show a box to a CCTV set and the camera identifies the box and projects the finished model, floating eerily above the physical box.
Then there's Snapchat, the photo-sharing social network run by Snap, which recently listed on the US Nasdaq, and whose users adore the ability to swap faces or give themselves giant eyes in photos on their smartphones. That's AR too -- even if it doesn't advertise itself as such -- and has helped Snap to a stockmarket valuation of more than $21bn.
Snapchat also illustrates a key difference from the "virtual reality" (VR) touted by everyone from Facebook to Sony. VR requires bulky strap-on helmets that project a different world onto screens directly in front of the eyes, and block everything else out. They make the user look strange, and isolate them from others.
AR, by contrast, is meant to be integrated into the world in front of you, and is visible to anyone who can see the screen. Tim Merel, the chief executive of Digi-Capital, reckons that 10% of spending on AR could be purely within business contexts, but also that "mobile AR will be adopted by the young. Platforms that can scale mobile AR with 13 to 29-year-olds will win."
The potential benefits of offering AR have attracted all the big companies in mobile, including Google and Facebook, and many smaller players - the most notable being Magic Leap, a US company that has already received more than $1bn in venture funding.
Mark Zuckerberg told Facebook developers in April that AR will "make our physical reality better," though he was short on detail apart from a camera effects function which looked suspiciously like Snap's.
Google has its "Tango" project, but only two phones, which sell in very limited numbers, are currently capable of running it. Microsoft, despite having no presence in mobile phones, is showing off its Hololens AR system, aimed at developers, in London this week.
But where's the money going to be spent? "The bulk of mobile AR revenue could be in sectors other than games," says Merel. "Over 80% of mobile AR revenue could come from mobile network data, e-commerce sales, advertising, consumer [non-games] apps and enterprise/business-to-business sales."
Mobile networks will welcome any app that means users have to buy pricier mobile data plans, and smartphone companies will be happy if people realise they need a new phone for the hot new trend.
In some respects AR looks like a gold rush, though it's one we've seen many times down the years. Layar, founded in 2009 and based in the Netherlands, works with brands such as Pepsi and BMW on mobile apps that add data to real-world scenes. We're already familiar with it from televised sports events - the flight of the ball in cricket replays, where tennis serves land at Wimbledon and green contours in golf.
What's changing is that smartphones are now powerful enough to do the processing in real time. It might start with a coffee cup, but AR could change the way we see and interact with the world -- even if it does still mean staring at a screen a bit more.
© 2017 Guardian Web under contract with NewsEdge/Acquire Media. All rights reserved.