Tam Le is the Regional Associate Strategy Director for Carat APAC.
This article explores the implications of facial recognition, both for advertising and for retail, as well as for future privacy rights.
7-10 minute read
Remember that scene in Minority Report where the protagonist, played by Tom Cruise, walks through a corridor of fully personalised brand advertisements from companies like Lexus and American Express and then ducks into a Gap where the virtual sales assistant greets him by name whilst inquiring about his previous purchases?
Well despite the film being set in 2054, 37 years in the future, we’re actually not far off from this reality today. Google’s FaceNet claims >99% accuracy in facial recognition; Facebook’s DeepFace, >97% accuracy . And when you stop and think about how many pictures are uploaded of you onto Facebook (I’ve been on for over a decade and I’m not camera shy), the implications are overwhelming.
It is these implications of facial recognition that I want to explore in this article, both for advertising and for retail, as well as for future privacy rights.
ADVERTISING: I SEE YOU LOOKING AT ME
The first level of facial recognition (in order to gain public acceptance and normalize the idea) is to target messages and ads based on anonymized data like demographics (age, gender, etc.) or consumer behaviour.
Posterscope designed and executed a campaign for the GMC Acadia mid-size SUV that for the first time, linked responsive facial recognition technology to dynamic displays that presented personalized content in an out-of-home campaign . The digital screens were fitted with video sensors and partner Quividi’s audience and context aware platform that anonymously detected and determined whether a passing shopper was a man or woman, alone or with a group, adult or child or even frowning or smiling. Once detection was made, the digital screens were populated with video content and brand messaging tailored to the identified audience. No data or images of any type were collected, stored or shared at any time, ensuring privacy.
In another facial recognition application, Women’s Aid was able to use the technology in a powerful way that underscored their message about domestic violence. Their digital billboards displayed an image of a bruised woman and recognized when people were paying attention to the ad. As more people looked at the ad, her bruises and cuts healed faster, communicating that if people took notice, domestic abuse could be halted .
In the future, as facial recognition in OOH becomes more publicly accepted, the displays can start collecting and storing more data on people (what display they pass, at what time, whether they go to a store shortly after viewing an OOH display with a strong sales message, etc.) and building profiles with a history of behaviours.
Fused with the data we will have of people’s online behaviours, this starts creating a more complete overview of the consumer journey, addressing the common concern of not being able to tie offline sales data with online ad impressions. This will also bring us closer to a singular universal ID for each person, which will contain their complete profile and track and retarget them both online and off.
RETAIL: I’M WHOEVER YOU WANT ME TO BE
Outside of pure advertising advantages, facial recognition would also have many implications for the retail experience as our physical and digital worlds merge.
KFC, in partnership with Baidu, is already testing out personalization based on the data it gathers through facial recognition at one of its restaurants in Beijing . Installed image recognition hardware scans customer faces, detects mood, gender and age and recommends menu items accordingly. For example, as Baidu claimed in a press release, the system would tell “a male customer in his early 20s” to order “a set meal of crispy chicken hamburger, roasted chicken wings and Coke for lunch,” while “a female customer in her 50s” would get a recommendation of “porridge and soybean milk for breakfast.” The setup also has built-in recognition to provide better services for returning customers: it can “remember” their order history and suggest past favorites.
In my utopian view of the future, all McDonald’s would have a centralized global database of their customers’ faces so they could remember, in any country, that I’m going to order either the Big Mac meal with an extra order of fries or the Double Cheeseburger meal if I’m just looking for a snack. Wouldn’t it be great if McDonald’s tracked me from childhood and suggested new menu items as my taste buds developed over time? Additionally, they would know when I reached major life milestones, like becoming a mother, once I start ordering Happy Meals to go with my Big Macs. They could literally see me grow up in front of their very own image recognition hardware, like some kind of distant relative that provides me with fast food.
In an even more maternal move, today the Luce X2 TouchTV vending machine can not only greet you by name and remember your past purchases, but it can also, with access to your medical records, deny you certain items, like a sugary snack if you have diabetes or items that may contain nuts if you have an allergy . The system could also be connected to a retailer’s loyalty points system or linked to the room numbers in a hotel. The opportunities for receiving personalized service from this once faceless, impersonal machine, are endless.
Let’s apply this to a larger retail space, say a supermarket of the future. Building off the concept of Tesco’s virtual subway stores in South Korea, we could use facial recognition to have the store “shelves” customize themselves for each person in a way that would make customers more likely to purchase and purchase more. I’m more likely to buy certain brands, items, and categories than you are, and vice versa. How can retailers exploit this and get the both of us to buy more?
For example, if I stepped up to a digital shelf in the milk “aisle”, it would only show me soy, coconut, or almond milk made with very few ingredients (the only types of milks I buy) whereas it could show someone else baby formula. But then the retailer also knows I have a sweet tooth so it would show me some cookies that would go great with milk, which I may not have bought in a physical store because the cookies are placed far from the milk.
Additionally, brands could pay to be in the prime front-and-center shelf space (which they do in physical stores), but do it in a cost-effective way, like only paying for a prime position on target consumers’ virtual shelves, instead of wasting money through prime placement on everyone’s shelves.
Facial recognition would also have many implications for the branding industry. With virtual shelves, brands can also cost effectively A/B test new packaging design in-situ and at scale to determine each option’s effectiveness or alter the claims shown on packaging per individual to tailor their message. This could mark the end of one-design-fits-all.
On top of all of these advances, facial recognition also offers the benefit of frictionless payment, like Alibaba’s Smile to Pay technology. And as we know from technological advances of the past, such as moving from cash to credit card, the further we move away from exchanging physical goods and the easier we make it to pay, the more people will spend.
You’re probably now wondering, if these new innovations allow for greater and greater marketing effectiveness, and it seems like we already have the technology to implement these ideas, what’s stopping us from getting to this advertisers’ utopia?
PRIVACY RIGHTS: YOU’RE CREEPING ME OUT
“We recognize the creepy, but we don’t want to stifle innovation. If we cross that line from cool to creepy, people will stop using that service,” recognizes Carl Szabo, a lawyer with NetChoice, a tech industry group that represents companies like Facebook, Google, and Yahoo .
I recognize not everyone is as excited as I am at the prospect of McDonald’s recognizing you and watching you grow up or store shelves rearranging themselves when you start approaching. In fact, even with today’s relatively limited use of facial recognition, a backlash is already starting.A couple examples of this come from artist and technologist Adam Harvey, who has designed both face camouflage and anti-surveillance clothing. The face camouflage, called computer vision dazzle (or CV dazzle), uses a strategic application of paint and hair-styling to throw-off patterns that facial recognition algorithms look for, such as the degree of light and dark in the cheekbones, or the way color is distributed on the nose bridge—a baseline amount of symmetry . When CV dazzle is executed properly, it transforms a face into a mess of unremarkable pixels causing a momentary burst of confusion for the computer, allowing the wearer to go undetected. The anti-surveillance clothing, dubbed the Hyperface project, involves printing patterns on to clothing or textiles, which then appear to have multiple eyes, mouths and other features that a computer can interpret as a face, overwhelming facial recognition systems by presenting them with thousands of false hits so they can’t tell which faces are real .
As evidenced by these self-iniatives, many consumers don’t want to be tracked by cameras and their images sold by corporations like Google and Facebook to advertisers. The path to societal acceptance of facial recognition technology will have to be slow and transparent. Szabo, the lawyer with NetChoice  admits that “legislation cannot move at the speed of innovation,” and suggests companies make their facial recognition policies hyper transparent and explicit so that consumers can “vote with their feet” if they are “creeped out.”
“If we did start recognizing people en masse then I think outdoor would have the same problem that digital display is having with people getting fed up and install ad blockers. You can’t do something just because the technology is there, you have to be led by the consumer and not the technology,” says Chris Pelekanou, commercial director at Clear Channel, one of the world’s largest out-of-home advertising business . This is something we as marketers must keep in mind as advertising is most effective when people embrace it.
This article was developed with much help from Ben Milne, Head of Innovation at Posterscope.