Adversarial Attacks And The False Promise of Biometrics

A touch of your finger to unlock your phone, a glance of your face to open a door, staring into an iris scanner to get into a vault: biometrics seem to be everywhere these days. But as use of the technology spreads, so are efforts to circumvent it – not necessarily by bad actors, but by researchers bent on revealing its weakness.

Biometrics is based on the fact that our faces, irises and fingerprints are almost always unique between individuals. But the artificial intelligence used to recognize that uniqueness is remarkably fragile and easily confused. The vulnerability has spawned a whole field of research into “adversarial attacks” of machine learning models, revealing flaws and proposing solutions. But as AI improves, so do the adversarial attacks designed to defeat them. The result is a kind of race between developers and their academic attackers meant to harden AI systems for real world applications.

For years, hackers have demonstrated that face, fingerprint, iris and even vein scanners can be fooled with facsimiles, whether it be a photograph of a face, or a rubber cast of a fingerprint. Those attacks succeed largely because of what experts in the field call the necessary ‘smudge factor.’ Scanners need to allow some tolerance in what they see in order for the devices to work in poor light or with sweaty fingers.

But more sophisticated attacks use machine learning to target the machine-learning algorithms that make biometric scanners work. One of the most common methods confuses the pattern recognition systems behind the scanners by introducing hidden signals to what the scanner sees. Researchers have long known that changing just a few pixels in an image can make a computer vision system identify a school bus as an ostrich, for example. In the real world, stickers on a stop sign can make a computer vision system read it as a forty-mile-an-hour sign.

The same method can be applied to face recognition cameras. A group of researchers at Carnegie Mellon University has shown how printed patterns on eyeglass frames can fool face-recognition systems into identifying the wearer as a celebrity - John Malkovich, for example. A group of Chinese researchers has used tiny infrared lights attached to the underside of a hat brim to fool another face-recognition system into identifying an Asian male as the Caucasian musician, Moby. Similar tricks can foil voice recognition systems and handwriting recognition systems.

Given the vulnerability of vision systems there is increasing interest in identifying a person by their gait – the cadence of their walk, which is as unique as faces or fingerprints – using the gyroscopes and accelerometers embedded in most smartphones. But researchers at the University of Michigan have shown that those sensors can be perturbed with high frequency sound that makes the algorithms misread the gait and misidentify the person. So alarming was the Michigan study, the National Cybersecurity and Communications Integration Center issued a warning to companies that employ accelerometer sensors.

One early use of adversarial research was in techniques to circumvent spam filters. But the field really took off in 2014 amid the spread of deep learning – multilayer networks of mathematical formulas that learn to recognize patterns in data. Christian Szegedy, a Google researcher, led a team that showed such networks could be fooled into misidentifying images by making small changes to the images that were imperceptible to the human eye. That superhuman-performance algorithms could be fooled even with imperceptible noise was alarming to people even outside the research community.

As deep learning moved into security critical applications, such as autonomous vehicles and biometrics, the field of adversarial research exploded. Much of the research proposes methods to protect against such attacks, but adversarial examples continue to multiply.

Because so much code is published openly, such academic papers risk becoming a kind of roadmap for malicious users. One group of researchers has even proposed making sensitive code subject to a standard licensing agreement that would allow legitimate researchers access while keeping it away from the public.

There are two kinds of attacks: so-called white box attacks, in which the attacker knows the algorithms and architecture of the system being attacked, and so-called black box attacks in which the attacker knows nothing about the internal workings of the system. Because adversarial attacks against one system often succeed against another, black-box attackers build their own system and design an attack against it that can then be applied to the system the attacker knows nothing about.

Another kind of attack ‘poisons’ the data on which deep learning networks are trained. In 2016, Microsoft introduced a chatbot dubbed Tay that would learn from interactions with users on Twitter. But mischievous users quickly poisoned Tay’s training data by feeding it racist and otherwise offensive sentences. Tay was pulled from the market. Since then, chatbots and virtual assistants are trained in controlled settings and respond with set phrases, called pre-programmed intents.

Most consumer-facing fingerprint-based authentication systems use small sensors that read only part of the finger and compare the partial scan against partial records. The small sensors on smartphones and other devices never stitch those partial prints into a whole. Several people can record their fingerprints on such a device and it will unlock with any one of their partial prints.

Because many fingerprints share common features, someone trying to pick such a finger-print-protected biometric lock need only match a partial fingerprint in order to be granted access. In 2017, A group of researchers did just that, demonstrating that many fingerprints share enough common features to open a large percentage of biometric devices, such as iPhones. They called those fingerprints MasterPrints.

Most recently, researchers at New York University’s Tandon School of Engineering used a machine learning process to generate virtual fingerprints that combine features common to the fingers of large segments of the population. They called the fake fingerprints DeepMasterPrints.

“We used artificial evolution to create fingerprints that would match as many people's fingerprints as possible,” said Julian Togelius, one of the researchers who conducted the study. He said he and his colleagues generated fingerprints that could each can authorize 22 percent of all users, or in other words, unlock the devices of 22 percent of all users. The success rate is high enough that a selection of fake fingerprints could theoretically unlock most iPhones on the five tries that the phones allow.

Mr. Togelius said the DeepMasterPrints “can bypass all the authentication methods we know of.” The same machine-learning technique could theoretically be used to generate faces with enough common features to fool face recognition cameras.

Lars Ericson, a program manager in the Office of the Director of National Intelligence said biometric devices will have to increase their sensor size to defend against such attacks. But larger sensors and tighter tolerances will mean higher failure rates and far less convenience.

Many security experts say that consumer-level biometric scanners are simply not secure.

“It's a convenience tool, it's not a real serious security tool,” said Ben Schlab, a security consultant at Germany’s Security Research Labs. He added that PINs and passwords, kept securely, are still the safest way to protect access to devices and information

Indeed, most fingerprint-protected devices require a password or PIN when they are first started or after a period of inactivity. But once unlocked, many sensitive smartphone applications  – the money transfer app, Venmo, for example – can be activated with fingerprint authentication.

As interconnected digital devices proliferate, identity authentication is becoming increasingly critical. The answer might be systems that combine multiple features to authenticate identity – face recognition, gait and other behavioral markers, for example. It is much more difficult to devise an attack that can correctly spoof multiple features at the same time. To make such multilayered systems convenient, companies are looking at passive authentication that does not require action by the user.

A Canadian startup makes floor tiles that can recognize gait. Other startups such as UnifyID build authentication systems that combine motion and behavioral factors with signals from the user's environment, such as the Bluetooth emissions from a user's Fitbit.

Such multilayer systems raise the cost and complexity of any adversarial attack, but won’t necessarily eliminate them. “Machine learning itself can be the weakest link in the security chain,” wrote Battista Biggio, a researcher at Italy’s PRA Labs at Italy’s University of Cagliari, in a paper on the vulnerabilities of such systems.