Font Size: A A A

Home > Security Products > Biometrics > Live facial recognition


Live facial recognition

In a few years, video-based facial recognition has gone from the realms of science fiction and Hollywood blockbuster to become a highly effective, readily deployable, real-world solution. Raphael de Cormis, VP Innovation Labs at digital security product company Gemalto, pictured, writes that live facial recognition now has the potential to offer greater security and convenience in applications that extend across the public and enterprise sectors. However, in many parts of the world, important issues such as privacy and consent still need to be fully addressed, and clear regulatory frameworks put in place to define precisely how, when and where it is acceptable for this distinctive biometric technology to be employed.

Is live facial recognition about to turn us into ‘walking ID cards’? That’s the view of at least one privacy pressure group in the UK, a country notable for the absence of any form of compulsory national identity scheme. Moreover, concerns over the implications of this fast-emerging technology are not limited to civil liberty activists. On the other side of the Atlantic, the president of Microsoft, Brad Smith, recently called for the creation of a bipartisan, congressional expert commission, to look specifically at the need for better regulation of facial recognition systems. Not surprisingly, his tone was rather more measured. However, in support of his argument he cited the ‘broad societal ramifications and potential for abuse’ that deployment could entail. Make no mistake, right around the world, the introduction of systems that can literally pick out a face in the crowd has given fresh impetus to a long running debate: how do we strike the right balance between the pursuit of greater security and adequate protection for our right to privacy? And it’s a vital conversation. In contrast to other biometric techniques, such as fingerprint and iris scanning, live facial recognition can match faces in a public space with images from a database, without the subjects ever being aware that the process is underway. Combined with reports that have called into question the accuracy of facial recognition systems in real-world applications, a reaction is both inevitable and healthy. To date, the regulators have struggled to keep pace with developments in both technology and public opinion; but there is undoubtedly broad consensus behind Brad Smith’s demand for more robust frameworks.

Before considering some of the wider political and ethical issues related to live facial recognition systems, it is perhaps worth reviewing exactly where we stand in terms of the capabilities of the technology now on the market. Specifically, what makes live facial recognition such a compelling tool, not just for law enforcement agencies, but a host of other public bodies and enterprises? What benefits can it genuinely offer all these potential operators? And why has the technology suddenly become the subject of so much attention?

Out of the lab

Until recently, live facial recognition systems were the preserve of science fiction novels and Hollywood blockbusters. However, the last few years have seen dramatic advances in the speed and accuracy with which facial images from live streams can be matched against potentially very extensive image databases. What’s more, the latest solutions do not necessarily need new video infrastructures to be implemented; in many cases, existing systems (such as CCTV) can be utilized. As a result, live facial recognition is now well and truly ready to move from the lab and into the real world.

Protecting public spaces

Given the nature of the current terrorist threat, and the deep impression it has made on both public and government psyches, it is not surprising that live facial recognition is immediately being seized on as a powerful means of helping to defend public spaces and high-profile events. But this is far from the full story in terms of how the technology can be used to enhance law enforcement and border control strategies. Moreover, there are numerous exciting opportunities for enterprises to boost the speed and convenience of customer interactions, and deliver truly individualized and personalized services. Indeed, even at these early stages of roll-out, use cases around the world serve to highlight the potential reach and scope of live facial recognition.

Exploring the commercial

In China, for example, commercial operators as well as government agencies can freely and legally harvest biometric data via live facial recognition. As a result, the country provides an insight into the avenues that could be pursued by the enterprise sector in the years ahead. Notable projects already underway include a tie up between the Beijing branch of fast food giant KFC, and Baidu, the leading Chinese search engine company. The aim is to develop a system that can predict or recommend menu options, based on the customer’s age and mood, or by recalling his or her previous choices. Similarly, Yum China, which operates KFC and several other fast food brands, has teamed up with Alipay, the mobile payments firm, to develop ‘smile to pay’ verification. Last year also saw China Southern Airlines use live facial recognition in place of traditional boarding passes.


In Europe, the operating environment is somewhat different. At the moment, the focus is more on public sector deployment, and testing both the technical efficiency of the technology and public reaction to its use. In Germany, for example, police and the Interior Ministry recently ran a trial at Berlin Südkreuz railway station, using 400 volunteers to play a serious “where is Waldo”. In the UK, a number of police forces have gone a step further, assessing the ability of live facial recognition technology to pick out genuine individuals of interest. This has encompassed several music concerts and sports fixtures, as well as the Notting Hill Carnival, the largest event of its type in Europe.

Pilot projects

In both countries, public and media reaction has been mixed. In particular, criticism has been levelled at the number of false positives registered by the systems. For example, across the events where it used live facial recognition, South Wales police force in the UK found that 91pc of ‘hits’ turned out to be perfectly innocent individuals rather than valid matches with database images. At the Notting Hill Carnival in London, the figure was 98pc.

Getting the measure

These reservations about accuracy have combined with longer-standing concerns over the right to privacy and control over personal biometric data. However, it is important to keep the headline figures being reported from initial trials and pilots in proper context. Head-to-head tests, such as those sponsored by the US Homeland Security Service and Technology Directorate, clearly demonstrate that speed and accuracy standards can vary markedly from one system supplier to another; in fact, best-in-class solutions can now achieve accuracy rates in excess of 99pc. At the same time, various factors will inevitably influence the performance of any such system, including the resolution of the video equipment utilised, environmental conditions, and the quality of images in the database which are being matched with the live feed. It’s also worth bearing in mind that even a well-trained human eye can only expect to achieve accuracy of around 85pc. Ultimately, live facial recognition should be regarded as a tool that supports the work of staff such as police and border control officers, not a replacement for them. Performance is of course a vital consideration, but so is how the information generated by live facial recognition systems is utilised. In other words, what controls are in place to ensure operators act responsibly, and can be held to account effectively by end users, citizens, and other stakeholders?

Regulatory patchwork

In many parts of the world, the answers to these questions are far from straightforward. Gaps, grey areas and inconsistencies are commonplace in current legal frameworks. In the US, the federal system of government has spawned an evolving patchwork of rules and regulations. In 47 states, it is legal to use software to identify an individual in a public space without their consent. But in Illinois and Texas, this does not extend to commercial use. And in June 2017, Washington became the third US state to pass a law formally protecting biometric data. Nationwide, NIST (National Institute of Standards and Technology) has drafted regulations for suppliers of non-commercial biometrics technology for public use.


In Europe, stakeholders also face a mixed picture of regulation. In France, there are national rules governing the use of video surveillance. So transport companies, for example, are not allowed to store information collected by such means, and there must be clear, visible notification that cameras are operating in any area protected by this technology. Going forwards, however, EU-wide regulation is set to play an increasingly influential role. Notably, the GDPR (General Data Protection Regulation), which came into force in May of this year, provides a much more coherent framework for the collection, storage and use of personal information, including biometrics. Key principles include:

Lawful, fair and transparent processing: the GDPR insists that it must be clear why data is being collected.
Purpose limitation and data minimisation: organisations should not collect any piece of data without a specific purpose – and should only store the minimum amount of data needed for that purpose. Moreover, citizens now have a ‘right to be forgotten’.
Confidentiality and security: an organisation is now solely responsible for ensuring that personal data collected is protected against negligence or malicious attack.
‘Clear affirmative action’ – all citizens must have given positive consent for data to be collected.

Security versus privacy?

To date, the debate around this technology has tended to be framed in the relatively simplistic terms outlined at the outset of this article: on one side, the right to individual privacy; on the other, the need for greater collective protection against threats such as terrorism, crime and human trafficking at the border. However, a more rounded discussion also takes into account the potential of live facial recognition to improve community cohesion and reduce the likelihood of friction between the state and citizens. In contrast to capturing biometric information via methods such as fingerprint scans, live facial recognition is completely unobtrusive. As such it can help authorities handle important issues, including how minority groups are treated, much more sensitively. Furthermore, whilst concerns about racial bias have been levelled at some facial identification systems, in principle they should never be subject to the conscious or unconscious discrimination that could potentially skew the judgement of any human tasked with live identification. In the commercial sector, meanwhile, there are clearly opportunities to engage with facial recognition in the same way we do ‘cookies’ in the online domain, agreeing to a certain level of tracking and analysis in return for a smoother, more personalised user experience.

Room for improvement
Before drawing conclusions on its overall merits, stakeholders should also recognise that it is still early days for real world deployment of live facial recognition systems. Huge strides have already been made in terms of accuracy, and it is reasonable to expect that experience in the field will only serve to continue this process of improvement.

Defining the future

Live facial recognition joins a growing list of biometric technologies that are being deployed in pursuit of greater security and convenience in our everyday lives. For the most part, the introduction of solutions such as fingerprint scanners – to unlock smartphones, authorise transactions or speed border control processes, for example – has been accepted by end users without any great degree of controversy. Indeed, if anything, consumers have proved enthusiastic adopters of a fast and safe alternative to time-consuming username/password-based procedures or lengthy airport queues. But live facial recognition is different. Fast, remote identification is a hugely powerful tool; as yet, we have merely scratched the surface of what is possible. But these very same characteristics also raise important questions, particularly in terms of how we consent to the use of our biometric data, how that information is used, and how operators are held accountable. In an ideal world, the responses would perhaps serve to frame a consistent, internationally recognised set of guidelines. In practice, it seems far more likely that different countries and regions will reach their own conclusions. The good news is that, in many parts of the world, real progress is now being made in listening to the various opinions being expressed and defining the boundaries that all stakeholders urgently need. With the technological solutions now readily available, there can be little doubt that it is this process – of designing and implementing regulations fit for the biometric age – that will ultimately play the key role in determining the future direction of live facial recognition.


Related News