Yitu Technology won first place in prestigious global facial recognition benchmark test

SILICON VALLEY, Calif., Sept. 5, 2017 /PRNewswire/ -- In the recently released results of Face Recognition Vendor Test, a prestigious international benchmark, Yitu Technology, a company from China, won first place in four different categories, overtaking the Russia company Vocord, a long-established international security brand that beat Google in various facial recognition benchmarks for years, and Morpho, a well-known French biometrics company.

Face Recognition Vendor Test (FRVT), is a benchmark organized by National Institute of Standards and Technology (NIST), which serves as the official guideline for U.S. government purchases. The test results are the golden standards of global security industry in practice.

The test datasets of FRVT are all of real-world scenarios provided by the U.S. Department of Homeland Security, e.g. massive number of photographs taken by border control agencies and in criminal investigations. Compared to other popular tests setup by academia, such as LFW and MegaFace, FRVT is more practical and at a larger scale. Not only is the test dataset scale in the order of millions, but also it is not available to the public, thus, results in effective prevention against algorithmic over-fitting or any manner of unfair practices.

Table 1: Comparison on the test scenarios, datasets and key factors of facial recognition.

Table 2: Comparison on the results of renowned brands on three test datasets (Hikvision did not provide data)

Labeled Faces in the Wild (LFW), is collected and maintained by the University of Massachusetts. he test dataset scale is in the order of ten thousand. A number of companies from China, including Baidu, Tencent, Dahua, Megvii, and SenseTime, achieved results of over 99.5% accuracy.

MegaFace is a public facial dataset collected and maintained by the University of Washington, which is 100 times larger than LFW and more capable of distinguishing performance differences of algorithms developed by participants.

The long-established Russia company Vocord won first place, with 91.7% accuracy, which is 8% ahead of Tencent, which ranked second.

In the highest class of competition, the advantages in training dataset and computations do not guarantee the best algorithmic performance.

FRVT is organized by NIST

The test results have the highest credibility, which originate from the rigorous and comprehensive.

1. Test datasets are from real-world scenarios, meaning good results imply immediate straight deployment of the technology.
2. The dataset is in the order of 1 million out of 10 billion possible samples.
   the evaluation processed with such a large-scale test dataset gives consistent results on algorithm performance, thus distinguishes the leading companies in the field.
3. The test dataset is not available to the public (blind test), thus fairness is assured.
   Due to the unavailability of the test dataset, the participants are not able to use them as a training set, thus over-fitting, or any other unfair advantages were effectively avoided. Therefore, FRVT reflected fair results over LFW and MegaFace.
4. FRVT adapts professional and objective approaches. FRVT provides detailed evaluations of recognition performance, such as, the speed of the models, the size of characteristics databases, comparative thresholds, etc. on different areas like background, ethnicity, age and gender.

According to the latest official report, Yitu ranked No.1

The result is 95.5% accurate, subjected to the false match rate of ten-millionth (based on 10 billion samples). This is 2% higher than the Russia company, Vocord, whose accuracy is only 93.5%.

This is the ever-best result from the most prestigious public evaluation based on the large scale dataset, which shows a significant increase from 2014 when NEC, a Japanese company, ranked first.

FRVT evaluates recognition performance in details concerning different scenarios, ethnicities, genders and ages. As described in the official FRVT report, its test dataset covers real-world scenarios such as border control, ID passes and city-level security.

Yitu ranked No.1 in four major scenarios.

The algorithm developed by Yitu was consistent in its performance in detailed ethnicities and gender tests. However, as the difficulty increases, the algorithms submitted by its rivals showed a  drop in recognition accuracy. The results justify the adaptability of Yitu's algorithm on different datasets of various real-world scenarios. For example, Yitu's algorithm performed well despite current issues with darker skin tones.

The report as of June 19, 2017: 
https://www.nist.gov/sites/default/files/documents/2017/06/20/frvt_report_2017_06_19.pdf

The unbelievable increase of Artificial Intelligence

Over the last for years the key testing factors and false match rates have dropped from one-thousandth to ten-millionth. The accuracy has increased ten thousand times. It could be foreseen that recognition performance comparison subject to the key test factor of false match rates of thousand-millionth, one-billionth, or even ten-billionth will emerge very soon.

This would be another AI recognition power growth of thousand times.

We can imagine that high performance facial recognition based on super-large-scaled comparison, as large as one in one-billion, with super-low false match rate of one-billionth, will unleash this technology to wider range of daily settings.

Facial recognition at all levels, from a single building, to a city, or metropolitan areas, to even a whole nations, is becoming possible.

For more information contact:

Li Jingyi 
[email protected]

Han Yang
+86-021-6151-7311 
[email protected]

SOURCE YITU TECH