When are FAR and FRR values statistically significant?
A value is considered statistically significant when it is likely that is falls within a given error interval and the probability of falling outside this area by chance is relatively low. Statistical significance is dependent upon the number of trials or sample size. Because biometric values are difficult to model, the existence of statistical significance is hard to estimate. As a rule of thumb (“Doddington’s rule”), one must conduct enough tests that a minimum of 30 erroneous cases occur [Porter 1977]. Example: An FAR of 10-6 can be considered reliable, when 30 errors occur in 30 million trials. One error in a million trials also has an FAR of 10-6, but statistically is far less significant. One can see that biometric tests are very expensive if performance needs to be very high. The situation would be easier, if further information could be considered along with the yes/no questions (or accept/reject), as for example the proximity of a decision to the acceptance threshold.
