The need for research
41.1. A number of witnesses spoke of the requirement for further research.
41.2. Professor Champod said that there was a paucity of structured research into fingerprints probably because fingerprint comparison had been understood to be "a profession ruled by certainty". Appreciation that the process involved uncertainty with grey areas such as the impact of distortion and varied levels of tolerance necessitated research into all aspects relating to the interpretation of fingerprints.1
41.3. Mr Nelson agreed that more research is needed.2 He acknowledged that a great deal more requires to be done in trying to understand phenomena such as double touch and movement and pressure.3 Scientific research is required to provide proper standards for people operating the non-numeric approach in assessing, for example, quality of characteristics.4
41.4. Mr Rennison referred to a need to be clearer as to how the decision making processes work, including matters such as contextual bias,5 and for research to underpin better interpretation models that allow professionals room to use their judgment, but within a very clear framework.6
41.5. The need for research was identified by the OIG in its report on the Mayfield case7 and the June 2011 review provides an update on the research that has been undertaken. That includes research on the permanence and reproducibility of friction ridge detail (including third level detail) and research on the accuracy and consensus of decisions; and work is on-going to develop software to assist fingerprint examiners to assess the quality of marks and to measure the sufficiency of corresponding features to support a conclusion.8
41.6. It is appreciated that this Report is written at a time of economic uncertainty, and in circumstances where resources are limited in the public sector. Mr Nelson indicated that SPSA does not have a budget sufficient to fund all the necessary research.9 Mr Nelson said that there was an opportunity for research in Scotland through the Scottish Institute of Policing Research developed by ACPOS and the Scottish universities10 and the SPSA had signed a memorandum of understanding with Strathclyde University.11 Within the UK Mr Rennison has already identified the lack of any coherent strategy and has made representations to the Home Office about this12 and it is evident that areas of research identified by this Inquiry are similar to those identified in the United States for example by the OIG and others.13 There is value in international co-operation in such research and sharing of results so as to avoid unnecessary duplication.
41.7. Research is not confined to the methodology of fingerprint comparison and decision making. There is a need to review the presentation of evidence in court, including, for example, trials of electronic methods of presentation.14 The failure to use digital images was one of the criticisms made by the Court of Appeal in R v Peter Kenneth Smith.15 Having seen electronic methods in use over many days of evidence at the Inquiry I would encourage work in this area.
The role of statistics
41.8. One particular research initiative merits more detailed discussion and that is the potential to apply statistics not only at the stage when an examiner is carrying out his comparison but also as a basis for presenting evidence in court.
41.9. Professor Champod and Mr Chamberlain argue that a probabilistic premise underlies the inferential judgment that the presence of a certain combination of features can justify a conclusion of 'uniqueness'.16 The examiner is reasoning that the probability of finding the same combination of characteristics in any person other than the one being identified is zero.
41.10. In current practice the probabilistic premise can be said to be at most 'implicit' because it is not expressly stated by an examiner when expressing a conclusion and the examiner may be unaware that he is applying such reasoning.17 Indeed, since examiners do not use statistics or any probability computation it may be misleading to characterise their conclusion as 'probabilistic'. It is more accurately described as an empirical, subjective judgment based on training and experience.18 As Mr Pugh expressed it, examiners are using pattern recognition skills rather than thinking about probability.19
41.11. Examiners are using their training and experience to assess the rarity of the combination of characteristics but there is scope for differences in opinion in part because of differing levels of experience. One examiner may treat a combination of characteristics as rare, as he may seldom have come across such a combination. Another examiner may have come across such a combination more often. Examiners presently have insufficient objective evidence by which decisions as to the rarity of characteristics are assessed, and to the extent that such data is available, it is not utilised by examiners. The National Academy of Sciences has noted: "In most forensic science disciplines, no studies have been conducted of large populations to establish the uniqueness of marks or features. Yet, despite the lack of a statistical foundation, examiners make probabilistic claims based on their experience. A statistical framework that allows quantification of these claims is greatly needed."20
41.12. Mr Pugh and Miss Hall suggested that there might be a role to use statistical frequencies or evaluation to supplement or support the opinion of the fingerprint examiner and even to help them arrive at the conclusion.21 Use of statistics could also provide an objective measure to enable the court or the jury to understand the examiner's intuition or experience that a particular characteristic or coincident sequence was rare.22
41.13. Some statistical data is already available on the prevalence of level 1 features and level 2 characteristics.23
Probabilistic analysis
41.14. Mr Rennison found it odd that fingerprint examiners were expected to give an absolute yes or no answer and that there was no room for doubt. His view was the aim for the future should be to base opinions on probabilities. Probabilistic analysis works in other areas, such as DNA, and people had grown to understand it.24
41.15. In 1980 the International Association for Identification adopted a resolution stating that friction ridge identifications are positive and officially opposing evidence based on possible, probable or likely identifications.25 In 2010 the Association passed a resolution laying a foundation for validated mathematical models to be used to support an examiner's opinion but not as the sole determinant of identity.26
41.16. Probabilistic models for fingerprint work were, at the time of the Inquiry hearings, under development27 and one developed by FSS was in the process of validation.28 Probabilistic analysis expresses the chance that a mark may have been left by a person.29
41.17. Professor Champod explained how probabilistic models work in fingerprints in his report to the Inquiry30 and in his oral evidence. The models invite consideration of two specific questions:
(i) the probability of observing the degree of similarity (or dissimilarity) between two impressions if they come from the same source; and
(ii) the probability of observing this degree of similarity (or dissimilarity) if the two impressions come from different sources.31
The ratio between the two probabilities is the likelihood ratio.
41.18. To answer question (i), the model has to encapsulate knowledge about distortion, to account for the possibility that the mark may be distorted, smudged, or moved. To answer question (ii), a database of prints taken from different people is required, as a response to this question is obtained from the statistical analysis of the distribution of characteristics on fingers coming from different sources.32
41.19. The likelihood ratio does not allow an expert to say either by absolute terms or by degree whether the mark was left by a specific person.33 It does, though, enable him to assign a weight to his conclusion.34 Mr Chamberlain indicated that the likelihood ratio is derived from the precise configuration of characteristics. The likelihood ratio for different configurations of the same number of characteristics varies because one configuration may be less common than the other.35
41.20. Probabilistic modelling has its limitations. Professor Champod, an advocate for more statistical input to fingerprint decision making, cautions that statistical modelling is not the holy grail.36 It will not eliminate the role of the examiner: "the extraordinary power of the human eye-brain combination has to be recognised and examiners will remain best placed for designating the features available on a mark."37
41.21. The model is dependent on the inputs by the examiner who must designate respectively on the mark and on the print the features that he considers to be in agreement. The model does not extract these automatically38 nor does it pick up differences that the examiner may have overlooked. The model generally assumes the truth of the existence of the input data and will not detect errors of observation, hence Professor Champod stressed that the model will not replace a proper application of ACE-V.39 Professor Champod provided an illustration of the dependence of the model on the skill and judgment of the examiner under reference to a specific mark and print.40 If an examiner were to conclude that there were ten points in agreement in that example the model would produce a likelihood ratio in the order of 300,000, which would be "very powerful evidence to support the view that the mark has been left by the same person as the person who produced the print." If a second examiner were to conclude that there were only five points in agreement in the same example the likelihood ratio would fall to an estimated five, which would provide "some evidence" but of quite a different degree.41
41.22. The Metropolitan Police expressed some scepticism as to the value of such models. All proposed models that use a biometric system are not as accurate as adequately trained examiners.42 They cannot assess marks holistically like the human brain.43
41.23. The use of these models could potentially achieve two different objectives. The first is to move decision making away from arbitrary, personal thresholds to a "statistical data driven approach". The second is to give the opportunity to introduce more fingerprint evidence by placing some evidential value on marks that are currently classed as inconclusive, insufficient or 'no value', including marks with a relatively low number of features.44 Different considerations apply to these two objectives.
41.24. The underlying studies to gain knowledge about distortion and the distribution of characteristics on fingers coming from different sources could bring a layer of systematic study to guide examiners' decision making and give them back-up data to suggest how rare a configuration may be.45 By providing statistical data probabilistic analysis could also provide the basis for moving away from personal experience based thresholds46 and provide a mathematical measure of weight as an additional piece of information to be used in the evaluation stage by the examiner47 and in due course by the court or jury.
41.25. The second objective reflects the suspicion that some potentially useful evidence either for court or police intelligence48 may be being lost under the current system49 but the Metropolitan Police expressed doubt whether any substantial volume of useful evidence is being lost and supported the call for research in this area.50
41.26. The models can be applied to marks with a number of minutiae (i.e. level 2 characteristics) as low as three51 but the Metropolitan Police were concerned that the results in cases which examiners currently consider to be inconclusive could be potentially misleading and this needed careful consideration.52 As Mr Pugh explained it, the concern is that a low number of characteristics might suggest a high probability of a match when a difference somewhere else in the mark means that it is actually an exclusion.53
41.27. This problem is most acute when the mark is a relatively small part of the print, the concern being that the part reproduced in the mark may contain the similarities with the differences, lying in some other area, not being reproduced. That scenario in fact puts to the test the second premise of (conventional, as opposed to probabilistic) friction ridge identification as presented by Ashbaugh, which includes the proposition that: "...friction ridges are unique in a very small area due to the shape, alignment, and relative pore location of the connected ridge units."54
41.28. That prompts the question: how small an area can produce a reliable identification? Reference to Ashbaugh's book does not supply an answer beyond indicating that it depends on the clarity of the mark: "... when discussing the friction skin, it can be said that the friction skin is unique in a very small area. This statement, however, only applies to a friction skin print if clarity is present. When clarity is absent it may be an incorrect statement."55 Later he states: "clarity may also affect the size of the area of friction ridges required to individualize."56 No finite answer can be given relative to conventional fingerprint identification, it being yet another matter calling for the subjective judgment of examiners. Those who are developing probabilistic modelling are conducting research on the point. The application of likelihood ratios to partial prints is being studied by the University of Lausanne and the National Forensic Laboratory of the Netherlands.57
41.29. Mr Geddes referred to many instances worldwide where marks with clearly defined detail have been identified on as few as three or four characteristics in combination with level 3 detail such as pores and the width of ridges.58 Neither the fact that probabilistic analysis may be based on only a small part of a print, nor that it may utilise as few as three level 2 characteristics is a point of distinction between conventional fingerprint identification and probabilistic analysis. In each case, of course, the critical question is the reliability of the method as applied in the comparison being undertaken. In the case of probabilistic modelling that will doubtless be an issue to be considered in the validation of the method.
Commentary
41.30. Research into the statistical basis for fingerprint identification is to be encouraged. Probabilistic analysis should also continue to be developed.
41.31. The more controversial issue relates to the application of statistics in general and probabilistic analysis in particular. Probabilistic analysis could be used in one of two ways.
41.32. The first would be to provide background data to assist fingerprint examiners with their evaluation of marks and to enable them to express the strength of their conclusion in a transparent and verifiable manner. The conclusion on identity or exclusion would continue to be based on the skill and experience of the examiner but at least the examiner would be able to call upon the support of statistics to aid verification of the conclusion and assist with the articulation of reasons in support of it.59
41.33. The second use would be the possible application of probabilistic analysis to comparisons that examiners would otherwise consider to be inconclusive, the objective being to produce evidence that could be used in court of the probability of a match. The reliability of the use of probabilistic modelling in this second way requires careful validation.
41.34. The limitations of statistics and probability models must be appreciated. As Professor Champod said, they are not the holy grail. The output of the mathematical models is no more reliable than the inputs and there is force in the observation by Mr Pugh that applied to complex marks with very few features probabilistic modelling could produce a "veneer of robustness": for example, supporting an identification which ought properly to be an exclusion if account is taken of differences overlooked by the examiner.60 The evidence heard by the Inquiry demonstrates that there is variability amongst examiners regarding the inputs (i.e. variability as to both the observation and the interpretation of characteristics) and that will have a bearing on the 'accuracy' of the statistical output. The errors made in Y7 and QI2 Ross related to the finding of a minimum of 16 'matching' points identified by the SCRO examiners and the failure to provide a cogent explanation for differences. Those errors would not have been picked up by the application of probabilistic analysis and, indeed, the application of that analysis may have exacerbated the problem because the presentation of the result in terms of a statistical likelihood ratio would have cloaked the evidence in the veneer of scientific accuracy. Probabilistic modelling requires proper controls as to the inputs, including strict adherence to ACE-V methodology,61 with particular emphasis on the enhanced verification of complex marks (i.e. a technical review) to select the points on which the examiners have reached consensus.62
Awareness of significant developments in fingerprint law and practice
41.35. In chapter 3863 it has already been observed that lessons can be learned from cases of mistaken identification.
41.36. In 1998, prior to the trial in HMA v McKie, Mr Dunbar and Mr Mackenzie had had experience in the McNamee case of the possibility of differences of opinion among fingerprint examiners resulting in a court being unable to reaching a safe conclusion on identification. Experience in the McNamee case should have led to an appreciation that an examiner cannot be 100% certain that another competent examiner comparing the same impressions will necessarily come to the same conclusion; and, accordingly, the SCRO examiners would require to prepare in detail to justify their conclusion on Y7 when it was challenged. When the existence of a defence challenge to the identification of Y7 was made known to Mr Stewart he sought guidance within SCRO but due to a combination of circumstances he received none and the SCRO examiners were left to their own devices, as if the scenario were unique, which it was not.64 It would appear that there had been no dissemination within SCRO of the experience of Mr Dunbar and Mr Mackenzie in the McNamee case or, in any event, lessons were not learned from that experience.
41.37. The Mayfield case also showed that '100% certainty' was unsustainable and offered insight into a number of weaknesses in fingerprint methodology at the time when Scotland was moving to the non-numeric system. Mr Innes, Head of the Scottish Fingerprint Service (the predecessor to SPSA) until April 2007, said that he distributed an article about the Mayfield case in 2005.65 The OIG report is dated January 2006 and preceded the move to the non-numeric system in Scotland in September 2006.66 It was not discussed between the Scottish Fingerprint Service and COPFS in the lead-up to that change and, indeed, Mrs Tierney was unaware of it until early 2007.67 The findings of the OIG regarding deficiencies in the practices of the FBI were not being taken into account in formulating operating procedures for SPSA.68 The Mayfield case had not been the subject of any detailed discussion by SPSA's Scientific Advisory Group.69 Again the opportunity to learn lessons of relevance to practice in Scotland was missed.
41.38. From the evidence of Mrs Tierney and Mr Pattison, it appears that no-one in either SPSA or COPFS was tasked with maintaining awareness of cases or inquiries in which fingerprints were disputed or in which courts or investigating bodies made findings which might be relevant to the development of fingerprint practice.70
41.39. Investigation of 'erroneous' identifications may afford an insight into weaknesses in the methodology of fingerprint identification, and not just practitioner error. SPSA should monitor for reports of such investigations. This is an additional reason for maintaining close links with the academic community working in this field.71
41.40. It is understandable that COPFS should rely to some extent on SPSA, or in the past, its predecessor bodies, to advise on developments in the specialist fields for which they are responsible. However, COPFS has historically taken a legitimate interest in the reliability of fingerprint evidence and for the future I would expect not only SPSA but also COPFS to take note of decided cases (such as McNamee) or official reports (such as the OIG report) that have relevance to the presentation of fingerprint evidence in court.
Recommendations
Research and development
41.41. Requirements for research and development should be identified and collated and an appropriate scheme of research and development prepared by SPSA. Appropriate arrangements and funding (when available) should then be provided to enable research and development to take place on a UK basis and where appropriate through co-operation on an international basis.
41.42. Specific matters to be included in the scheme of research and development include:
(i) the matters covered by the recommendations that have been made in chapter 35 paragraph 134 and chapter 39 paragraph 23; and
(ii) the use of data as to the frequency of particular characteristics or combinations of characteristics as a means of assisting examiners in their work.
41.43. The use of probabilistic analysis should continue to be developed.
41.44. The SPSA should keep its practices under review in the light of developments in research.
Familiarity with developments and disputes in fingerprint law and practice
41.45. The SPSA should task identified staff with:
(i) maintaining up-to-date knowledge of cases, at least in anglophone jurisdictions, in which fingerprint evidence has been disputed; and
(ii) monitoring cases in which courts, inquiries or other investigating bodies have made significant criticism of existing fingerprint practice and of considering whether those criticisms should be taken into account in developing and improving fingerprint practice in Scotland.
Such staff should be tasked also with advising an identified contact in COPFS as to these matters.
41.46. COPFS should task identified staff with maintaining up-to-date knowledge of cases, at least in anglophone jurisdictions, in which fingerprint evidence has been disputed, and with liaising with an identified contact in SPSA in relation to any implications that the findings of such courts may have for fingerprint evidence in Scotland.
1. Professor Champod 25 November pages 113-115
2. Mr Nelson 13 November pages 12-13 and FI_0153 para 95 Inquiry Witness Statement of Mr Nelson
3. Mr Nelson 13 November page 14
4. Mr Nelson 13 November pages 17-18
5. See chapter 35 para 111ff
6. Mr Rennison 8 July page 99
7. US Department of Justice, Office of the Inspector General (2006) Review of the FBI's Handling of the Brandon Mayfield Case (Unclassified and Redacted) (US Department of Justice) URL: http://www.justice.gov/oig/special/s0601/final.pdf, pdf pages 207-208
8. US Department of Justice, Office of the Inspector General (2011) A Review of the FBI's Progress in Responding to the Recommendations in the Office of the Inspector General Report on the Fingerprint Misidentification in the Brandon Mayfield Case, URL: http://www.latent-prints.com/images/FBI%20Mayfield%20Progress%20062011.pdf, pdf pages 19-21
9. Mr Nelson 13 November pages 16-17
10. Mr Nelson 13 November pages 12-13 and FI_0153 para 95 Inquiry Witness Statement of Mr Nelson
11. Mr Nelson 13 November page 18
12. Mr Rennison 8 July pages 96-97; see chapter 40 para 26
13. Committee on Identifying the Needs of the Forensic Sciences Community, Committee on Science, Technology and Law Policy and Global Affairs, Committee on Applied and Theoretical Statistics Division on Engineering and Physical Sciences, National Research Council. Strengthening Forensic Science in the United States: A Path Forward, Washington, D.C.: National Academies Press, 2009, page 8
14. Mr Rennison 8 July pages 95-96
15. [2011] EWCA Crim 1296, para 61(viii)
16. See Champod C. and Chamberlain P. Fingerprint, in: Fraser J. and Williams R. (eds) Handbook of Forensic Science, Willan Publishing, 2009, page 69
17. Professor Champod 25 November page 83
18. See chapters 33 para 49 and 35 para 100
19. Mr Pugh 24 November page 26
20. NAS, Strengthening Forensic Science in the United States: A Path Forward, 2009, pages 188-189
21. Mr Pugh, Miss Hall 24 November pages 120-121
22. Miss Hall 24 November page 121
23. ED_0003 para 48ff
24. Mr Rennison 8 July pages 112-113
25. Professor Champod 25 November page 77
26. IAI (2010) IAI Resolution 2010-18 Passed 16 July 2010 URL: http://swgfast.org/Resources/100716_IAI_Resolution_2010-18.pdf
27. See for example EC_0001 pages 19-20, Professor Champod 25 November pages 91-92 and ED_0005 slide 75
28. FI_0136 paras 57-59 Inquiry Witness Statement of Mr Chamberlain and EC_0001 page 23
29. Mr Chamberlain 18 November page 74
30. ED_0003 para 72ff
31. Professor Champod 25 November page 88
32. Professor Champod 25 November pages 89-91 and ED_0003
33. Professor Champod 25 November pages 88-89
34. ED_0003 para 79
35. FI_0136 para 53 Inquiry Witness Statement of Mr Chamberlain
36. Professor Champod 25 November page 94
37. ED_0003 para 79
38. ED_0003 paras 86, 93
39. ED_0003 para 93
40. ED_0005 slides 77-79
41. Professor Champod 25 November pages 92-94
42. MP_0008 pdf pages 31-32
43. Mr Pugh 24 November page 111
44. FI_0136 para 55 Inquiry Witness Statement of Mr Chamberlain, Professor Champod 25 November pages 94-95 and ED_0005 slide 80
45. Professor Champod 25 November page 94
46. FI_0136 para 55 Inquiry Witness Statement of Mr Chamberlain
47. Mr Chamberlain 18 November pages 68-69
48. Mr Chamberlain 18 November page 71
49. Mr Rennison 8 July pages 112-113 and Professor Champod in ED_0003 para 28
50. MP_0008 pdf pages 32-33 and Mr Pugh 24 November page 115
51. Mr Chamberlain 18 November page 71; Neumann et al, 'Computation of likelihood rations in fingerprint identification for configurations of three minutiae', Journal of Forensic Science, November 2006, 51:1255; and Neumann et al, 'Computation of likelihood rations in fingerprint identification for configurations of any number of minutiae', Journal of Forensic Science, January 2007, 52:54
52. MP_0008 pdf page 33
53. Mr Pugh 24 November pages 112-113
54. Ashbaugh D. Quantitative-Qualitative Friction Ridge Analysis: An Introduction to Basic and Advanced Ridgeology. Boca Raton, Florida: CRC Press, 1999, page 92 (in Ashbaugh this is referred to as the third premise, in this Report it is the second: see chapter 2 paras 8-10; see also Wentworth, B. and Wilder, H.H. Personal Identification: methods for the identification of individuals, living or dead, T.G. Cooke (2nd Edition) 1932, page 325 and Mr Dunbar in CO_0050 page 2
55. Ashbaugh D. Quantitative-Qualitative Friction Ridge Analysis, 1999, page 93
56. Ashbaugh D. Quantitative-Qualitative Friction Ridge Analysis, 1999, page 94 and see also page 131
57. Professor Champod 25 November page 91
58. Mr Geddes 26 June page 126
59. Mr Pugh, Miss Hall 24 November pages 120-121
60. Mr Pugh 24 November pages 113-114
61. Mr Chamberlain 18 November pages 82-83
62. Professor Champod 25 November page 92; and chapter 39 para 14ff
63. See chapter 38 para 54
64. See chapter 11 para 90
65. FI_2410 paras 100, 103 Inquiry Witness Statement of Mr Innes
66. See chapter 33 paras 35 and 59
67. Mrs Tierney 12 November pages 51-52, Mr Pattison 13 November pages 156-158 and CO_4428 pdf pages 3-4
68. Mrs Tierney 12 November pages 80-104
69. Mrs Tierney 12 November page 55
70. Mr Pattison 13 November pages 156-158
71. See chapter 35 para 129