BIAS & FORENSICS
Whereas other human activities are dominated by fashions, fads, and personalities; science is supposed to be constrained by agreed rules of procedure, rigorous tests... (and) is an impersonal, dispassionate and thoroughly objective enterprise.
This is, of course, manifest nonsense.
Paul Davies - Director: BEYOND - Center for Fundamental Concepts in Science
Introduction to Richard P. Feynman’s Six Easy Pieces: Essentials of Physics
For three decades I have been a forensic engineering investigator; at times called an expert witness. Unlike the fight against criminal law breakers by the heroes and heroines portrayed on TV and in the movies, forensic scientists and engineers most often operate in the calm and relatively slow-moving world of civil law. It involves working alongside, and in opposition to, experts in other academic fields – chemists, physicists, and statisticians - in the search for clues and facts to establish [in my case] why and how a material, structure, or product failed in service.
Counter-intuitively civil forensics can be more complex than criminal forensics. Criminal law often only requires the ostensibly clear-cut task of “matching” one pattern to another: where evidence collected from both the suspect and the crime scene are compared. Patterns may also be available from a data base (a fingerprint, a DNA sequence, or a facial likeness); developed by tests performed under known conditions (blood splatter patterns, fire spread patterns, characteristic skid marks, marks on a bullet, or even decaying biological matter); or based on data established from past cases. The science and engineering of civil forensics is much more broad-based.
The term forensic science, or forensic engineering, both refer to the application of science and technology to legal matters. The terms have also come to mean the specialist investigation of any matter that occurred in the past, and even to estimating the unfolding of future events.
Discovering how an event played out in the past involves three procedures: 1] the observation and analysis of objects as they currently exist - after the event; 2] the simulation of how the event may have played out; and 3] if feasible, the analysis of the information generated while the event was actually happening. From the combination of all three procedures (ideally) an opinion is abductively formed which is then subjected to critiques by experienced scientific and legal minds. As a consequence one’s ideas and understandings are continually under inspection for their accuracy, logic and rationality.
These processes create a passionate need for precision. But they also show the sharp differences between the professions of engineering and research science. While the professions appear to be closely related there are significant and important differences which can influence each at a philosophical level.
The priority for engineers is the practical aspects of design, human safety, and property security. The priority for research scientists can be summed up as producing knowledge or discovering “facts”. One profession builds while the other explores. Engineers live in a world of mostly existing data; scientists in a world of finding data; and both operate in an arena where complete knowledge is almost always unattainable. To get around this substantial problem each must make one or more assumptions.
Due to its practical nature engineering assumptions can use straightforward tests to establish probabilities on the facts of a matter and determine appropriate degrees of safety and durability. Engineers can readily tell if their work is below standard: parts may not fit together properly; they are not sufficiently stiff or strong; materials age prematurely or cost too much to build; or the client is dissatisfied with the product for other practical reasons.
The unfortunate scientist, in pursuit of facts and truth in a complex world, has little or no opportunity for making such assessments. A scientist cannot predict probabilities regarding a complex experiment when the exact conditions have never been previously encountered. The best that can be done is that tests correlate to the assumptions; or the work is supported by other scientists - who likely made the same set of assumptions as the original researcher.
The 2013 report by Kassina, Drorb, and Kukucka entitled The Forensic Confirmation Bias: Problems, Perspectives, and Proposed Solutions was highly critical of forensic scientific evidence. It said: the forensic sciences are subject to contextual bias and fraught with error. It continued: … context can taint people's perceptions, judgments, and behaviors. … Forensic Confirmation Biases … corrupt lay witness perceptions and memories as well as the judgments of experts in various domains of forensic science.
A further example of the difficulties with science was highlighted by Dr. P. A. Ioannidis in his 2005 report Why Most Published Research Findings Are False where he demonstrated statistically that 98% of scientific papers have errors (this report being based upon statistics, and not assumptions, would likely not cause it to fall so far short of the truth). His work also analysed some earlier broadly accepted research findings and found that 32% were subsequently contradicted (16%) or shown to have effects that were smaller than originally determined (16%).
To clearly and unambiguously address the human biases in the assumptions that are evidently driving these mistakes a number of straightforward procedures could be instigated for scientific research. A voluntary solution is suggested for all complex scientific work, especially those with a social or cultural goal. As part of an appendix to any report the author(s) should be encouraged to include a disclosure statement. This statement should detail the steps taken to ensure the research was carried out impartially. It would allow a well-informed reader to establish the work’s impartiality.
This proposed voluntary “impartiality statement” should address the effects of the known human biases and paradigms (as discussed in this site). The following points, based upon the findings of philosophers of science refined over the five hundred-year history of modern science, would elevate the reputation of science to many serious thinkers. These items should be provided:
1. Details of the research carried out, if any, which was specifically designed to disprove the hypothesis or conclusion.
2. Details of independent replication work, if any, that was carried out for all or any part of the work.
3. Details of the assumptions, paradigms, or assertions of truth, upon which the conclusions rely – other than the foundational assumptions of science [e.g.: There are consistent laws of science throughout the universe; humans are capable of understanding them; and they have not changed over time].
4. A list of the anomalies discovered during the research work; along with their possible relevance and alternative conclusions;
5. The sources or details of the boundary conditions and variables used in all supporting mathematical models.
6. The sources of funds for the research. If the funds, in whole or in part, were directly or indirectly received from a public source, all the raw data shall be readily available to the public; and not treated as the intellectual property of any research entity or person.
7. Names and contact information of the formal peer reviewers.
8. A list of other verifiable steps taken to ensure research impartiality.
9. An affidavit regarding the following:
o that all data collected has been disclosed as recorded; with data not used in the conclusions clearly identified;
o identical standards of proof and logic have been applied for all processes required to formulate the conclusions;
o the abstract, discussions, and conclusions of the report are fully supported by the data and findings.
The above disclosure would allow an informed user to consider its impartiality and dependability. To the extent that a report fails to comply with this logical voluntary procedure it should be interpreted by a critical thinker as a direct acknowledgement that its conclusions will contain errors or omissions; and that the conclusions should be interpreted as paradigm affected conjectures.