29 October 2014

Oh the uncertainty, how do we cope?

Researchers cannot escape uncertainty
Uncertainty plays a paradoxical role in research. It is both the starting point of research and the ending point.
Uncertainty is uncomfortable for many people and will generally give rise to varying levels of uncertainty-related anxiety
The positive aspect of uncertainty is that it drives the human need to know and so motivates research.

The resulting evidence provides knowledge which is used to guide actions, practice and policy. It is disconcerting therefore, to have to acknowledge that research is imperfect, and subseequent actions may be wrong.

Why uncertainty remains

Research may help reduce uncertainty, but it fails to eliminate uncertainty for two reasons.

The first is the problem of induction which has been with us at least since antiquity. Pyrronhic sceptic, Sextus Empiricus and the philosopher, David Hume both recognised that observation provides insufficient justification for inferences about unobserved instances.

As noted by philosopher, C.D. Broad ""inductive reasoning [is] the glory of science [and] the scandal of philosophy."

In the 20th century, Karl Popper's idea of falsification became a popular response to the induction problem. In this view, we can only disprove notions, but never prove anything. While it is logically appealing, it does not help satisfy the need of researcher and humans in general to know what is true rather than that which is false.

Another approach to the problem is to at least parse out the the errors contributing to uncertainty. In general, it is acknowledged that there are two types of errors that we might make. The first is to claim a truth which proves to be false, a false alarm, a false positive or a Type I error in the language of statisticians. The second is to declare false something that proves to be true. This is a miss, a false negative, or a Type II error.

The problem that faces induction is essentially that of making a false alarm. That an inference will be drawn that turns out to be false.

In this context, statistical science in particular seeks to quantify this risk typically expressed as the p-value. Scientists then make an arbitrary decision to reject the false alarm risk if it is less than say, 5%.

The problem that falsification addresses is the Type II error. It is a conservative rule showing only what we do not know. In effect, falsification seeks to minimise misses, Type II errors.

Researchers it appears, cannot escape uncertainty.

How do we cope with uncertainty?

One approach is to deny the uncertainty, to act as if the eureka moment is true. However, overconfidence does not eliminate the uncertainty as incorrect theories, conclusions and claims based on research often reveal. Sometimes even the most famous get it very wrong as Mario Livio details in his book, Brilliant Blunders.

Another approach is to accept that there is doubt about what is true, being careful to distinguish doubt from denial. The confusion of the two is seen in the common use of the word sceptic as a denier of the research (e.g., climate-change sceptic). 

However, a sceptic in the philosophical sense of the word acknowledges that what is true is uncertain. Scepticism is a factor that limits confidence as revealed in T.H. Huxley's definition of agnosticism: "In matters of the intellect do not pretend that conclusions are certain which are not demonstrated or demonstrable."

For instance, in any modelling of future weather be it tomorrow, next week, or two decades hence, it must be acknowledged that there is doubt about what will happen. However, a researcher expressing such a view about future climate projections in the current environment is very likely to be howled down by those that dogmatically divide the world into believers and deniers.

Claiming evidence-based knowledge and uncertainty simultaneously is a tough position for the researcher to hold, but arguably a very important one. For this reason, Elizabeth Pisani and Michael Crichton have both observed that while research feeds policy, there is much danger when the two become entangled, and in particular, when researchers become political. 

More practical people, those calling for action can be especially intolerant of uncertainty and may seek to simply dismiss it. A role for researchers is to remind everyone that knowledge is uncertain.   

A good researcher will maintain a degree of scepticism according to philosopher Le Morvan who describes "the doubtful scientist" and "the humble scholar" as prototypes of "healthy scepticism". 

The third option is resignation and despair. This however does not solve the problem of uncertainty. Rather, it simply returns us to the observation that uncertainty is unsettling for many.

Uncertainty is unsettling. Researchers help in the search for truth, but must acknowledge the truth of residual uncertainty as well. Uncertainty is important for discouraging hubris among the advocates for action.

For researchers, uncertainty is a motivating force. And perhaps the fact that no research is final and uncertainty always remains ensures that there is always plenty more work to do.

No comments:

Post a Comment