QPAM: Uncertainty

A first form of uncertainty is randomness. It is a stochastic behaviour that can be dealt with sensitivity analysis, estimates from experience (actuarial) or hedging.

A more complicated form of uncertainty is indeterminacy. It describes situations that are qualitatively known, but cannot be reliably quantified. It is often addressed by attempting to quantify it anyway , using heuristics or stylised facts.

Another form of uncertainty is based on reductionism. Reductionism arises when a complex system is not completely understood and proxy relationships are established. It is a form of epistemological uncertainty and often addressed with lay knowledge (and bringing in lay people) and mixed methods (quantitative and qualitative) .

Yet another form of uncertainty is paradigmatic. Expert knowledge can narrow perspectives and neglects the unseen. Consequently, paradigmatic blind spots arise which can only be dealt with by interdisciplinary co-production of knowledge and staying curious.

The last form of uncertainty is based on unknown relations. This arises when something has not happened before (e.g. how cyber crimes work was unimaginable 30 years ago). It can be summed up as ontological uncertainty. It can only be addressed with humility and the ability to adapt.

Type III errors

Uncertainty may also arise from committing errors. More commonly known are these errors:

  • Type I: False positive, reject null hypothesis when true
  • Type II: False negative, accept null hypothesis when false

An additional third type of error can be summed up as the correct answer to the wrong question. These errors usually arise by using the wrong method (i.e. model design) or use a discipline specific approach (i.e. context) to a non-applicable field (e.g. it was tested whether rats die from heroin-laced water when they could also choose normal, which they did and it was concluded that addiction was so strong that it would make them kill themselves. Follow-up studies showed that when they have other rats and entertainment around rats don’t kill themselves on heroin. So the original research actually answered the question whether rats would commit suicide when being alone and without entertainment).

In the worst case it is used intentionally to distract from a real problem by a form of mental bait-and-switch.

Framing

Yet another source of uncertainty is the frame in which a discussion takes place. Describing a problem often circumscribes the solution. It determines what kind of methods and options are open for debate. It recasts a subjective reality as “objective”. It is a unusual field for engineers and natural scientists who assume an objective reality (e.g. physics). Any issue that comes up for policy analysis has most likely been framed before it is handed to analysts and scientists to process. For instance, economic growth is a usual assumption that cannot be challenged by any solution proposed.

Value conflict resolution

Another source of uncertainty is that value conflicts need to be resolved. Previously mentioned was the problem space. Any solution is essentially political and will always be a negotiation of social forces. It is not typically an academic field and is often dealing with red lines (deeply vs. weakly held values), shifting from why to how (it solves the problem), procedural vs. substantive fairness, obfuscated players (grassroots vs. astroturfing) and it is often a space for missing issues to be attached. Academics are usually hidden players that get called in after the fact to compare minor differences.