Downward counterfactual analysis is a concept in psychology that reimagines history to be contrary to what actually happened. In this exclusive interview, Dr Gordon Woo of RMS talks to us on how this can be useful to risk modelling by expanding the data available based on what could have happened.
There have been surprises that affected the insurance industry over the past decade or so, when something unusual happens that leads to a major loss. “Often the CAT modellers are blamed, for producing an insufficient model, or being told that ‘they should have known about this’. Typically, whatever has happened, you could have gotten some idea about in history; not what happened but what might have happened,” said Dr Woo.
In statistical analysis, historical data is usually treated as fixed rather than one possible version of many that could have occurred if various influencing factors had been different.
This can be a weakness in risk modelling. For example, in the case of modelling rare extreme events the lack of loss data may give a false picture of the actual threat level, which could have been distorted by near misses and good fortune.
He used an Air Canada ‘near miss’ as an example, an event that took place on 7 July 2017 at San Francisco International Airport, which could have ended up being the worst disaster in civil aviation history. “If the pilot hadn’t managed to adjust in time, aviation premium prices would have gone up, because aviation underwriting is driven by supply and demand, not so much what the risk is. But as nothing happened, everyone carried on their normal day, the rates were not affected because no loss had happened,” he said.
In defence of the aviation industry, Dr Woo said that the safety of airlines has been improving, with 2017 being the safest year in aviation history. “So, it’s not unreasonable for the rates to come down, but human error is human error. But the idea of having parallel loss books – one for your real losses and another for your virtual losses – does make sense.”
He promoted using downward counterfactual analysis on these near misses and on past events to set a better benchmark for risk pricing and risk modelling. By expanding the conceivability of a risk, Dr Woo argues, future risks might be averted.
In a report commissioned by Lloyd’s, he used the Fukushima disaster in 2011, caused by the magnitude-9 Tohoku earthquake as an example: “The maximum credible earthquake had been thought to be only M8.2 (Ruff and Kanamori, 1980). The significant underestimate of the maximum possible tsunami height led to the release of radioactivity from the Fukushima nuclear plant.”
It can also help mitigate the inbuilt bias inherent in some risk models that are based on a single standard dataset. Although catastrophe risk modellers work independently of each other and take different views on many topics where risk ambiguity exists, there can be bias in the data if there are substantial common elements in the risk modelling for a particular peril.
A report written by Dr Woo, commissioned by Lloyd’s states:
“Counterfactual risk analysis can be applied to any set of risks but is particularly useful for those that have limited loss history. It could help create structured, transparent, scientific and evidence-led scenarios for non-modelled risks (especially emerging risks) for which there is not much data and could be used to validate traditional probabilistic natural catastrophe modelling.
Counterfactual risk analysis helps address the bias that can be inherent in some models that are based on the same data sets. By expanding the data available based on what could have happened, these models can be built with less reliance on single-source data, which might improve their accuracy.”
In the report, Dr Woo also said that counterfactual analysis could also help (re)insurers communicate future risks and model uncertainty to board members, policyholders, policymakers and risk managers, as well as non-experts, as downward counterfactual examples are always based on actual historical experience.
For catastrophe risk quantification, counterfactual risk analysis can be applied in all three core catastrophe modelling activities of a P&C (re)insurer, namely pricing, capacity management and capital calibration.
Introduced in October 2017, downward counterfactual analysis has been picking up traction amongst insurers. “The reception has been very positive. It’s an idea that has begun to snowball, even though people are not used to thinking like this,” he said.