TL;DR (“Too long; didn’t read”)
A bowtie diagram is primarily a qualitative tool for visualising risk management quantitative, a simplified approach to scoring is more in line with classical risk management techniques. Once adopted, a numerical approach shows the relative risk from specific triggers and highlights why some controls are more critical than others.
The purpose of a bowtie diagram
Bowtie diagrams are generally used as a qualitative risk management model. They illustrate potential causes and effects of a single high-impact event (the Top Event, usually meaning loss of control of a major hazard). This is the core purpose of our SharePoint diagramming tool, Bowtie Designer.
At the next level of detail, they also capture the controls put in place to prevent the occurrence or mitigate the potentially bad outcomes of that loss of control scenario.
So a bowtie diagram is a graphical tool in the first place. It expresses the relationship between threats and the possible downstream consequences of a severe incident, should those threats materialise. As suggested by its name, its graphical appearance resembles a bow tie, with many threats and outcomes connected through a single loss of control event.
Focus point 1: In the first instance, the Bowtie Method (bowtie diagrams) are about listing hazard management mechanisms rather than measuring them.
How much use is a qualitative model?
In a complex process, whether in an industrial or organisational context, any kind of risk analysis has to begin by listing areas of concern. That is a precursor to further refinement and understanding. Detailed analysis benefits from structured thinking. The first ‘win’ in the bowtie method is clarity of approach, and the value of this is doubled again, if you are clear what you wish to achieve from the thought process.
Focus point 2: Bowtie analysis helps to organise your thoughts, prior to more detailed analysis
Having identified a severe loss of control scenario, it’s a matter of preference whether to continue by looking at the causes or effects. Getting down to the specifics of either may well challenge your original understanding or definition the bowtie diagram’s Top Event.
For example, a natural gas escape (and subsequent ignition) should be expected to have different outcomes, depending on whether it is at a remote site with a low density of people, or close to a population centre, or in a busy industrial facility. Even though the technology aspects may be similar, almost identical, it could lead to entirely different worst-case consequences.
Quantitative bowtie analysis
As soon as we acknowledge this principal, then a quantitative measure has crept in. A threat to a large population must be worse than the same threat to a smaller number of people. Up to a point, people intuitively recognise the difference in scale (or frequency) of threats. This is a good thing, because they can be very hard to measure with any certainty. Lack of certainty is, after all, the definition of risk.
However, as your bowtie diagram grows in size and detail, picking out high-frequency, high-risk elements becomes essential in order to prioritise areas for attention. All approaches to risk management sooner or later come to require such prioritisation. Even when precise statistics are not available, there is a need to estimate scale of risk the very least. Bowtie diagrams are no exception.
Focus point 3: Sooner or later you will want to quantify risk, even if approximately.
What to quantify?
Comparing the basic inputs and outputs of a bowtie diagram with a classic risk analysis matrix, the question of what to quantify is relatively simple.
In a risk matrix, the likelihood of an event forms the y-axis, and the severity becomes the x-axis. The equivalent measures in the bowtie diagram are, for causes on the left (trigger events) – likelihood. For outcomes on the right (consequences) – severity.
Considering the relative probability of the different triggers is a good starting point. In this way, you may prioritise your treatment of controls which reduce the likelihood of the most common trigger events from causing loss of control
If a trigger event were expected to occur once every three months, and our timeframe was daily, then that would be roughly once per 90 days, i.e. about 10-2 if taken as a daily frequency or 1% as a daily probability.
Focus point 4: Numerical ratings enable prioritisation, but also suggest the overall likelihood of the Top Event occurring.
At the next level, being slightly more mathematical about it, the probability of the bowtie method Top Event is the combination of all the possible triggers. The probability of one or other triggers occurring is the simple sum of probabilities, if they are mutually exclusive. If not mutually exclusive, this might slightly overestimate the combined probability, if we were being exact.
What’s most useful is the ability to combine the likelihood of all the triggers into a single number. This can then be shown on a risk matrix.
A different approach makes sense for outcomes on the right hand side of the bowtie diagram, because of the problem of “comparing apples with pears”. Many organisations find it more productive to keep different categories of outcomes separate. For example, mapping safety risks on a scale apart from, say, financial impacts.
Focus point 5: Outcomes are harder to equate because of differences in their nature, but they can be compared across different bowtie analyses.
If we accept that different categories of outcomes should be kept apart, then what is the value of quantifying them? To answer this, we need to consider the next level of detail, which is controls.
Quantifying effectiveness of controls
Central to the bowtie method, is illustrating the controls which reduce the likelihood of an event occurring. Not all controls are equally effective. For example, having an alarm to indicate a problem may be less effective than having an interlock which prevents the problem state from occurring. However, having both types of control in place insures against either of them failing.
If the effectiveness of controls is expressed as “fails once in a hundred times” or “once in a thousand”, then not only can their relative effectiveness be understood, but they can be rolled up into the single number which indicates the likelihood of a trigger event turning into loss of control. Or, similarly, of loss of control leading to a harmful outcome. Bowtie Designer uses this kind of approach, which you can get a flavour of from this 3-minute video:
We go into more detail about the specifics of estimated values in our post about Measuring Left Hand Side Risks in a Bowtie Diagram.
Another example of evaluating safety risks on a logarithmic scale was used in the ARMS SIRA Excel tool, which applied a single rolled-up number to express the value of all controls on the left and right-hand sides of the bowtie diagram.
Moreover, because the same controls can often be applied to multiple triggers or outcomes, then the overall risk picture may be influenced not only by the effectiveness of the control, but also by the number of places it is employed and the relative frequency of the triggers (or severity of outcomes) that it affects.
Focus point 6: Once you have rated the effectiveness of controls, then the usefulness (or criticality) of each control can be expressed across the whole bowtie model.
We can extend this idea as a “sensitivity model” by considering the effect on the bowtie model as a whole, if each control is made more or less effective.
How to quantify effectiveness?
As stated earlier, a numerical model should be treated carefully because of the large amount of uncertainty around many bowtie scenarios. Namely, lack of data and sensitivity to actual conditions. This can add large error margins to any statistical model. Therefore, we avoid over-investing in exact numbers.
The approach we favour, similar to that used in the ARMS SIRA Excel tool, is a logarithmic scale for frequency of events and effectiveness of controls. That is to say, using powers of 10 to indicate how often an event may occur, or how often a control may fail. To say that a control works in 10% or 1% or 0.1% of cases (or almost never) is enough to extract value from rating it, without becoming bogged down in a numbers game.
What are key learning points?
- In the first instance, bowtie diagrams are about listing hazard management mechanisms
- Bowtie analysis helps to organise your thoughts, prior to more detailed analysis
- Sooner or later you will want to quantify risk, even if approximately
- Numerical ratings enable prioritisation, but also suggest the overall likelihood of the Top Event occurring
- Outcomes are harder to equate because of differences in their nature, but they can be compared across different bowtie analyses
- Once you have rated the effectiveness of controls, then the usefulness (or criticality) of each control can be expressed across the whole bowtie model
Is a quantitative approach truly justified?
Throughout this article, we have emphasised the importance of the bowtie diagram as a visual representation of risk management arrangements. And enforcing a purely qualitative approach at the outset will help to focus on the comprehensive picture. However, when addressing the question of how and where to allocate resources towards reducing effective risk, a numerical approach is not only justified but is also to be expected, in line with classical risk management approaches.
Not only does Bowtie Designer allow this level of numerical analysis, but it also provides reports which support further investigation.