A couple of weeks ago the IOSH raised the alarm. “IOSH Says More Action Needed on Preventable Deaths” said the header, “The emphasis comes after an annual rise in work-related deaths in Britain” the press release continued. Now that sounds serious, but it also triggers some questions.
Lies, damn lies and …
First question: “an annual rise in work-related deaths in Britain”?
Let’s look at what that means. Ah, well, the numbers went from 142 last year to 144 this year. In absolute numbers this is indeed a somewhat higher number. But looking at trends (which we shouldn’t do from year to year, of course) this is not a rise, it’s a fairly stable level. And the slightly higher number is most likely explainable by random fluctuations. As you may recall from a few weeks back, Norwegian road traffic fatalities went down and up again by 30, and I suppose that Norwegian road users is a smaller population that the working population in Britain!
Doing just a superficial check (Google: “fatalities Britain”), the first hit brought me to the most recent statistics on fatalities in the workplace in Great Britain 2016. You can download the report, or see them online. What do we see? There has been a steady decline in fatalities from the mid 1990s on (with ups and downs, as expected - pity that the HSE didn’t provide a rolling average) which has been levelling out the past few years. So: rise? No. Has something dramatic happened? No.
Of course every fatality is one too many. It’s a tragedy for the people involved, especially those left behind. But if you want to convey that message that then you should just say so and not wrap it in some nonsense (non-existent) trend.
Another reason to be cautious about these ‘Cry wolf’ press releases is that they may trigger simplistic interventionism which may work entirely counter-effective.
Second question: What does the adjective 'preventable' add?
It is an expression for hindsight, for sure and as that it gives me a bad taste. It says that ‘they’ (employers, employees, others?) should have known better. Some of these deaths could have been prevented, if only… And yes, some, maybe even many of them could have been prevented, but what does this conclusion help us? On the positive side it tells us what we can do better next time. On the other, negative side, it’s an expression of blame that gives us an adversarial start. Not a good starting point for improvement I’d say.
But that is in retrospect. It’s also possible to read the IOSH’s press release forward looking: “More action needed on preventable deaths”. But that raises of course another problem. Because, what is a ‘preventable death’, or more general, a ‘preventable accident’? How on earth would you know in advance?
Some may offer the ‘All Accidents Are Preventable’ slogan as an answer. I’m not sure if Shelley Frost, executive director of policy at IOSH, meant this when she stated that “All deaths are avoidable”, but still, don’t both statements make the term ‘preventable’ entirely redundant?
But are they really? All accidents are only preventable if we have full control, full foresight and unlimited resources. Since we have neither of these (after all, we are living in a messy, uncertain world and have to do with limited knowledge, time and resources) not all accidents are preventable.
That doesn’t mean that we shouldn’t try to do our utmost, and so I applaud that IOSH is committed “to supporting professionals in building capability within organisations, enabling them to deliver an effective health and safety agenda for their workforce”. Which then (hopefully) will prevent many accidents, and harm. But I still wonder about the clumsy, binary and unrealistic use of language. Proper use of language is extremely important of doing effective health and safety work, after all!
I understand the sentiment and I appreciate the engagement, because every fatality is a tragedy for those involved. I also understand that organisations like IOSH use each and every opportunity to reach the media in order to raise awareness and get attention for safety. Still… I find it rather unprofessional to seek sensation and beat some drum that isn't there. Now, I only get a feeling that the IOSH (an organisation that has to stand for quality in the profession!) apparently cannot tell the signal from the noise.
There are ways to address the issue without spinning the information this way. Why not frame the message in line with the facts. For example:
“We see no improvement…”
“There are still high levels…”
“Every fatality is one too many, and therefore…”
And please drop that ‘preventable’ nonsense. Talking from hindsight is not a good idea.
Also published on Linkedin.
Heroes are an important element of mythology. As a ‘Safety Mythologist’, I am more than just a bit interested in all kinds of elements of mythology. Let’s take a look at heroes in relation to safety. Not an exhaustive review, but just some musings and reflections.
Believe it or not, heroes are to some degree a problematic thing in Norway. This may come as a surprise, what will all the Viking sagas and their quest for fame through heroic deeds? Well, seems they left that behind when they were done raiding most of the civilized (and part of the then not-yet civilized) world. Sometime in the course of the past 800 years Viking heroism was replaced by something called janteloven that goes against focus on individual achievement and (most likely) against heroism. Yet, you will surely find heroes in the workplace.
When you look at safety literature, or management literature in general, you may notice that there is also some promotion of heroes or champions. Examples are (top) managers that are to front and promote an implementation, or the people who have brought upon an exceptional performance (e.g. reported many incidents, came up with a great solution, or ‘saved the day’).
Recently, I read a book on behavioural change. This book encouraged managers to compliment performance of a task where the employee is in kind of a ‘stretch’ situation: a task where he either physical or mentally acts at the boundaries of his capacities. Sounds like a typical trait of a hero.
I understand the message, and I even agree under certain conditions. If we want to develop, we often have to go beyond boundaries and stretch ourselves to achieve something. Role models can (and should) be used in a positive way, as an inspiration and an example to follow. There is, however, also an important pitfall hidden in here. The question is what are these people labelled a heroes for? Are we talking about stretching in personal development or is the hero fixing problems with regard to competing objectives? And, as always, there is the question of what side-effects there will be.
This reminded me of the following anecdote that I was told by my friend and colleague Beate Karlsen. I’m sure that similar incidents have happened other places and you can add your own experiences.
Our story takes place at an organisation that manages infrastructure. They are responsible for both the development of new infrastructure, and the maintenance of existing. There were particular challenges with regard to particular critical technical competence that was necessary for both maintenance and construction projects. There were only a few people with this competence.
In order to deal with these challenges and do maintenance and personnel planning in a good manner, the organisation has a Masterplan. This Masterplan looks at planned maintenance 24 months in advance. A more detailed plan is made with a 12 months perspective and then there is a 4 month planning where things are even further detailed. 14 days in advance the actual jobs are put into a time table and after that there is no major planning; the working orders are distributed. Everything that comes within that 14 day timeframe will be disruptive to the plan and lead to adjustments or cancellations.
Parallel to the day-to-day business of maintenance there was a highly profiled and prestigious construction project. As most major projects, it was managed on budget, time and progress. Funding was the least of their problems. Time and progress, on the other hand, were more problematic because the opening date for the new infrastructure had been decided on a high political level and celebrities and media had been booked for the official opening.
Despite the established planning regime described above, with a planning department working full-time with the Masterplan and more detailed plans and working orders, the project seemed to have ignored or not noticed this regime. The project was in the habit of requesting scarce resources really late, often on the same day they needed them.
Usually the technical department would agree to the request from the highly profiled project. Consequences were that it became harder and harder to make the shifts to go up, maintenance had to be postponed or rushed through and people became tired and stretched thin. Obviously one solution would have been to decline all last-minute requests from the project, but this would at the same time mean that one would miss the income from well-paid hours by the project. Besides there was a lot of perceived and real pressure to deliver, not in the least because the same top-manager was responsible for the project and maintenance.
Therefore the technical department would stretch themselves into extremes to meet all demands. They would also try to solve the situation in a ‘soft’ way, by discussing and explaining the challenges to the project, take up issues with regard to task risk assessments that never had a proper level of quality when done in the last minute, availability of assets and so forth. This seemed to create some understanding from the project, but no lasting improvement.
As a next step, the technical department tried to use incentives. It was communicated to the project that all requests for support had to be done within a reasonable timeframe, otherwise the cost for the support would be many times the ordinary price. But, as said, money was not a problem for the project and they happily paid whatever was necessary to get the job done. The project would also show their gratitude for the help by awarding the ‘heroes’ that fixed their problem by sending cake to the department to celebrate when a job was done.
Celebrations are great, but in this case they were yet another factor that over time contributed to a drift into an unwanted direction with negative consequences. These included signs of burn-out among the employees with critical competence, several serious near-misses because of the very superficial risk assessments and not getting done all necessary maintenance for safe production on the existing infrastructure.
There is kind of a happy end: the project was finished in time and hoorays all around.
What Can We Learn From This?
Firstly you should be very conscious about what heroes you want to cultivate. It may lead to problems in the long run because it can send signals about the preferred behaviour (‘sexy’ projects above ‘boring’ maintenance, production above occupational health, fixing problems above structured and systematic work, etc.). In this case it more or less awarded bad planning with serious disruptions to other tasks, and even endangering safety and health.
This case also contains some great lessons about how incentives and competing objectives work. Research has actually shown that paying a fine for ‘breaking a rule’ kind of legitimizes the behaviour (e.g. a study with fining parents for picking up their children late at a day care center in Israel), so simply making the hours more expensive is not a solution.
Concluding, I am all in favour of flexibility and a strong orientation on solving problems in a practical way. I’m also very much in favour of praising people who have delivered a great job. But watch out for side-effects. Celebrating heroes is a strong expression of the logic that lives in your system and organisation and it is amazingly enduring. It creates expectations - maybe even obligations - for the future. And, ironically, sometimes when a hero fixes a problem, he actually contributes to prolonging it!
Also published on Linkedin.
Compliance is often the most basic place to start working on safety. Rules are often needed to set some basic standards of what society or the organisation sees as an acceptable standard. Rules are also needed to weed out the really bad organisations; by fining them, or even shutting down their business.
Lots of things to say about compliance, as you can see from this introduction (if you want more, it is discussed among others in Myth 41 of my book), but right here, right now I’d like to focus on just one thing: many people seem to live with a clear misunderstanding that compliance is the same as safety. It is not.
What few people seem to realise is that rules are almost always compromises between different agendas. Also keep in mind that no rule is applicable in each and every situation. Rules often deal with common situations, specifying one best way of how to deal with a situation. The real world is much messier than that, alas, and will throw specific situations right in your face where the rule does not fit so well.
History has taught us that even if you comply with rules, even so an accident may occur. Sometimes the combination of several acceptable factors (especially if they are bordering the threshold of acceptance) can cause accidents. Erik Hollnagel calls this functional resonance. Even though all factors are within acceptable limits, their unfortunate combination together can cause an accident.
The other day, my former colleague John Awater told me a fabulous way to illustrate the issue. The example below was inspired by this:
Say your system exists of four different elements that work together. Each of them is to uphold high standards. The norm says that they are to operate at a minimum of 75%. The system as a whole will be clearly unsafe when it drops below 50%.
Say that the individual components are functioning really well, at 80%, 80%, 85% and 85%, so not even close to the threshold. The total effectiveness of the system, however, is not better than 0,8 * 0,8 * 0,85 * 0,85. Which is just above 0,49 - but below our safety threshold of 0,5!
This is of course not the REAL way things work (if only because it will be hard to put simple numbers on something so complex) and just a gross approximation, but it is a nice little model to show you why compliance can fail.
It is also a good illustration to demonstrate that reductionism is often not helpful. In general, it is not the separate parts that create safety, but the parts in interaction. So (back to last week’s post) why so much focus on behaviour?
Also published on Linkedin.
After last week’s heavier stuff, I'd like to tell this time a little story about the stuff that sometimes - with the best of intentions, no doubt - is released onto the world in the name of safety. This is a relatively harmless example, but it’s from a rather successful and reputed European consultancy firm. An example that doesn't necessarily give confidence in the scientific soundness of safety work. As a profession, we have to shape up!
I found this rather silly example in a recent book about safety culture and (mostly) behaviour that I was asked to review by an associate. Lots to comment on that one, which I will do at a later point. Suffice for now to say ‘old wine, new bottles’.
One of the building blocks presented in the book is the ABC model of behaviours: Activators, Behaviour and Consequences. The rationale behind the model: People display certain (wanted, or in this case safe) Behaviour because of a trigger that came before (an Activator, or Antecedent) and that what happens afterwards (Consequences). As simple as ABC, indeed.
The book then tells us that Activators account less for behavioural change than Consequences. Activators get between 5 and 20% and Consequences between 80 and 95%.
Let’s stop here for a minute.
Firstly, I think what they mean to say here is that Consequences do facilitate permanent or long-lasting behavioural change more than Activators. But, that might be a slip of the pen. Still, I wonder, how would you know, except that it has intuitive appeal? We do things because of the things that happen to us.
Secondly, let's assume that these numbers are actually based on something substantial (I mean, other than: “We’ve used 5 and 95 earlier in the book, people recognize that. Let’s also take the 80/20 rule - everybody knows that one.”). Just for the sake of the argument. They mention that the number “depends on the source”, but the sources are not clearly specified (hidden further on in the book they do point to some sources, but when I tried to verify some of them, the results were Zero).
Thirdly, let's also ignore the crudeness of the ABC model (article coming soon). And, fourthly, we must not forget to ignore that the book takes the liberty to redefine A and (to a lesser degree) C.
Having done that, we proceed to the next chapter where we are treated to a discussion of positive feedback (a form of Consequences). We are encouraged to do so because, as we saw, acting on Consequences is 80% more effective than acting on Activators and, by the way, positive feedback is 80% more effective than criticism and other forms of feedback.
I’ll leave it to you to spell out what’s wrong here, and then I’m not talking about the question where that final 80% came from, by then I had already disconnected (alas, dutifully I had to struggle with another 140 pages).
No sources, most likely fantasized numbers and shoddy basic maths… What did I say about shaping up?!
Ow, crap, I did it all wrong - this wasn’t positive feedback, after all… Well, the book looks great and is colourful, how’s that?
Also published on Linkedin.
Originally, I was working on a piece about the ‘relevance’ of outcomes. Reading Ben Goldacre’s “statistics toilet book”, however, I came across the term Surrogate Outcomes. An interesting subject and so ‘relevance’ went on hold for a while. Surrogate Outcomes are when you measure something else than what you really are interested in because you have a hard time measuring one things and you know or assume that the other things is correlated to it. Based on your measurements you draw conclusions about the first thing. I had heard the term before, but this time it struck a note and I decided to look a bit deeper into the subject.
In clinical trials, a Surrogate Outcome (or Surrogate Marker) is a measure of effect of a specific treatment that may correlate with a real clinical endpoint, but does not necessarily have a guaranteed relationship. Surrogate Outcomes are used when the primary endpoint is undesired (like for example death), or when the number of events is very small, thus making it impractical to conduct a clinical trial to gather a statistically significant number of endpoints.
One known example to illustrate Surrogate Outcomes is cholesterol. It is known that higher cholesterol levels increase the likelihood for heart disease, but the relationship is not linear. Many people with normal cholesterol levels develop a heart disease, while many others with high cholesterol do not. The primary endpoint in this case is ‘Death from heart disease’, but ‘cholesterol level’ is used as the Surrogate Outcome. A clinical trial may show that a particular drug is effective in reducing cholesterol, without showing directly that it prevents fatalities.
Because we have problems to measure Safety directly (huge problems, in fact), we use quite a lot of Surrogate Outcomes to measure our efforts. Somewhat confusingly, maybe, after reading the general explanation above: one of the commonly markers is the number of fatalities. Let’s look at a real life example to see how this works and what pitfalls we may encounter.
Last week, a news report on our intranet caught my attention. It was about the Road Safety PIN Award 2016 for Outstanding Progress in Road Safety, awarded to Norway by the European Transport Safety Council (ETSC). The article echoed the press release of the National Road Administration (Statens Vegvesen, SVV) that claimed that Norway was honoured with a European award for the safest roads in the world.
Let’s be generous and not dwell on the obvious mistake in the title of SVV’s press release, which is likely a slip of the pen of an over-enthusiastic PR-consultant, extrapolating Europe to Global scale. Let’s instead focus on other elements. Without having the possibility to go in-depth (road safety is a fairly complex, yet fascinating, subject that would require much more study), I’d like to address some important points.
How to Measure Safety?
Now this is a difficult question, and one for that to my knowledge no one has managed to find a satisfactory solution. This does not stop people pretending that they can, and so we find a wide selection of statistics everywhere. The most common way is of course counting the number of fatalities or injuries. Ignoring for the moment the dilemma whether one should measure something by its absence, I think that the biggest problem lies in the certainty with which some people conclude that absence of accidents (or negative outcomes) means that something is safe.
That is clearly a logical fallacy. It is a basic ground rule in safety (but one that is little understood and practiced by professionals, politicians and public alike) that while the occurrence of accidents can mean that there are problems with regard to safety; the absence of accidents does NOT mean that things are safe. You can achieve the same for example by pure chance, luck or under-reporting.
There is another major problem, namely the relative randomness of consequences like fatalities and injuries. Related to road safety, the difference between life and death, or between serious and light injuries may lie in things like what kind of traffic participant you are, the type of car, speed, the angle in which you are hit, and obviously the number of people involved in the accident. A much better indicator for safety than the number of outcomes would be the number of accidents (e.g. collisions). We cannot tell from the data, but maybe there are just as many accidents in Norway as in previous years and it’s just through some of the aforementioned factors that fewer people die?
It is hard for organisations like ETCS to get good data. The ETCS bases her report on numbers of fatalities (and serious injuries) that each country reports. They encounter very much all of the problems above, and some more. Even though I doubt that many fatal accidents will go unreported, some countries have shaky registrations and routines, some only register certain accidents, and definitions of what a serious injury is differ from country to country. Information about the number of accidents seems to be unavailable. I tried to find them for Norway, but I fear that this information is spread over many players, including the Police, National Road Administration, municipalities, insurance companies (even Statistics Norway does not have the info, I checked) and probably the minor things go entirely unreported.
Toying with Relatives
What I often find problematic, is how reports like this toy around with relative numbers. Even though I often prefer relative numbers to absolute numbers, they can also contribute to confusion or paint a more positive (or negative for that matter) picture than you would. Be wary when you are presented with a series of percentages that all serve to support a certain point of view. As Gerd Gigerenzer has taught us, we have always to ask “percentage of what”. And just check if you are suspicious.
I am not saying that the ETCS has done something obviously wrong, when they first write that there was a 44% decrease between 2010 and 2015 and then continue with “an impressive 20% drop in 2015 compared to 2014 levels”. It is just a bit redundant or unclear what they want to say. Most likely, they are trying to stress the good news. That message can be constructed in a number of ways, however. Look at the statistics and you can calculate that there was a 20% reduction in 2015 compared to 2012 levels as well.
As we saw, outcomes can be seemingly random and fickle. They can just as well go up again. The Norwegian minister of traffic acknowledges this. He is pleased about the praise from ETCS saying that “the best reductions were reached in Norway, where the number of road deaths decreased by 20%...”, but appears to be better briefed than the Council. “The number of accidents on Norwegian roads will vary due to randomness from year to year”, he said.
Indeed, the 20% reduction may sound like a lot, but from 2012 to 2013, there was an increase of over 40 fatalities (29%, just to confuse you further with percentages). Easy come, easy go. In situations where the variation can be so large, flinging around relative numbers compared to the previous year have little or no value and one should rather observe long-term trends. Still this practice is very common in many safety reports.
More Surrogate Markers
I do not know how the concept of leading and lagging indicators goes together with Surrogate Markers, but let’s just try and maybe start a discussion. If I am entirely missing the point, please point this out to me!
Judging from the use in clinical trials, Surrogate Markers tend to be leading indicators: lower cholesterol should lead to lower chance of heart disease. Fatalities, however, or accidents are clearly a Lagging Surrogate Marker for Road Safety. Apparently, the ETSC also uses Leading Surrogate Markers. Their press release states that “Declines in the level of police enforcement of traffic offences are contributing to Europe’s failure to cut the numbers dying in road collisions”, and continues “In a separate report on enforcement, ETSC found that, in over half the countries where data is available, the number of tickets issued over the last five years for use of a mobile phone while driving has reduced, suggesting lower levels of enforcement across Europe”.
In this case enforcement is seen as a Surrogate Marker for Safety (defined as ‘fewer fatalities’) because it is assumed that more enforcement leads to better compliance with traffic safety regulations leads to fewer accidents leads to fewer fatalities and better safety. This reasoning makes intuitive sense to many people, but is not without problems because things are not always that linear. More enforcement can also lead to a greater deal of keeping up appearances and after the control post is passed people speed up just an extra bit to make up for ‘lost time’.
There are other problems. The press release tells us: “Sweden, The Netherlands and Finland are among countries that have reported falls in speeding tickets issued”. I do recall that several years ago the Dutch Police force (and without any doubt Police in other countries too) had specific numerical goals for the number of speeding tickets per year. I do not know if they still have, but abolishing these ridiculous goals will probably lead to a different focus (most likely whatever new political goal they got). Speaking of focus, complaining about reduced traffic enforcement clearly ignores other priorities that might just be a bit more important for society right now - like terror attacks and dealing with the largest wave of fugitives in Europe since Atilla the Hun.
Besides, also speeding tickets are nothing but a Surrogate Marker, because it is assumed that is says something about speeding behaviour. Of course, there are other possible reasons for fewer speeding tickets, like better general compliance of speed limits (not that I serious believe that this is the case, but hypothetically speaking). Interestingly, also this one is a marker that tries to measure something (traffic safety, or rather compliance) by its absence…
Like it or not, unless someone comes up with a brilliant way to measure safety as the ‘real deal’, we will have to work with Surrogate Outcomes for the time being (and all of future). This is okay as long as we understand the limitations and communicate within those limitations.
Where Is the Systems View?
There is one more comment that I must make. The cry for more enforcement is basically a claim that the system is safe if it were not for those stupid and non-compliant people in it. The title and the message from the press releases mentioned above also reflect this view. The SVV claims that the award is about “the safest roads" (not so strange after all that is what they are all about) while they probably should have talked about the traffic system as a whole.
Because, are the roads really so safe? Intuitively I would say Danish roads are much safer than Norwegian ones. And compare the German Autobahns to motorways in Norway and you can count proper motorways almost on one hand (even though there is clearly improvement in recent years). Many Norwegian roads can be characterised as narrow, there are many tunnels (many of them also pretty narrow), the roads go through challenging landscapes, they are strongly affected by weather and seasons and do not underestimate the presence of rather large wild animals. Thinking of those, collisions with large animals like moose or reindeer rarely lead to fatal accidents (or serious injuries for people), but lead to major to damage and have a rather high potential. Although there are many of these accidents, I do not think they reflect in the ETCS numbers at all because they lack particular outcomes.
So, when talking safety on the roads, this might be in many cases thanks to the drivers (who adjust their speed to conditions and handle the constant variability in a rather good way) and not be despite of the drivers. Another important factor is certainly the improving quality of cars, and the ability of many Norwegians to afford them.
It would be fun to do a proper study of all these factors, and I would not be very surprised if the findings would echo many of the remarks that John Adams already made in his brilliant book almost 25 years ago. Adams also mentioned already an issue that was raised by SVV Director Guro Ranes: “We are concerned about the negative trend for seriously injured pedestrians and cyclists”. Adams questioned the fact that there was much attention protecting the best-protected traffic participants (car drivers). This only lead to riskier behaviour, while there was little attention for the weaker parties.
Safe, or Not Safe?
Having made all these critical remarks, it must be said that there appears to be a steady decline in fatalities since the 1960s. Also, the number of (light and serious) injuries also seems to have a steady decline. If you feel like it, you can download numbers from Statistics Norway and play with them to check for yourself (as I did).
I cannot say what has led to this trend, but there has been a lot of good and serious work on road and traffic safety, so one can assume that from a variety of measures at least some have had positive effects. Has the traffic system become safer? We have some circumstantial evidence pointing that way, but I would hold back hallelujah-stories, because there are still many hazards and also worrying new ones, like for example a growing number of East European trucks in doubtful state, with unfit tyres, etc.
As an aside, doing a bit of research on the web I found an interesting British take on the matter, commenting that they were the second safest, but were ‘punished’ for not making more progress from an already safe situation. Well, yet another argument to leave Europe…
A positive, yet critical, conclusion
One might wonder why some Safety People (among which yours truly) appear to be so grumpy. Can we not just be happy that there is a low number of fatal accidents? Should we not celebrate a low number of traffic fatalities? Yes, we should, because it is good news! However, should we also conclude that we are safest based on some outcome number? No! We just cannot tell without additional information! So please learn to be reluctant to draw quick and easy conclusions, even if it is flattering.
Road traffic is a complex system where safety is created by (or emerges from) a large number of factors and their relationships. Do not give in to over-simplification (especially illogical forms), and whenever you see positive numbers also ask for the Bad News and/or look for evidence that disproves a Good News hypothesis. Confirmation can be (too) easy. Trying to falsify may be harder, but it will make your findings more robust and valuable!
Not even a week went by, and one of the messages above was confirmed - fatality levels on Norwegian roads were much higher in the first half of 2016 than the same period the previous year. Sometimes we just hate it to be right...
Also published on Linkedin.