Internal Controls Design website by Matthew Leitch (tutor, researcher, author, & consultant)
New website, new perspective: www.WorkingInUncertainty.co.uk - Related articles - All articles - The author - Services

Requirements Graphic


Requirements of Risk Management processes

by Matthew Leitch, 15 April 2007


Please respond!

This article suggests a list of requirements for risk management processes, designed to be open to a variety of techniques, and yet to exclude poorly designed processes.

An unusual feature is that it is designed so that you can easily give feedback on each item, telling me what you think of it and any good or bad points. I will look at these closely and future versions of this web page will be improved as a result.

I am not the only person who is convinced that existing guides and standards on risk management, such as COSO's ERM framework, the IRM/AIRMIC/ALARM document "A risk management standard", and many others are currently promoting poor practices and the world deserves better.

However, it is proving extremely difficult to make any headway because, although many agree that some big changes are needed, it is much harder to agree what would be preferable. So, on this web page I've listed some requirements carefully worded to be open to a wide variety of specific methods but to eliminate bad techniques from consideration. I would like to know which of these requirements people find agreeable and useful. With enough feedback it would be possible to evolve a set of requirements that are at least helpful, right, and generally acceptable, though not necessarily as detailed as many would like.

Requirements defining scope

The risk management process shall be for organisations.

[Note: This is not to say that you could not do something for an individual, but the target of these requirements is risk management processes for organisations.]


Comment:

In principle the risk management process shall involve all people within the organisation, but potentially to different extents, and potentially with some people having no actions to take.


Comment:

The risk management process shall address all forms of uncertainty that matter, regardless of whether potential outcomes can be identified and whether any outcomes identified are favourable or unfavourable.


Comment:

The risk management process shall address all forms of action in response to uncertainty including regular actions, actions in response to situations, and one-off actions.


Comment:

Requirements defining objectives

The risk management process shall be designed to add value by improving the way risk and uncertainty are managed.


Comment:

The risk management process shall also be designed to provide information about its effectiveness and efficiency.


Comment:

Requirements of methods in general

The risk management process shall be documented.


Comment:

The risk management process shall include situation analysis (in the widest sense), design of courses of action (including what are often called controls), and other decision making activities.

[Note: This is far reaching since most existing guides give little or not attention to generating and refining courses of action. They are written as if no design effort is required.]


Comment:

The risk management process shall include activities designed to improve the risk management process itself.


Comment:

The risk management process shall be designed to counter the common human tendency to underestimate uncertainty.


Comment:

The risk management process shall avoid introducing significant systematic bias.

[See below for more detail on this crucial requirement.]


Comment:

Requirements relating to evidence

The risk management process shall require that when decisions are taken relevant evidence including historical data will be used where the benefits are thought to outweigh the costs of doing so.


Comment:

The risk management process shall include activities designed to make use of evidence including historical data more cost effective over time.


Comment:

The risk management process shall require that where evidence is used in decision making (or judgements affecting decisions) the extent of data and thoroughness of work done to use it shall be summarised and written down with other input to decision making.


Comment:

Please use this box for any comments, suggestions, or questions you may have on this subject. (Remember, if you want a reply I need to know your email address.)

And finally

Thank you for participating.


Appendix: Avoiding significant systematic bias

This simple requirement eliminates many familiar risk management techniques because bias is easy to introduce without realising it. The wording of this requirement is intended to allow bias that is tiny overall and to allow "bias" that works differently for each element of an analysis so that the overall impact is low.

Systematic biases are more dangerous. For example, if you generate random numbers and then round them using the usual rule then the average of the rounded numbers is not much different from the average of the un-rounded numbers. Each individual number is "biased" but because the effect is sometimes in one direction and sometimes in the other the overall effect is low. In contrast, if you round every number down then the average of the rounded numbers will be lower than the average of the un-rounded numbers. All, or most, of the items have been biased in the same direction. This is systematic bias.

Here are some common systematic biases. In most cases they relate to analysing and weighing risk(s) so their importance can also be reduced by using processes where management of risk does not depend heavily on accurate analysis. This is preferable to having decisions about actions strongly influenced by risk analyses that are systematically wrong.

Bias

When important

Possible fixes

Understatement of uncertainty caused by separate ratings of impact and probability/frequency.

Wherever risk items have uncertain impact or multiple possible outcomes are included within the same item, or both. In common practice nearly all risk register items are like this.

Capture probability distributions, in some way, potentially very simplified.

Understatement of risk caused by ignoring correlations between risk items.

Wherever there is significant correlation between risk items.

Some kind of causal or correlation modelling or consideration.

Overstatement of risk impacts caused by double counting impacts when one risk item is a cause of another.

Wherever there are significant causal links between risk items.

Consider the causal links and use a method of weighing impacts that does not double count.

Undervaluation of actions affecting lower weighted risks and overvaluation of actions affecting higher weighted risks caused by using a risk "appetite" cut-off line.

Wherever risk appetite cut-off lines are used.

Do not use this method!

Understatement of risk overall and undervaluation of actions affecting lower weighted items caused by focusing on "top" risk items e.g. the top 10.

Wherever a significant amount of risk is excluded from further analysis on the grounds that individual risk items are low weighted.

Partitions of the risk universe; controlled disaggregation.

Understatement of risk caused by weighing all risk items by their probability weighted average money value (or another scale of value that does not include a weighting for risk).

Wherever there are significant risk items whose impacts can be extreme from the point of view of the party or parties affected.

Some more refined risk weighting method.

Understatement of risk caused by modelling all risks as distributions less spread than the true distributions. E.g. using Normal distributions where Levy distributions would be more accurate.

Wherever the difference between these is significant. Hard to give a rule of thumb on this.

Use appropriate distributions.

Understatement of risk caused by modelling risk items using a symmetric distribution when the true distributions are all skewed one way. E.g. task durations in a project.

When the true distribution is typically skewed in one direction.

Use a more appropriate skewed distribution.

Undervaluation of courses of action caused by failing to capture the value of being able to make decisions in future once more is known.

Wherever there is significant scope to react to events.

Model or at least consider potential discoveries and actions that could be taken.

(Typically) overvaluation of actions caused by the 'Flaw of Averages' i.e. the belief that giving 'average' numbers for uncertain inputs in a model will give output numbers that are 'averages.'

Where the model is non-linear.

Monte Carlo simulation.

Click here to return to the requirements list without losing any entries you may have made so far.



© 2007 Matthew Leitch
New website, new perspective: www.WorkingInUncertainty.co.uk - Related articles - All articles - The author - Services

If you found any of these points relevant to you or your organisation please feel free to contact me to talk about them, pass links or extracts on to colleagues, or just let me know what you think. I can sometimes respond immediately, but usually respond within a few days. Contact details

Matthew Leitch - Author

About the author: Matthew Leitch is a tutor, researcher, author, and independent consultant who helps people to a better understanding and use of integral management of risk within core management activities, such as planning and design. He is also the author of the new website, www.WorkingInUncertainty.co.uk, and has written two breakthrough books. Intelligent internal control and risk management is a powerful and original approach including 60 controls that most organizations should use more. A pocket guide to risk mathematics: Key concepts every auditor should know is the first to provide a strong conceptual understanding of mathematics to auditors who are not mathematicians, without the need to wade through mathematical symbols. Matthew is a Chartered Accountant with a degree in psychology whose past career includes software development, marketing, auditing, accounting, and consulting. He spent 7 years as a controls specialist with PricewaterhouseCoopers, where he pioneered new methods for designing internal control systems for large scale business and financial processes, through projects for internationally known clients. Today he is well known as an expert in uncertainty and how to deal with it, and an increasingly sought after tutor (i.e. one-to-one teacher). more

Please share:            Share on Tumblr