Internal Controls Design website by Matthew Leitch (tutor, researcher, author, & consultant)
New website, new perspective: www.WorkingInUncertainty.co.uk - Related articles - All articles - The author - Services

Quiz results Graphic
Results of a performance management survey

by Matthew Leitch, 19 February 2004


Summary
Favourite ratings
Disagreement
Criticism and suggested alternatives
Common ground and areas of disagreement
Conceptual grouping
Best and worst arguments
Effect of Beyond Budgeting Round Table advocacy
Comments by respondents
Respondents
The 30 arguments
Further information

Thank you

First, thank you to everyone who responded to this survey. Whether you like "Beyond Budgeting" or think it's dangerous nonsense I hope you will find something in these results of interest. There are lessons in advocacy here, and hints about what people are really looking for from their management systems, whether Beyond Budgeting or not. You've certainly given me a lot to think about.

Summary

The survey was conducted online and respondents were self-selected. The first significant question asked if the respondent was already convinced that the Beyond Budgeting model or something like it was a good idea. Then the respondent read 30 arguments in favour of such management models and rated each for how persuasive it was. A negative rating indicated that the respondent thought the argument backfired and argued against Beyond Budgeting and similar models.

Overall, there was a lot of disagreement between respondents. There are many views on these issues.

Arguments that criticised more conventional management models tended to provoke more disagreement, especially among respondents who favoured conventional models. For this and other reasons some arguments were generally accepted by respondents while others split respondents.

The findings suggest that anyone wishing to promote a new idea in management control should explain what to do instead and justify the new ideas on the basis of generally accepted premises, if possible.

Arguments driving for faster, more proactive management were the most popular and persuasive. Arguments in favour of decentralisation, a key part of the Beyond Budgeting model, were not well supported. In addition, arguments often used by the Beyond Budgeting Round Table were barely more persuasive than others used in the survey.

One argument not tested in this survey is that fixed targets should be replaced by targets relative to competitors. This argument tends to provoke a lot of angst and confusion in debates. Two arguments in the survey shed light on this. One argument was that "Working towards a goal of relative superiority over the competition encourages people to consider a wider range of future scenarios and make more flexible and effective plans." Another was that "Re-planning just once a year is far too slow. Quarterly is better. Monthly is better still." Both were popular suggesting that respondents are happy to revise their thinking more often than once a year and to use relative goals, but not happy to abandon absolute targets.

It is sensible to have specific targets and many planning procedures demand it, but it is not sensible to continue driving towards targets even when experience has shown that they need to be adjusted.

The results of this survey suggested to me that it would be helpful to view management models in terms of how fast and proactive they are, and recognise a wide range of methods for achieving excellence on these performance characteristics. I've written a paper along these lines called "A framework for accelerating management systems".

Favourite ratings

Respondents were given 30 arguments in favour of 'adaptive' management systems and asked to rate how compelling each was. A rating of 0 meant it had no persuasive effect. A rating of +5 meant the argument was compelling in favour. A rating of -5 meant the argument backfired, and was compelling against 'adaptive' methods.

A look at the distribution of ratings across all arguments and all respondents shows something about how people chose their ratings. From the histogram you can see that the most popular ratings, along with my interpretation of them, were:

There is a strong skew towards positive ratings which is to be expected as the arguments were chosen specifically because they have often been used in favour of Beyond Budgeting and similar management models.

Extent of disagreement

Of the 30 arguments all but one were rated as compelling in favour of something like Beyond Budgeting by at least one respondent, with a score of +5. However, 13 arguments were rated as compelling against something like Beyond Budgeting by at least one respondent with a score of -5, and all arguments had at least one respondent who thought they backfired to some extent.

Before I explain which arguments provoked the most disagreement there is one aspect of the way the arguments were put that strongly affected ratings. This is a lesson in advocacy, especially if you are promoting a minority viewpoint.

Criticism and suggested alternatives

Some of the arguments used in the survey stated an alleged disadvantage of budgeting or a similar approach based on setting fixed targets annually. Some arguments combined this with a positive suggestion on what should be done instead. There were also two arguments that simply stated a positive reason for doing something more like Beyond budgeting, without criticism of alternatives. These were abbreviated to n - negative, m - mixed, and p - positive respectively.

The results showed that the presence of criticism and suggested alternatives was important.

Even though a mixed argument has more content and so you might think it would be more compelling, the results seem to show that a purely positive argument is slightly more effective. However, the low number of purely positive arguments in the survey makes this finding unreliable. The helpful effect of talking about the preferable alternative is clear, however.

The differences between positive and negative arguments were most marked for respondents who, overall, were not convinced of the advantages of something like Beyond Budgeting. People who were very strongly in favour were quite happy with purely negative arguments. The graph below is an analysis of average ratings between p, m, and n arguments, split between respondents who were "authors" in the field and strongly in favour of Beyond Budgeting or similar, people who were "pro" i.e. supportive of the idea but not authors, and "anti" i.e. people who were not already convinced.

Common ground and areas of disagreement

Overall, the level of disagreement between respondents was high. However, certain arguments were welcomed by nearly all respondents indicating that they are common ground, whereas other arguments divided the respondents.

This graph shows the effect. The green bars show the percentage of respondents agreeing to some extent with the argument. The red bars show the percentage disagreeing with the argument. The white bars below show the percentage rating zero (scale is on the right).

The common ground (defined as statements that 10% or fewer respondents disagreed with to any extent) included:

3Negotiating annual targets and budgets takes far too much management time.
7Fixed targets encourage gaming behaviour, especially near the year end.
10Working towards a goal of relative superiority over the competition encourages people to consider a wider range of future scenarios and make more flexible and effective plans.
17It makes more sense to give people resources when they have something useful to do with them than give people resources as an entitlement in the hope they'll think of something worthwhile later.
19Budgets are purely financial. We should be monitoring a range of KPIs aligned with our strategy.
24A rolling forecast is a much better way of looking forward and planning for the future.
25Re-planning just once a year is far too slow. Quarterly is better. Monthly is better still.
28People feel powerless when targets set a year earlier cannot be met because conditions have changed.

The arguments that most sharply divided respondents (defined as 25% or more respondents disagreeing) were:

4Fixed targets encourage unethical behaviour ranging from small lies about what is an achievable target to Enron-style fraud.
5Regular discussions about variances against budget are not very useful. Time is wasted analysing the variances which should be spent on more important discussions.
8Budgets get in the way of good risk management. People think they know what is going to happen in the future.
9Stating fixed targets to analysts and investors is a recipe for embarrassment.
11Budgetary control involves far too much detail.
12Running a budgetary control system makes the Finance department unpopular.
13If people are ahead of budget/target they slacken off.
15Target/budget setting encourages people to talk down the prospects, while their bosses talk them up.
20Budgetary control is a way for top management to force people to do what they want, but it's like a Soviet style command economy and doesn't work well.
22Budgetary control doesn't work because often there just isn't a way to get back on track.
26Budgets aren't even good plans. They're just negotiated numbers used as part of the annual struggle over pay and recogition.
27Decentralisation is needed if you want to be responsive and budgetary control tends to block empowerment.

Notice how the way an issue is raised can make all the difference. While few people disagreed that "Fixed targets encourage gaming behaviour, especially near the year end." many more disagreed with the statement that "Fixed targets encourage unethical behaviour ranging from small lies about what is an achievable target to Enron-style fraud."

Also, both the purely positive statements (arguments 10 and 24) appear in the "common ground" list.

However, there is more to these lists than mere presentation.

Further evidence as to which arguments split the respondents is provided by analysing the average ratings given by people classified into the following groups: Authors (typically very positive on most arguments), Pro (said they were convinced, but not authors), and Anti (said they were not convinced).

The averages for Author and Pro groups stay positive for all arguments, but the averages for the Anti group dips negatively for the arguments that split the respondents.

Conceptual grouping

Within the list of 30 arguments there were certain themes that came up more than once. Here are the conceptual groups and their members, as I saw them, and ratings within these groups tended to correlate quite well.

Games
4Fixed targets encourage unethical behaviour ranging from small lies about what is an achievable target to Enron-style fraud.
7*Fixed targets encourage gaming behaviour, especially near the year end.
13If people are ahead of budget/target they slacken off.
15Target/budget setting encourages people to talk down the prospects, while their bosses talk them up.
16*When negotiating fixed targets people build in hidden contingencies to try to deal with risk. It is better to be open.
21*Budgets encourage people to spend money or lose it.
30If people do not think they can reach a target they stop trying, or hold back results, and hope to get an easier target next year.
Speed
1Business has become faster moving and that is why we should adapt more than is possible with fixed annual targets.
6Budgets quickly become obsolete as the year goes on.
14Instead of having targets annually that are negotiated, fixed, and then constrain action it is better to be flexible, manage continuously, and have objective goals relative to competitors.
25Re-planning just once a year is far too slow. Quarterly is better. Monthly is better still.
Risk management (usually expressed in terms of being proactive/forward looking)
2Budgetary control involves waiting for problems to show up in our results. That is too slow. We need to be anticipating and acting much earlier.
8Budgets get in the way of good risk management. People think they know what is going to happen in the future.
10Working towards a goal of relative superiority over the competition encourages people to consider a wider range of future scenarios and make more flexible and effective plans.
16When negotiating fixed targets people build in hidden contingencies to try to deal with risk. It is better to be open.
24A rolling forecast is a much better way of looking forward and planning for the future.
29Budgetary control is reactive; we need to be proactive.
Decentralisation
11Budgetary control involves far too much detail.
20Budgetary control is a way for top management to force people to do what they want, but it's like a Soviet style command economy and doesn't work well.
27Decentralisation is needed if you want to be responsive and budgetary control tends to block empowerment.

The Games group is for arguments to the effect that budgetary control leads people to do dysfunctional things. This group is the largest and easiest to identify. It contains some arguments that were generally agreed with (marked above with asterisks) and others that provoked high disagreement.

The Speed group is for arguments about how management need to review things more often. These have quite good ratings and low disagreement.

The Risk Management group is for a collection of issues related to risk, and the ideas are often expressed in terms of being proactive rather than reactive. This is the most popular group overall, though it includes one of the least popular arguments, which is the little-known observation that "Budgets get in the way of good risk management. People think they know what is going to happen in the future." This group has the advantage that it contains the two purely positive arguments.

The Decentralisation group contains arguments related to decentralisation. It suffers from the disadvantage of including the argument about budgets being too detailed, which does not seem to correlated with other arguments related to decentralisation, and to the argument referring to "Soviet style" government, which some probably found provocative. Nevertheless, even the moderately worded argument number 27, "Decentralisation is needed if you want to be responsive and budgetary control tends to block empowerment.", scores poorly with a low average rating and high disagreement.

When the effects of positive, negative, and mixed forms of argument are adjusted for the concept groups retain their relative position: Games - mixed, Risk Management - top, Speed - close second, Decentralisation - last.

There seems to be more agreement that management needs to be frequent and proactive than there is that it should be decentralised.

A related message comes from the ratings of arguments generally favouring more control versus arguments generally favouring less control. In the graph below, arguments have been divided into three groups: m - more control, l - less control, and o - other arguments. I have classified arguments as favouring less control if they favour decentralisation (without specifically claiming that this leads to greater control) or they argue for not doing budgets on grounds of being time wasting without suggesting an alternative. Since it has already been shown that decentralisation arguments and purely negative arguments are unpopular the result of this analysis is no surprise, though it does suggest another reason why those groups were unpopular.

Best and worst arguments

Here are the top 10 arguments in terms of average rating, in descending order, with their average ratings.

Average ratingArg #Argument text
3.1317It makes more sense to give people resources when they have something useful to do with them than give people resources as an entitlement in the hope they'll think of something worthwhile later.
3.057Fixed targets encourage gaming behaviour, especially near the year end.
3.0519Budgets are purely financial. We should be monitoring a range of KPIs aligned with our strategy.
2.882Budgetary control involves waiting for problems to show up in our results. That is too slow. We need to be anticipating and acting much earlier.
2.7024A rolling forecast is a much better way of looking forward and planning for the future.
2.5821Budgets encourage people to spend money or lose it.
2.4529Budgetary control is reactive; we need to be proactive.
2.203Negotiating annual targets and budgets takes far too much management time.
2.2025Re-planning just once a year is far too slow. Quarterly is better. Monthly is better still.
2.1810Working towards a goal of relative superiority over the competition encourages people to consider a wider range of future scenarios and make more flexible and effective plans.

These are the top 11 arguments in terms of percentage of respondents disagreeing.

% dis-agreeingArg #Argument text
517It makes more sense to give people resources when they have something useful to do with them than give people resources as an entitlement in the hope they'll think of something worthwhile later.
57Fixed targets encourage gaming behaviour, especially near the year end.
519Budgets are purely financial. We should be monitoring a range of KPIs aligned with our strategy.
524A rolling forecast is a much better way of looking forward and planning for the future.
525Re-planning just once a year is far too slow. Quarterly is better. Monthly is better still.
7.53Negotiating annual targets and budgets takes far too much management time.
1010Working towards a goal of relative superiority over the competition encourages people to consider a wider range of future scenarios and make more flexible and effective plans.
1028People feel powerless when targets set a year earlier cannot be met because conditions have changed.
12.52Budgetary control involves waiting for problems to show up in our results. That is too slow. We need to be anticipating and acting much earlier.
12.529Budgetary control is reactive; we need to be proactive.
12.514Instead of having targets annually that are negotiated, fixed, and then constrain action it is better to be flexible, manage continuously, and have objective goals relative to competitors.

And these are the bottom 10 arguments in terms of average rating, in ascending order (i.e. worst arguments first).

Average ratingArg #Argument text
0.1012Running a budgetary control system makes the Finance department unpopular.
0.3520Budgetary control is a way for top management to force people to do what they want, but it's like a Soviet style command economy and doesn't work well.
0.6222Budgetary control doesn't work because often there just isn't a way to get back on track.
0.719Stating fixed targets to analysts and investors is a recipe for embarrassment.
0.7313If people are ahead of budget/target they slacken off.
0.788Budgets get in the way of good risk management. People think they know what is going to happen in the future.
0.9011Budgetary control involves far too much detail.
0.9515Target/budget setting encourages people to talk down the prospects, while their bosses talk them up.
1.105Regular discussions about variances against budget are not very useful. Time is wasted analysing the variances which should be spent on more important discussions.
1.1026Budgets aren't even good plans. They're just negotiated numbers used as part of the annual struggle over pay and recogition.

Finally, these are the bottom 12 arguments in terms of percentage of respondents disagreeing.

% dis-agreeingArg #Argument text
4012Running a budgetary control system makes the Finance department unpopular.
37.58Budgets get in the way of good risk management. People think they know what is going to happen in the future.
3520Budgetary control is a way for top management to force people to do what they want, but it's like a Soviet style command economy and doesn't work well.
32.511Budgetary control involves far too much detail.
305Regular discussions about variances against budget are not very useful. Time is wasted analysing the variances which should be spent on more important discussions.
3026Budgets aren't even good plans. They're just negotiated numbers used as part of the annual struggle over pay and recogition.
3013If people are ahead of budget/target they slacken off.
3022Budgetary control doesn't work because often there just isn't a way to get back on track.
27.59Stating fixed targets to analysts and investors is a recipe for embarrassment.
254Fixed targets encourage unethical behaviour ranging from small lies about what is an achievable target to Enron-style fraud.
2527Decentralisation is needed if you want to be responsive and budgetary control tends to block empowerment.
2515Target/budget setting encourages people to talk down the prospects, while their bosses talk them up.

Effect of Beyond Budgeting Round Table advocacy

The Beyond Budgeting Round Table has been operating for some years now, running courses and seminars, distributing papers, and generally promoting its management model. In 2003 a book was published called "Beyond Budgeting", written by Robin Fraser and Jeremy Hope, published by Harvard Business School Press.

Arguments often used by the BBRT should fare well in the survey because they are more likely to be familiar and well understood by at least some of the respondents, and because over the years the BBRT will have learned to concentrate on the most effective arguments.

Dividing the 30 arguments in the survey between those commonly used by the BBRT and those not was not easy as so many arguments have been advanced by the BBRT in its various publications and speeches. I did it using my knowledge of their materials.

A look at the scattergram shows that there is little if any difference in impact between arguments used often by the BBRT and other arguments.

However, the average ratings and percentage of disagreement show a slight effect in the expected direction. The most sensitive indicator, percentage of respondents disagreeing, averaged 17.8% for arguments which are often used by the BBRT, and 21.4% for arguments not often used by the BBRT.

Comments by respondents

At the end of the survey respondents had the opportunity to make other comments. Here are the ones not related to the design of the survey, edited in places to preserve anonymity:

"Quite interesting answering this from a public sector point of view. Is it not possible to have annual budgets etc. (useful because given annual amounts of money from government and funders and looking to achieve long term rather than short term goals) and flexibility, proactivity, acknowledgment of changing conditions etc.? This doesn't seem like an either or to me (but I don't claim to be particularly well-informed!)"

"Deviation from aim control charts are very helpful to make budgeting/forecasting more meaningful - especially if the measurement process is bottom-up."

"Agree with concept of adaptive systems but remain unconvinced of success unless culture in many organisations is changed otherwise back to old story of using metrics as a measurement tool rather then a tool to help improve; metric driven versus goal driven.  If you can change culture then you will reap benefits of systems that will help decision support in reacting to market demands etc and benefit the overall organisation rather then reacting to internal budget needs."

"Beyond budgeting is mainly applicable in private sector companies. What about public sector organisations? Any example of beyond budgeting in public sector organisations at hand? They need it probably much more than private sector."

"Organisations only have a finite amount of cash. An awareness of how we are spending is essential."

"I don't think it's as black-and-white as "budgetary control and other systems with fixed performance targets should be ditched and replaced with more adaptive management systems". Why not have both and use whichever is more appropriate? Standard systems as a baseline, with adaptive systems in order to be responsive to change? I particularly disagree with Q8 - you need a baseline against which to identify risk."

"In budget controls we are looking at the rationality of spending the money we already have. However, we need to look at the market and explore areas where we can create business and make money - increase our business and worth. Like, Act on opportunity costs."

"In a (UK) Government Department situation, where (in general) it is not easy to carry unspent funds forward to the next financial year, budgetary control is essential. The answer is to link budget monitoring more closely with measuring how far objectives are being met."

"All of the criticisms above are addressed in well-managed budget systems.  It is not so much budgets that cause the problems as they way in which they are used - normally in businesses which are not particularly well-managed anyway.  I am not saying we definitely have to keep budgets, but I have yet to see a proposal which significantly improves on the situation without raising its own weaknesses."

"In our City Government, City Commissioners and the citizens are used to hearing the City Manager talk about what the City plans to do based on the budget.  I believe it would be difficult to eliminate the budget process in our City.  However, using the concepts in the beyond budgeting model may help the City Manager make the transition away from a purely budget basis for discussions regarding what the City will do each year or over the next 5 years."

"It is essential that the schedule and budget reflect a best estimate of the actual work to be accomplished. If this cannot be projected 1 year, six months or whatever, ahead then you must move to near real-time planning (actual planning window determined by the advance time that task are known, together with their expected duration) Note: I use the term "task" to reflect a series of activities associated with expenditure of revenues and the development of end products."

"We are privately held, so as a non-public firm we do not encounter many of the issues noted in the survey."

"In order to decentralise expenditure target setting and control, all employees with expenditure delegations must have an up to date understanding of the strategic goals of the organisation, and be prepared to be accountable for achieving these goals. Too few organisations do this, and are afraid to ditch the budget process as a result."

Respondents

There were 40 respondents. All completed the survey online. Most have a background in performance management, audit, or accountancy. Two were classified as authors, 20 more were in favour of Beyond Budgeting or something similar but were not authors in the field, 12 were not convinced/against, and 6 could not be classified. This classification was based on the question in the survey but in three cases this answer was lost due to a technical fault but I inferred agreement from the high average ratings. In nearly all cases of known opinion, an average rating of more than about 1.75 indicated overall support.

The 30 arguments

These are the arguments used in the survey, in the order they were presented:

Arg #Argument text
1Business has become faster moving and that is why we should adapt more than is possible with fixed annual targets.
2Budgetary control involves waiting for problems to show up in our results. That is too slow. We need to be anticipating and acting much earlier.
3Negotiating annual targets and budgets takes far too much management time.
4Fixed targets encourage unethical behaviour ranging from small lies about what is an achievable target to Enron-style fraud.
5Regular discussions about variances against budget are not very useful. Time is wasted analysing the variances which should be spent on more important discussions.
6Budgets quickly become obsolete as the year goes on.
7Fixed targets encourage gaming behaviour, especially near the year end.
8Budgets get in the way of good risk management. People think they know what is going to happen in the future.
9Stating fixed targets to analysts and investors is a recipe for embarrassment.
10Working towards a goal of relative superiority over the competition encourages people to consider a wider range of future scenarios and make more flexible and effective plans.
11Budgetary control involves far too much detail.
12Running a budgetary control system makes the Finance department unpopular.
13If people are ahead of budget/target they slacken off.
14Instead of having targets annually that are negotiated, fixed, and then constrain action it is better to be flexible, manage continuously, and have objective goals relative to competitors.
15Target/budget setting encourages people to talk down the prospects, while their bosses talk them up.
16When negotiating fixed targets people build in hidden contingencies to try to deal with risk. It is better to be open.
17It makes more sense to give people resources when they have something useful to do with them than give people resources as an entitlement in the hope they'll think of something worthwhile later.
18Paying people bonuses on the basis of whether they met a target negotiated a year earlier when we knew little about the conditions that would be faced is unfair and stressful.
19Budgets are purely financial. We should be monitoring a range of KPIs aligned with our strategy.
20Budgetary control is a way for top management to force people to do what they want, but it's like a Soviet style command economy and doesn't work well.
21Budgets encourage people to spend money or lose it.
22Budgetary control doesn't work because often there just isn't a way to get back on track.
23Studying actuals properly is more effective than looking at budget variances.
24A rolling forecast is a much better way of looking forward and planning for the future.
25Re-planning just once a year is far too slow. Quarterly is better. Monthly is better still.
26Budgets aren't even good plans. They're just negotiated numbers used as part of the annual struggle over pay and recogition.
27Decentralisation is needed if you want to be responsive and budgetary control tends to block empowerment.
28People feel powerless when targets set a year earlier cannot be met because conditions have changed.
29Budgetary control is reactive; we need to be proactive.
30If people do not think they can reach a target they stop trying, or hold back results, and hope to get an easier target next year.

Further information

If you would like to analyse the original data yourself I can provide a matrix of the ratings given, classified between Author, Pro, Anti, and unknown. The information will not allow you to identify respondents or their organisations.

Copies of the original survey are also available. Please contact me at matthew@internalcontrolsdesign.co.uk.



© 2004 Matthew Leitch
New website, new perspective: www.WorkingInUncertainty.co.uk - Related articles - All articles - The author - Services

If you found any of these points relevant to you or your organisation please feel free to contact me to talk about them, pass links or extracts on to colleagues, or just let me know what you think. I can sometimes respond immediately, but usually respond within a few days. Contact details

Matthew Leitch - Author

About the author: Matthew Leitch is a tutor, researcher, author, and independent consultant who helps people to a better understanding and use of integral management of risk within core management activities, such as planning and design. He is also the author of the new website, www.WorkingInUncertainty.co.uk, and has written two breakthrough books. Intelligent internal control and risk management is a powerful and original approach including 60 controls that most organizations should use more. A pocket guide to risk mathematics: Key concepts every auditor should know is the first to provide a strong conceptual understanding of mathematics to auditors who are not mathematicians, without the need to wade through mathematical symbols. Matthew is a Chartered Accountant with a degree in psychology whose past career includes software development, marketing, auditing, accounting, and consulting. He spent 7 years as a controls specialist with PricewaterhouseCoopers, where he pioneered new methods for designing internal control systems for large scale business and financial processes, through projects for internationally known clients. Today he is well known as an expert in uncertainty and how to deal with it, and an increasingly sought after tutor (i.e. one-to-one teacher). more

Please share:            Share on Tumblr