By Lewis Atkinson and Barbara A. Collins

All strategy is a hypothesis and its implementation is an experiment in execution (Atkinson and Collins, 2023). Strategic actions selected for implementation are those with the greatest likelihood of moving the organization closer to achieving its vision, mission, and values.

Unfortunately, when making decisions about strategy, most leaders rely too heavily on intuition and their causal reasoning may be flawed (Lovallo and Kahneman, 2003).

The objective of this article is to propose a novel method of inquiry based on evaluative thinking that can aid leaders in explaining how and why a particular combination of strategic actions are working in the way that they intended. The proposed method can also help them gradually improve the quality of evidence supporting their causal claims (e.g., increasing advertising increases sales).

Evaluative Thinking: What Works, For Whom and Under Which Conditions

The strategic question we are concerned with is: What is the causal effect of a strategic action on an observed result? For example, what is the effect of increasing advertising spend on sales? Strategic decision-making involves leaders acting on hunches about a chain of causes and their effects to achieve a desired result under specific conditions. This sort of thinking is also known as causal inference. The desired results or ends are what we aim to achieve, and the means are the way in which we choose to achieve them. Put differently, strategic action A causes result B, mediated by conditions C and D. 

To understand causality, strategy professionals and their leaders must adopt a disciplined approach to testing and adapting each strategic action as a theory, rather than a fact. The evaluation should consider how things will change (i.e., hypothesised causal mechanism), including forensic analysis of assumptions about the conditions necessary for the intended result to be achieved. This discipline, called evaluative thinking, necessitates “thinking about your thinking” – the way one reasons, plans, and acts. It involves examining potential flaws in one’s own causal reasoning, motivation, biases and wishes, and learning from failures as well as successes. This self-awareness must be coupled with a commitment to continuous learning and a willingness and ability to modify views in light of reasoned arguments and evidence (Anonymous I, 2023). In the real world, the way a strategic action causes a result cannot be seen. Nor is the observed result a direct consequence of the action. There are conditions such as capabilities and resources and causal mechanisms, or a combination of underlying enabling tactics and business models, processes, or structures, that also impact observed results. Therefore, the cause must be inferred from observed evidence and causal reasoning. Evaluative thinking begins with a clear articulation of the evidence that would indicate the emergence of the desired outcomes from the selected means. 

In practice, confidence in causal claims increases by stepping up the rungs of a “hierarchy of causation” (Pearl, 2000) as shown in Table 1.

Method for Using Evaluative Thinking to Achieve Better Results

A strategic management system (SMS) is a comprehensive “system” that allows leaders to “transform the organization in an effective, efficient and agile manner” (Hadaya, Stockmal et al., 2023). This transformation results from achieving three goals underpinned by three universal premises (see Table 2) which guide the enterprise-wide implementation of the SMS (Haines and McCoy, 1995).

The five-phase method proposed ties the activities of strategy formulation implementation, and execution together in an iterative cycle of continuous improvement, even though presented linearly. The following paragraphs describe each phase for which the objective, a description, techniques, and linkage to the relevant SMS goal or premise are presented.

Phase 1: Achieve a Shared Understanding of Vision, Goals, and Strategy Mechanisms

The objective of this phase is to create a shared understanding of the organization or program vision, goals, and causal mechanisms. The most effective way to do this is through a technique called Parallel Stakeholder Engagement, whereby those implementing the strategy are involved in its formulation. This early and frequent involvement enhances engagement with underlying logic and hypothesised causal mechanisms and assumptions, thereby creating a sense of ownership necessary for successful execution; a key premise of the SMS (see Table 2).

Phase 2: Quantify Shared Understanding by Identifying Evidence of Achievement

The objective of this phase is to identify clear and measurable results (the ends) for the organization or program. This involves translating detailed definitions of the shared vision, mission, and values into tangible evidence of achievement that is clearly understood by both the Board and executives. Two techniques help participating stakeholders focus and agree on customer-centric measures of success: the mission triangle and the customer value-added star. The mission triangle technique asks three clarifying questions in sequence: Who is the customer we serve? What needs do they have that we want to fulfill? What do we do to meet those needs? The customer value added star technique is then used to rank relative competitive positioning of what the organization does for the customer across five criteria: cost, responsiveness, choice, service, and quality. Quantification of these ends can only come after SMS Goal #1 has been achieved. Only then can the means of achieving these ends be formulated.

Phase 3: Select Strategic Actions That Have the Greatest Likelihood of Achieving Results/Ends

This is where the formulation of the means to achieve desired ends begins. The objective of this phase is to select and prioritise strategic actions to transform the operating model by building on strengths and leveraging partnerships to fill capability gaps. Strategists can use a technique called current state assessment here, which includes internal and external analyses to determine what is working now and the best alternative use of resources in the form of new strategic actions. This phase contributes to the achievement of SMS Goal #2 because it prioritises strategic actions, which transform the existing operating model and realign it to the new strategy.

Phase 4: Map Strategic Actions to the Relevant Results/Ends in a Matrix of Causal Relationships

The objective of this phase is to be very explicit about the hypothesised, causal relationships that connect actions to results. These connections, devised in Phase 3, are tested/validated by the Board and executives in Phase 5. This phase maps the action/result connection to show the extent (i.e., casual strength and over what timeframe [short-term vs. long-term]) each strategic action is expected to contribute to a change in each result. A simple matrix can be used to map actions (means) to results (ends) (Figure 1). This work allows the achievement of SMS Goal #3 because it provides a test bed for the experiment of execution.

Phase 5: Test and Resolve the Validity of Hypothesised Causal Relationships in the Matrix

The objective of this phase is to not only test and validate but to also refine the story of how each strategic action actually caused an observed change in result(s), whereby successive trial and error ensures that the best possible combination of interrelated strategic actions (means) evolves over time to best serve changing customer needs, translating to better results (ends). For each cell mapped in the matrix, stakeholder insights are used to validate and resolve possible causal mechanisms and the contribution of the strategic action to the observed changes in results. During this phase, the Board and executives are asked: “In light of the multiple factors influencing a result, has this strategic action made a noticeable contribution to an observed result and in what way?” The technique used to develop the story is called contribution analysis (Mayne, 2001). The purpose of this technique is to prove or disprove causality, or imagining and eliminating alternative explanations. It uncovers conditions, causal mechanisms, and/or associated feedback loops (both positive and/or negative reinforcing) between strategic actions in the matrix. Another useful technique is sensitivity analyses, which involves analyses of how sensitive the results are to variations in the levels of assumptions about prevailing mediating conditions (e,g., low, medium, or high). The sensitivity can be modelled for each causal claim to estimate the extent the observed results would change at each level, or if at all. The work done in this phase is guided by the SMS Premise #1, which entails building an internal management capability to support successful execution (buy-in) and ownership (stay-in) for implementation. 

To demonstrate how this method can be applied, the example of a causal claim that “more beef industry advertising increases demand for beef in Australia” (see Anonymous III for more case study details). To this end, Aggressive promotion of Beef in the Australian market was a strategic action chosen by Meat & Livestock Australia (MLA). It invested approximately $29.5 million in beef industry promotional campaigns in Australia between 2004-2005 and 2009-2010. The intent was to increase the value of beef sales in the Australian market by $300 million each year, over six years. The example is an important demonstration of the application of evaluative thinking:

Phase 1 confirmed a shared vison and common goal of increasing value of beef sales. The MLA Board and cattle industry levy-payer representatives discussed ways to achieve this. Based on their shared understanding they proposed the following hypothesis; “increasing beef industry advertising by $4.9 million per year increases value of beef sales by $300 million per year.”

Phase 2 sought to identify a clear and measurable increase in beef sales value arising from this $4.9 million per year advertising strategy. The MLA Board and executives decided to track the monthly change in the dollar-value of beef expenditure per buyer as a consequence of advertising. The monthly promotional campaign schedule during 2005-2010 was compared to a time-series of AC Neilsen Homescan beef expenditure data over the same period in 2004-2005 and 2009-2010.

Phase 3 considered the likelihood of the advertising strategy actually increasing beef sales value. The observed changes in monthly expenditure per buyer provided no evidence of any increase in the value of beef sales in the period after any of the scheduled dates for promotional campaigns, nor was there any evidence of a sustained long-term trend of increasing value of sales. This concerning result was confirmed by traders actually selling beef in the market (meat packing companies, supermarkets and retail butchers). The finding was further reinforced by evidence provided by traders that the $29.5 million spend by MLA was quite small relative to their own advertising spent over the same period. Yet, they did believe that beef industry advertising was complementary, albeit in a relatively small way, to their own private company marketing strategies. So, the MLA Board and cattle levy-payer representatives funding the strategy, agreed with marketing executives to re-configure the beef industry advertising strategy. It built on the strengths of the existing strategy and leveraged partnerships with traders in order to get better results from future beef industry advertising. 

Phase 4 mapping advertising strategy to relevant results. The analysis in Phase 3 identified two new strategic mechanisms thought to be at work at an all-of-market-system level: consumer awareness and channel engagement (Figure 2). The plausible alternative hypothesis to be tested was that the consumer campaigns (Figure 2, yellow section) could be re-configured to support the entire beef category by helping to maintain awareness of the positive attributes of beef among consumers. This heightened awareness of positive attributes could then be leveraged by traders advertising private-brands to increase the value of their own beef sales. It was also posited that complimentary channel engagement activities (Figure 2, grey section) could maintain market penetration/share for the traders’ private brands by increasing the amount retail shelf space for beef and thus increasing the opportunity for consumers to choose beef for purchase.

Phase 5 testing and resolving the validity of plausible causal relationships. New insights generated in Phase 4 enabled MLA Board and cattle levy-payer representatives funding the strategy to develop a new result for future beef industry advertising; “counter pressures (economic, health, environmental) to reduce red meat consumption by contributing to maintaining consumer expenditure on beef at $6.6 Billion.” The MLA marketing executives then worked to refine the story of how the hypothesised effect of the interaction between these two new strategic mechanisms combined to achieve this new all-of-market-system level result. Ongoing review and adjustment to beef industry advertising strategy now occurs in partnership with traders so that the best possible combination of interrelated strategic actions evolves over time contributing to achieving this all-of-market-system result. 

Conclusion

This paper proposes a novel method of inquiry based on evaluative thinking. Evaluative thinking is a key part of systemic management of strategy and crucial to more accurately defining strategies and removing human biases. Strategists wanting to include this method as part of their practice should not be discouraged by initial resistance from their Board and executives. Human nature, optimism, and confirmation seeking biases and in some cultures a tendency towards “saving face” means that leaders will rarely be inclined to invest in seeking reasons for why their chosen strategic action is not working as intended. However, careful facilitation of leaders and their Boards to help them “think about their thinking” and the way(s) they reason, plan, and act will enable organisations to benefit from better learning from failures as well as successes.

References

Anonymous I (2023) Evaluative Thinking. Better Evaluation. www.betterevaluation.org

Anonymous II (2023) Smoking Facts. American Lung Association. www.lung.org/quit-smoking/smoking-facts

Anonymous III (2023) Aggressive Promotion in the Domestic Market (Beef). Meat & Livestock Australia.

Atkinson, L., and Collins, B. (2023) Is This Strategy Working?: The Essential Guide to Strategy Development, Evaluation, and Goal Achievement. Systems Thinking Press.

Hadaya, P., Stockmal, J., et. al. (2023) IASPBOK 3.0: Guide to Strategy Management Body of Knowledge. International Association for Strategy Professionals.

Haines, S. and McCoy, K. (1995) SUSTAINING High Performance: The Strategic Transformation to A Customer-Focused Learning Organization. CRC Press.

Lovallo, D. and Kahneman, D. (2003) Delusions of Success: How Optimism Undermines Executives’ Decisions. Harvard Business Review, 81(7).

Mayne J. (2001) Addressing Attribution Through Contribution Analysis: Using Performance Measures Sensibly. Canadian Journal of Program Evaluation, 16(1), 1-24.

Pearl, J. (2009) Causality: Models, Reasoning, and Inference. Cambridge University Press, New York.

About the Authors

Dr Lewis Atkinson (CEng, CChem, BSc, PhD (Chem Eng), MBA, PSM) 
is a Global Partner with the Haines Centre for Strategic Management Limited (t/a The Systems Thinking Approach^®) and has government, private, and non-profit clients in Australia and internationally. Lewis is a unique innovation professional with a solid technical and financial grounding, which, combined with his systems-thinking superpowers, enables him to see patterns, pathways, and solutions that others do not.

E: This email address is being protected from spambots. You need JavaScript enabled to view it.

Barbara A. Collins (B.A. (Soc. Psy.), M.S. (Education), SMP Pioneer #019)

has more than 20 years of experience as a manager and department head in both government and private organizations and more than 30 years’ experience in organization development and strategic planning as both an internal and external consultant.

E: This email address is being protected from spambots. You need JavaScript enabled to view it.