People frequently ask me to tell them more about the scientific foundation of my HPO research. This is because they get lost in the forest of improvement techniques and models in the literature which makes them no longer see the woods because of the trees regarding the quality and reliability of these techniques and models. I tell them that I am a big fan of evidence-based management research which is defined as “the systematic application of the best available evidence to the evaluation of managerial strategies”[1]. This research technique requires a researcher to adhere to the following four research principles[2]:
- Use the best available scientific evidence from peer-reviewed sources. This entails conducting a review of the academic literature thus using the result of published scientific research;
- Systematic gather organizational facts, indicators and metrics, to obtain evidence from practice. This entails conducting field research;
- Conduct an evaluation assisted by procedures, practices and frameworks that reduce bias and improve the quality of the decision making; and
- Make ethical considerations weighing the short- and long-term impacts of decisions on stakeholders.
Evidence-based management research
The main advantage of using an evidence-based management approach is that its results form a solid scientifically proven basis for well-founded decision making, based on facts. In addition to using evidence-based management I regularly present my research on academic conferences and publish it in academic journals (to date I have publishes approximately 80 academic HPO articles) because in these forums your research is scrutinized by academic peers, making sure that what you present and publish can endure the scientific test of criticism.
So in the end my research sample contained data from bad, average, good and excellent organizations. No selection was made thus avoiding the dreaded selection bias.
Avoiding biases
Researchers know that principle (3), avoiding biases, is probably the most difficult one. A well-known bias in the research into high performance organizations is the selection bias. This occurs when the researcher only studies excellent organizations and draws conclusions on what makes these organizations high performing without taking into consideration what low and average performing organizations do and what the actual differences are between these various types of organizations. This bias often happens when researchers look for volunteers in their research: there is a big chance that good companies will offer to participate while worse performing organizations don’t because of fear of bad publicity. I made sure to select a ‘random sample’ for my study so selection bias could not happen. I managed to do this by collecting data via many different ways. For instance, I asked the students in my course Strategic Performance Management – which they had to follow because of their MBA program so they were not necessarily interested in my course per se, nor the topic of HPO – to fill in the HPO Questionnaire. Several of my colleagues at the business school, teaching completely different topics, did the same with their students. In addition, I asked participants on conferences where I spoke about various topics (many of which had nothing to do directly with HPO, like the aforementioned performance management or topics such as beyond budgeting) to fill in the HPO Questionnaire. So in the end my research sample contained data from bad, average, good and excellent organizations. No selection was made thus avoiding the dreaded selection bias.
Perception research
Another issue is the misunderstanding of whether perception research should be used in the study of high performance. In this type of research the respondents are asked about their opinion (i.e. perception) on certain topics, such as the quality of the organization and the colleagues the respondents work for and with. Some people think this type of questioning amounts to ‘butchers approving their own meat’ or in other words: how objective are the answers you get? I have two replies. Firstly, I do not ask the respondents to rate themselves but to rate the organization and its management and employees. This significantly reduces the risk of people giving high scores because they think they themselves are great, as they are not rating themselves but others. Secondly, it has been known for a long time in academic research that there is a strong correlation between the perception scores people give to the quality of their organization and the actual performance of the organization[3]. Thus using perception research is a scientific sound and accepted way of conducting a high performance study.
In the same vein there is always the danger of the ‘halo effect’. This is a specific type of conformation bias wherein positive feelings a person has in one area causes this person to view all other aspects also in a positive light. This also works the other way round, with negative feelings inducing a respondent to rate everything negative. In HPO research this means that if respondents like one aspect of the organization, they will have a positive predisposition toward everything from that organization. You would therefore expect that respondents more or less all rate their organization either high or low. In reality, when looking at the data collected by now from over 55.000 respondents, we do not see this happening. One organization can be rated either high, low or average by its employees, so there doesn’t seem to be the halo effect taking place. And even if there would be the risk of this effect, making sure that there are many different organizations with different performance levels in the research sample virtually eliminates the consequences of the halo effect.
Claim causality
A mistake many researchers in the high performance field make it to claim causalitybetween their HPO model and organizational performance, stating that their model drives performance and not the other way round, while they have only found a correlation (e.g. the two occurrences are related but it is not clear who caused what). The question of causality in HPO research basically is the question of whether the HPO Framework causes better performance or whether high performance organizations find it easier to adopt the HPO Framework, or even that both are the case. In practically it is very difficult to prove causality in the case of high performance because there are too many factors – that can have an influence on performance – for which it is not possible to control for in the study (meaning that you cannot measure their effects). Therefore researchers cannot make their claim of causality. And that is why I state about my HPO Framework that is gives you the best chance on improved performance (compared to other models) but that there is no guarantee.
Longitudinal research
I can make the statement in the last sentence because I have done a lot of longitudinal research on the workings and effects of the HPO Framework in practice. This is probably the distinguishing factor compared to all kinds of HPO models and techniques in the market which are proposed but never really tested in practice (except the occasional case study description). In this longitudinal research I follow an organization which is applying the HPO Framework for a number of years, and I compare HPO scores with the financial and non-financial results this organization achieves over the years. Interestingly I have also managed to do longitudinal research of a different type on the effects of the HPO Framework. In this research the HPO Diagnosis was conducted at a company, after which management implemented the HPO Framework. Two units of the company were selected as case studies. Data were collected, using a questionnaire and interviews, at the beginning and after 18 months, when the HPO Diagnosis was repeated. A linear regression analysis was performed to interpret the data which showed that, despite exposure to the same HPO framework techniques, the two units achieved significantly different outcomes. One unit achieved a higher HPO score and higher organizational results, while the other unit had a lower HPO score and lower organizational results. The key distinguishing factor was the degree to which unit managers applied the HPO framework consistently and with discipline. My conclusion: it pays to become an HPO! This research will be published later this year in the journal Management Research Review with the title ‘Investigating the causal link between a management improvement technique and organizational performance: the case of the HPO Framework.’
I do hope this overview in a nutshell of the evidence-based management techniques I applied has been useful for you and will make you apply the HPO Framework with confidence. I do!
Any questions about my research? Please let me know!
[1] Kovner and Rundall, 2006
[2] Rousseau and Barends, 2011
[3] this has been found by, amongst others, by Dess and Robinson, 1984; Geringer and Hebert, 1991; Bommer et al., 1995; Delaney and Huselid, 1996; Glaister and Buckley, 1998; Dawes, 1999; Deshpandé et al., 2004; Heap and Bolton, 2004; Murphy and Callaway, 2004; Wall et al., 2004; Jing and Avery, 2008; Vij and Bedi, 2016