Putting Psychometry in perspective
Category : Miscellaneous
Can psychometric tools predict someone’s success or failure in a job or leadership roles? Can these tools “measure” human behaviour? What is the “Unit of Measure” of human behaviour? Distance is measured in kilometres, volume in kilolitres, speed in kilometres per hour and weight in Kilograms. How could behaviour have calibrated measures like these?
Human behaviour is indeterminate and is a response to stimulus from the environment. Its frequency, strength, volatility or stability, flexibility and above all its impact cannot be measured, because we have not yet designed a calibrated scale.
Informed professional will challenge me by referring to the Rensis Likert’s scale. However Likert’s scale at best delivers broad judgment not measurement. When more than one person responds on this scale about the frequency, strength, volatility or stability of someone’s behaviour the scale helps estimation and not measurement, because no 2 persons’ experience of another person’s behaviour can be consistently recalled and reported accurately.
The other issue with the measurement logic is the scale used for measurement has to be calibrated. This means every point of measurement can be measured finitely, for us to arrive at a quantitative value, whose variance is within acceptable range, irrespective of who measures.
Let us take an example:
He/She understand’s the unexpressed motives of others.
Always Most of the times Sometimes Rarely
In a calibrated scale the distance in terms of measurement value between two measurement indicators on the scale has a fixed value or proportion. In the scale above, how fixed are the measurement values between say “Always” and “Most of the times”; “Most of the times” and “Sometimes”. Is it “one unit” between “Always” and “Most of the times”; “One unit” between “Most of the times” and “Sometimes” and “Two units” between “Always” and “Sometimes”? Always or Most of the times and Sometimes cannot have a fixed numerical value. It is different to have a scale “100 times in a year”; “50 times in a year” and “25 times in a year”.
Likert’s scale notwithstanding, we do not yet know how to measure human behaviour. We can confirm at best presence or absence with some degree of frequency and strength estimation. This is like estimating and not measuring the size of a room. We estimate not measure, when we say: Very big, Big and Small.
When the outputs from these instruments are presented as quantitative numerical values such as indices or normative values, you should take it with more than a pinch of salt.
Can psychometry predict job success or leadership success with any degree of predictive validity or reliability? For making valid predictions between one’s behaviour and an outcome, we need to be certain about the following:
That direct not surrogate outcome indicator connect a particular behaviour to the specific outcome. Performance or talent ratings which are surrogate indicators will not meet the muster.
That a particular behaviour is not merely related to a particular outcome but is the driver which causes the outcome. Hence correlation will not suffice. Correlation only tells us that when this behaviour is present a particular outcome is also present, but not which causes which. Fever and infection are related but it does not tell us what caused the infection.
That a particular outcome is not caused by any other behaviour.
That a particular behaviour only when present with some other behaviour or only when not accompanied by some other behaviour causes the particular outcome. Like care should be present with faith in other’s ability without competitiveness for nurturance to be the outcome.
That a particular behaviour is not context specific and universally independent of any contextual factors that causes the particular outcome.
We have to precisely know how much magnitude of frequency or intensity or stability or flexibility of behaviour is necessary to cause the particular outcome. Like for copper to melt there is a certain temperature required and at that temperature some other metal will not melt.
All these have to be tested and empirically established through longitudinal study with the same set of people, in pre-set contexts, where there is no other potential interference. How long should the longitudinal period be? How many people should be studied? How do we control and isolate the contextual factors? All these become critical before we can claim that we can predict job, role or leadership success based on a set of behaviors.
My proposition is not that we should rely on our current subjective judgmental approach and not use any tool. We currently blindly use behaviour profiling tools for behaviour measurement and prediction. We irresponsibly use it for selection and rejection.
Instead if we accept that profiling tools are not measurement or predicting but decision support data generating tools, we will make the better use of these tools.
Behaviour profilers are not like pathological tests which are truly measurement based and predictive. They are not photographs of people which accurately capture all the human features. These are at best thumb nail sketches of people. This is like using your hand to check out fever and not a thermometer. These are not calibrated maps but a pencil sketch on a piece of paper showing someone the general route to be taken. Certainly not “Google Maps”.
Profilers will not create confusion whether the person whose profile you are studying has the characteristics of Mike Tyson or Mother Teresa. However it cannot predict that all people with aggressiveness will create Tyson like outcomes in a boxing ring or outside or all people with compassion will end up as the saint like Mother Teresa. More importantly it cannot tell you how much of aggressiveness or compassion creates Tyson or Mother Teresa outcomes.
Aggression and Compassion like all behaviours are in the end neutral; it is the context and the magnitude that determines appropriateness or otherwise.