Testing for Competencies

5 10 2011

I was asked an interesting question yesterday, with regard to whether the Birkman Method tests for competencies.

No. It doesn’t.

But then, and I would argue this one into the teeth of a hurricane, nor does any other Questionnaire-based instrument. Let me explain.

A competency has at least two components, being a) knowledge about what should be done in a particular situation which is b) put into operation at the appropriate moment. Part a) by itself is not a competency, despite the many claims to the contrary. Knowing what you should do does not mean that you either will –  OR EVEN COULD – put that knowledge into operation.

Often this is because of a well-established problem within learning theory, namely the contextual element to the accessing of prior learning. Very much learning that should feed into the exercise of competencies never gets there because it is first acquired within a classroom setting of some kind; that makes it readily accessible when, for example, sitting an examination (or completing a competencies questionnaire) but rather harder to access in an operational setting.Context prevents much learning from ever being integrated into practice; it unfortunately doesn’t prevent it showing up in a completed questionnaire. So the first problem is that knowledge alone is not the same as competency.

So what should we do. We can, of course, ask respondents to self-report their competencies. The obvious problem is that people aren’t stupid: “do you invest time in developing your direct reports?” “Umm… that’s a hard one. What answer were you hoping for?” We can mitigate that by giving a range of options which at face value sound equally desirable and ask the respondent to indicate the one which best describes them. That still runs into the less obvious problem, which is that even at our most honest, we turn out to be not very good at accurately reporting our own behaviours. We tend to describe our behaviour in terms of what we believe to be the right thing to do, rather than according to what we actually do. So “Yes, investing time in developing my direct reports is my number 2 priority” may not equate to my direct reports feeling invested in at all.

This is not meant to be a counsel of despair, however. Here is what we can do to test for competencies:

  1. Observe and evaluate actual behaviour, preferably validating our observations by comparison with those of the people around the subject (e.g. peers or reports). Please note, I did not just say “run a 360, that should do it”. I remain unconvinced that 360s work as well as they need to; Stephen Covey’s story about the Israeli Air Force not withstanding (“are you suggesting we would spare a colleague’s feelings when the survival of the nation is at stake?”), there is often a reluctance to tell things as they are. The fact that some will use them as a chance to get even doesn’t solve the problem. Use a 360 as back up to actual observation by all means.
  2. But what about situations where the person is not yet on board or at least in the applicable role? I have never found a better answer than Marcus Buckingham’s one, which is to ask for a specific example of a competency in action AND THEN LISTEN LIKE MAD to the first thing they say. If you get something like “yes, I believe that competency x is absolutely critical” you can probably forget ever seeing x in action; if, on the other hand,  you get “okay, last Thursday I had to speak to …” followed by a concrete example of x in action, then Robert is your parental sibling*; you have got a positive match on that competency.

Any role for Questionnaire based tools then? I can’t speak for others, but Birkman definitely gives you a shortcut, not to the observation of the competency itself, but to the likelihood that it will or could be observed. With the breadth of data available in Birkman it is often very possible to establish whether an individual has the motivation and inclination to produce a particular behaviour. You might wonder how this adds anything, if you are going to observe or ask competency-based questions anyway. What it does is allow you to focus in on both likely strengths AND gaps in the competency repertoire for the individual, and ask exactly the right questions to elicit examples of the competencies that are there in spades, as well as those which will test whether the individual has in fact developed competencies which as less typical for someone of their make up.

If any of this sounds a bit involved, just ask yourself how often you have been let down by a battery of tests which appeared to demonstrate the presence of a competency in an individual who then failed to demonstrate said competency when it really mattered. If you are anything like the HR Directors I have spoken with on this subject the answer is probably “too often”.


*It is a requirement of my continuing Australian Citizenship that I occasionally say things like this. If you don’t recognise ‘Bob’s your uncle’ in this form, you might want to rethink that holiday booking; you’ll be up to your neck in drop bears before you get out of the airport. Straight up…