I had a query recently from a researcher who devised a variant of the GPS to incorporate trunk data. He’d submitted it for publication but the reviewer asked for evidence that the scale had been validated and he wanted to know how to respond. It made me stop and think about the whole process of validation. It’s one of those areas in which the concepts evolved within psychometrics, where they are relevant, have been allowed to spill over into other areas, where they are not.

For the uninitiated the field appears complex. I remember a PhD student once who we asked to validate a scale coming back a week later completely confused – she did master it eventually but there was a steep learning curve. Read the relevant chapter in Portney and Watkins for example and you are conducted on a whistle-stop tour of face, content, criterion-related and construct validity in 20 pages. Altman and Bland (direct link to article) whip through these even more quickly and add in internal consistency for good measure.

I don’t have enough space in a blog article to go into why this is all necessary (Altman and Bland provide a succinct summary) but I do want to explore when it is necessary which I feel is very poorly understood. Stating it rather boldly, validation of a scale is required when we don’t know what we are measuring. Psychometrics evolved to support psychologists and behavioural scientists who wanted to quantify concepts such as happiness or anxiety. Neither happiness nor anxiety is defined in terms of numbers so the researcher has to go through a process of convincing her or his peers that the scale she or he has devised is a valid measure of what the rest of us understand by the terms. In our own, field health related quality of life or patient satisfaction or even general terms like gross motor function or mobility are similar qualitative terms. If we want to assign a numerical value to these then we need to go through the same process. As our understanding of the underlying issues becomes more sophisticated then so does the battery of different types of validity that we need to establish in order to convince others that our scale is represents what we say it represents.

By contrast, however, such a process of validation is not required if we do know what we are measuring. If we are measuring length, time, speed or joint angle, moment and power then there are very precise definitions of the terms we are seeking to measure and there is absolutely no need to go through this full validation process. The question we need to ask is whether the tests are accurate rather than whether they are valid. This requires a completely different set of techniques. The GPS is a derivative of joint angle measurements and I would argue that a consideration of accuracy is required rather than one of validity.

Of course there is a subsequent question which is whether any measurement is useful. Just because a variant of the GPS including trunk data is well defined and accurate doesn’t necessarily mean it is useful in any particular context. That, however, is yet another and different question.