research governance

Publishing one paper to point out faults in another

This post is prompted by a discussion we had internally about a paper co-authored by one of my colleagues at the University (Dall et al. 2013). This was written as a response to an earlier paper (Tudor-Locke et al. 2011) based on data from the National Health and Nutrition Examination Survey (NHANES) for 2005-6 showing how many steps people took in each minute epoch as measured by an activity monitor. They assumed this was a measure of cadence and came up with the conclusion that:

Self-selected walking at 100+ steps/min was a rare phenomenon in this large free-living sample of the U.S. population, but study participants did accumulate 30 min/day at cadences of 60+ steps/min.

This is simply wrong. Whether the number of steps taken during any minute represents cadence or not will depend on whether the patient has been walking for a full minute or not. Take a person who is recorded as taking ten steps in one minute. This could come from someone who has walking difficulties and walked continuously for a minute but took only ten steps at a true cadence of 10 steps per minute. In this case steps per minute epoch is equal to cadence. Equally it could come from someone who had no difficulty walking and who walked ten steps at a cadence of 120 steps per minute but only for five seconds (ten steps) within the minute. In this case, which will be far more common than the first, cadence and steps per minute epoch are quite different. Recordings of 100 steps per whole minute is not rare because people walk with slow cadence but because it is actually very rare that we walk continuously for a whole minute (Orendurff et al. 2008). If you want to define a threshold value for cadence as was the original intention of Tudor-Locke et al. then you actually have to find some way of recording true cadence and not the number of steps per whole minute.

I think the issues are clear cut so far but then what should our response be? Malcolm and his colleagues had access to data collected with their activPAL device that would allow both true cadence and total number of steps per minute (step accumulation as they call it) to be calculated and demonstrated convincingly, but rather unsurprisingly , that the two are quite different. The published paper (Dall et al. 2013) makes a very interesting read – but should we have to go to this effort? Are there more effective ways of just telling people they are wrong!

Writing a letter to a Journal’s editor is one option but it always feels to me as if there is a time window on this – that the letter should really be submitted fairly soon after an article has been published. I’m not very good at keeping up with the current literature but when I’m working on a particular topic I will often read the relevant articles, both recent and not so recent, quite critically. Working like this it is often some time after publication that I read things that concern me.  A combination of my own inertia and the feeling that I am too late prevent me from doing any more about it.

Maybe I’m wrong in this – maybe we should feel free to use this route at any time that a mistake becomes apparent. Certainly this route ensures that the corrective letter is recorded in the same journal and under the same title as the original article and modern databases are becoming better at flagging this. A disadvantage of the approach of Dall et al. is that the new article is in a different journal published under a completely different title. In this case it has been published in a more technical journal (Medicine and Science in Sports and Engineering) which is unlikely to be read (or even searched) by readers of the original article (in the journal Preventive Medicine).

This wouldn’t be a problem if this were an isolated incident but biomechanics is a complex subject and I suspect that there are many more published mistakes and misconceptions than anyone in the field would want to acknowledge. In the worst case (again more common than we’d want to admit) published mistakes and misconceptions are adopted uncritically by other teams and before you know it what started off as an erroneous paper becomes first a series of erroneous papers and then a tried and trusted method (I’d see the use of CMC  (Kadaba et al. 1989) as a useful measure of repeatability of gait data as an example. Buy my book and read the appendix if you want to know more!).

The situation is exacerbated by the number of people who are involved in biomechanics as a secondary discipline. Some readers (and occasionally authors!) are not in a position to judge whether a method is valid or not. Does this increase the onus on those of us within the community who are aware of problems with specific papers to be more proactive in drawing people’s attention to them?

.

Dall, P. M., McCrorie, P. R., Granat, M. H., & Stansfield, B. W. (2013). Step Accumulation per Minute Epoch Is Not the Same as Cadence for Free-Living Adults. Med Sci Sports Exerc.

Kadaba, M. P., Ramakrishnan, H. K., Wootten, M. E., Gainey, J., Gorton, G., & Cochran, G. V. (1989). Repeatability of kinematic, kinetic, and electromyographic data in normal adult gait. J Orthop Res, 7(6), 849-860.

Orendurff, M. S., Schoen, J. A., Bernatz, G. C., Segal, A. D., & Klute, G. K. (2008). How humans walk: bout duration, steps per bout, and rest duration. J Rehabil Res Dev, 45(7), 1077-1089.

Tudor-Locke C, Camhi SM, Leonardi C, Johnson WD, Katzmarzyk PT, Earnest CP, Church TS. Patterns of adult stepping cadence in the 2005-2006 NHANES. Prev Med 2011;53:178-81.

Advertisements