governance

Producer pays publishing

In UK Higher Education research quality is assessed every four or five years through the Research Excellence Framework (REF). We have  just finished our submission to REF 2014 and it won’t be until late next year that we receive our results. Already, however,  the government’s plans for the next REF are starting to emerge. The latest communication was an indication that only journal articles made available for open access are likely to be accepted.

The drive is now well and truly on for open access publishing. If public bodies or charities have funded research then surely the results should be open to everyone. I feel a bit like Canute in arguing against the tide, but then again I feel like Canute quite a lot of the time.

The most obvious point to make is that most journal articles are open to everyone already. If you log on to any of the big publisher’s web-sites you can buy a copy of any journal article you like (cost is typically about £30).  The issue isn’t whether articles are open or not, it is what they cost. Most universities and academics are happy to supply copies of individual articles free if you just e-mail a specific request. “Green” publishing effectively formalises this process.

The argument that the research has been funded by governments or charities is still relevant but it is not clear cut. There are many instances where governments invest money but expect the user to contribute at point of use. I’m sitting on a train at the moment that benefits from considerable public subsidy but I still expect to buy a ticket for each individual journey. Even where freedom of information legislation applies there is generally a charge for that information being made available.

The research paper is a product. It’s a product that, if it is useful, someone should be prepared to buy. The general rule for pricing is that the value is determined by what the consumer is prepared to pay.  Open access publishing, at least in the “Gold” form where a fee is charged to the publishing institution, is moving to a model where the producer is required to pay. I can’t think of any other product whose cost is charged to the producer – it runs counter to the logic of the market economy.

The problem with shifting to a producer pays model is that the economics of the system becomes driven by the need to produce rather than any consideration of whether the product is of any value. Most of us in academia are not challenged by the scarcity of data out there – we are overwhelmed by the quantity of it, particularly that which is of low quality.  There is a real risk that by removing demand side drivers and increasing supply side drivers the quality sieve will become coarser, not more refined and there are indications that we are beginning to see this already.

Of course the elephant in the room, and it is a very big elephant, is the academic publishing industry. Its initial reluctance to embrace open access publishing now appears only to have represented the thinking space required to work out a business model to exploit it most productively. Having established this, the big publishers are wading in with “open” arms. It’s a reflection of their inertia that they’ve taken so long to realise that this is a licence to print money. They can now charge the consumer (academic institutions are continuing to pay large fees to have ready access to content) and the producer. Over time the balance will shift from the former to the latter but it is becoming increasingly apparent that publishing a product whose value is set by the producer is a much more attractive proposition than one whose value is set by the consumer.

Publishing one paper to point out faults in another

This post is prompted by a discussion we had internally about a paper co-authored by one of my colleagues at the University (Dall et al. 2013). This was written as a response to an earlier paper (Tudor-Locke et al. 2011) based on data from the National Health and Nutrition Examination Survey (NHANES) for 2005-6 showing how many steps people took in each minute epoch as measured by an activity monitor. They assumed this was a measure of cadence and came up with the conclusion that:

Self-selected walking at 100+ steps/min was a rare phenomenon in this large free-living sample of the U.S. population, but study participants did accumulate 30 min/day at cadences of 60+ steps/min.

This is simply wrong. Whether the number of steps taken during any minute represents cadence or not will depend on whether the patient has been walking for a full minute or not. Take a person who is recorded as taking ten steps in one minute. This could come from someone who has walking difficulties and walked continuously for a minute but took only ten steps at a true cadence of 10 steps per minute. In this case steps per minute epoch is equal to cadence. Equally it could come from someone who had no difficulty walking and who walked ten steps at a cadence of 120 steps per minute but only for five seconds (ten steps) within the minute. In this case, which will be far more common than the first, cadence and steps per minute epoch are quite different. Recordings of 100 steps per whole minute is not rare because people walk with slow cadence but because it is actually very rare that we walk continuously for a whole minute (Orendurff et al. 2008). If you want to define a threshold value for cadence as was the original intention of Tudor-Locke et al. then you actually have to find some way of recording true cadence and not the number of steps per whole minute.

I think the issues are clear cut so far but then what should our response be? Malcolm and his colleagues had access to data collected with their activPAL device that would allow both true cadence and total number of steps per minute (step accumulation as they call it) to be calculated and demonstrated convincingly, but rather unsurprisingly , that the two are quite different. The published paper (Dall et al. 2013) makes a very interesting read – but should we have to go to this effort? Are there more effective ways of just telling people they are wrong!

Writing a letter to a Journal’s editor is one option but it always feels to me as if there is a time window on this – that the letter should really be submitted fairly soon after an article has been published. I’m not very good at keeping up with the current literature but when I’m working on a particular topic I will often read the relevant articles, both recent and not so recent, quite critically. Working like this it is often some time after publication that I read things that concern me.  A combination of my own inertia and the feeling that I am too late prevent me from doing any more about it.

Maybe I’m wrong in this – maybe we should feel free to use this route at any time that a mistake becomes apparent. Certainly this route ensures that the corrective letter is recorded in the same journal and under the same title as the original article and modern databases are becoming better at flagging this. A disadvantage of the approach of Dall et al. is that the new article is in a different journal published under a completely different title. In this case it has been published in a more technical journal (Medicine and Science in Sports and Engineering) which is unlikely to be read (or even searched) by readers of the original article (in the journal Preventive Medicine).

This wouldn’t be a problem if this were an isolated incident but biomechanics is a complex subject and I suspect that there are many more published mistakes and misconceptions than anyone in the field would want to acknowledge. In the worst case (again more common than we’d want to admit) published mistakes and misconceptions are adopted uncritically by other teams and before you know it what started off as an erroneous paper becomes first a series of erroneous papers and then a tried and trusted method (I’d see the use of CMC  (Kadaba et al. 1989) as a useful measure of repeatability of gait data as an example. Buy my book and read the appendix if you want to know more!).

The situation is exacerbated by the number of people who are involved in biomechanics as a secondary discipline. Some readers (and occasionally authors!) are not in a position to judge whether a method is valid or not. Does this increase the onus on those of us within the community who are aware of problems with specific papers to be more proactive in drawing people’s attention to them?

.

Dall, P. M., McCrorie, P. R., Granat, M. H., & Stansfield, B. W. (2013). Step Accumulation per Minute Epoch Is Not the Same as Cadence for Free-Living Adults. Med Sci Sports Exerc.

Kadaba, M. P., Ramakrishnan, H. K., Wootten, M. E., Gainey, J., Gorton, G., & Cochran, G. V. (1989). Repeatability of kinematic, kinetic, and electromyographic data in normal adult gait. J Orthop Res, 7(6), 849-860.

Orendurff, M. S., Schoen, J. A., Bernatz, G. C., Segal, A. D., & Klute, G. K. (2008). How humans walk: bout duration, steps per bout, and rest duration. J Rehabil Res Dev, 45(7), 1077-1089.

Tudor-Locke C, Camhi SM, Leonardi C, Johnson WD, Katzmarzyk PT, Earnest CP, Church TS. Patterns of adult stepping cadence in the 2005-2006 NHANES. Prev Med 2011;53:178-81.