Getting physical

I had meant to move on from  this issue about the complexity of biomechanics and the quality of the research questions we ask … but then last night, in her comment, Quin drew my attention to a series of articles entitled “Biomedical research – increasing value, reducing waste” that were published in the Lancet in June (these are free to download but you need to register first – also free). They make fascinating reading. If you think I was a bit grumpy and cynical in what I wrote the week before last then you should have a look at what these guys are saying! (The articles are a bit heavy going and an easier and more entertaining alternative is Ben Goldacre‘s book Bad Science, it’s been around long enough that you can pick it up for 1p on Amazon and just pay the postage).

The issue marks the twentieth anniversary of an article, The scandal of poor medical research, written by the statistician Doug Altman in the BMJ which was perhaps the first public recognition of the poor quality of much clinical research with the tag line, “we need less research, better research and research done for the right reasons“. In another commemorative piece, Medical research still a scandal,  Richard Smith, who was editor of the BMJ at the time, laments on how little has changed, despite, perhaps, a wider awareness of the problems.

One of the responses to Bland’s original article which appealed to me has been given the title Theory must drive experiment. In it the author (a JA Morris from the Royal Lancaster Infirmary) attributes the problem of poorly formulated research questions to a failure of clinical scientists to develop an underlying theoretical basis for their experimental observations. This has always puzzled me as well and, over the years, I’ve come to the conclusion that, as someone who trained originally as a physicist, I’ve got a markedly different view of the world to many of my colleagues who trained in medicine or health sciences.

Arthur Eddington, “… do not put too much confidence in experimental results until they have been confirmed by theory”

As a physicist I expect to understand the results of my experiments and to be able to align them with an underlying theory. An understanding of that underlying theory then develops new research questions. My  knowledge continues to develop by the continued construction and refinement of the underlying theory (I’m sure there is a posh name for this in the philosophy of science). Taken to an extreme this results in Eddington‘s warning to the physicist  not to “put too much confidence in experimental results until they have been confirmed by theory“. Whilst at face value this sounds like an injunction against publishing experimental data it is actually a plea for careful consideration of the results in the light of the underlying theoretical framework and a refinement of that framework if  necessary.

Ernest Rutherford, “When it comes to science there is physics and there is stamp collecting”

I wouldn’t claim this as a unique skill of the physicist. Whilst over a hundred years ago Rutherford could quite reasonably(?) claim that “all science is either physics or stamp collecting” , the world has moved on. The rise of anatomy, physiology and particularly biochemical biology since that time mean that there are now underlying theoretical frameworks that we can use to explain the results of clinical and health sciences research.

We very rarely do though and I think this is partly attributable to education of doctors and allied health professionals being rooted in an earlier era. It wasn’t so very long ago that most results of clinical research where, effectively, beyond explanation. There was no point trying to fit those results into any underlying theoretical framework because the basic principles of that framework had not been established.  Knowledge in the clinical domain was essentially phenomenological – a knowledge of what happens rather than why it happens. Education then becomes a matter of teaching the facts rather than the underlying principles that link those facts. Of course if you don’t have an underlying theory to work from then you are going to find it much more difficult to generate sensible questions to drive the next generation of research. This is, of course, exactly the point that Morris was making and links to my post from last week.

As we move out of that era though we’ve got to put a much heavier emphasis on developing the underlying theoretical basis of our subject and using this to drive our research questions. Which leads me to my highlight of the ESMAC-SIAMOC conference which was Adam Shortland’s key-note talk, “The neuromuscular prerequistes of normal walking and the early loss of ambulation in cerebral palsy“. In it he reviewed what is now known about neurophysiological development and laid out a conceptual framework that explains much of what we observe in cerebral palsy and also provides a platform to generate new research questions … but then if you look at his CV you’ll see that he trained first as a physicist as well!



Training our PhD students to ask questions

This afternoon I give an annual lecture about what a PhD is. It is designed as part of the orientation for new PhD students and as a time for reflection to those already well established in their studies. It’s interesting for me to think about this talk after my post last week about how few of the papers I read or hear presented advance my understanding of a particular area.  Another way of looking at this is to comment on the number of papers and particularly conference presentations I see which don’t, to me, appear to contain a clearly formulated or insightful research question. Clearly if your research isn’t driven by a clear question then it is extremely unlikely that it is going to deliver a clear answer.

As research progresses, the fields within which we all function get more specialised and complex. Most of the obvious, simple questions have already been asked. This leads to a large choice of less obvious, derivative questions. Not only does productive research need to ask a relevant and interesting question but it needs to ask one that has the potential to be answered by the techniques that are available. As science progresses there are more and more techniques to choose from and more skill required in selecting the correct one (as in Kat Steels’ prize winning paper at ESMAC). In summary it would be quite reasonable to state that the principle skill that a research leader needs in the modern environment is the ability to think up well-formulated, interesting and answerable research questions

A PhD is many things but amongst these it serves as an entry level qualification for our future research leaders. The process  has changed markedly since I obtained mine (awarded almost exactly twenty years ago). I was chatting about this the other day with a colleague who was remembering completing hers with an absolute minimum of supervision. She commented on how this equipped her with a degree of self-sufficiency and independence that the current system doesn’t always deliver. I shared that experience, and on the one hand agree that it nurtured (inflicted?) independence. On the other, it meant that there are whole areas that I now consider essential for a PhD student to master that I didn’t even touch. No-one ever suggested I should have a systematic approach to searching the literature (at that time there wasn’t a great deal of literature to search). I don’t remember any consideration of the ethics of my research despite the fact that I was modifying external fixation frames that were being applied to patients in the local fracture clinic. There is no statistical analysis anywhere in my thesis. None of it ever got published. Perhaps most importantly of all, given the focus of this post, there is no formal statement of a specific research question. I’m glad that the 30 year old me didn’t have the 50 year old me as a PhD examiner, because I would have failed.

Given the complexity of research in the modern world, I don’t think we have any choice but to have highly structured PhD programmes. There simply isn’t time to allow students to do their own thing anymore. But there is a danger that, in cramming a PhD syllabus full of all the detail of how to systematically review, apply for ethics, master complex measurement techniques, perform rigorous statistical analyses and submit papers, we lose focus on the core research skill of asking a relevant,  insightful and answerable question.

This can be exacerbated by many PhD studentships these days being offered on the basis of funding to conduct a particular project. In these the research question may have been decided before the student even arrives. Sometimes only a general area has been identified, but even then supervisors will often have very strong views about the direction that research should take. I must admit to being guilty of this myself. If I’ve worked hard to get money to support a PhD studentship or am investing my time in supervision, I want to be certain that the resulting research answers the questions that I’m interested in rather than dreaming up their own.

And then of course there are pressures from the system. Universities in the UK are now judged, in part, by how many PhD students complete within the equivalent of 4 years full time study. To ensure this my own University asks the student to submit a Learning Agreement within 3 months, an Interim Assessment within one year and an Internal Evaluation including substantial drafts of thesis chapters before the end of the second year. The pressure is for them to hit the deck running. By far the easiest way to ensure this is to tell students what their research question is and let them to get on with it.

The one thing that is perhaps helping is the drift away from the classic PhD, which reported one large study, to the modern standard which reports several smaller ones. On this model the student gets the opportunity to design a number of studies. Each one will give practice in formulating a relevant, insightful and answerable research question. I wish students who are facing this challenge well and look forward to hearing them present their answers at the future conferences I attend.