UPDATE ON THE PACE TRIAL
Professor Malcolm Hooper
11th July 2012
Many people have submitted formal complaints about the flawed methodology of the £5 million publicly-funded PACE Trial; these include complaints about the with-holding of the recovery statistics, the mis-reporting of the results by Principal Investigators themselves and by the media through the auspices of the Science Media Centre, and the apparent manipulation of the raw data, widely believed to be an attempt by the Investigators to salvage a trial that could not be allowed to fail.
Those complaints have been made to the Medical Research Council (MRC), whose Head of Corporate Governance and Policy, Dr Frances Rawle, chose not to address the issues raised and then to mock the complaint (BMJ 22nd June 2011); to The Lancet, whose editors admitted in writing that its erroneous reporting of the trial results must be corrected but almost 18 months later have still not done so and who – against the Elsevier complaints protocol -- dismissed a formal complaint entirely; to the Secretary of State (who referred it back to the MRC); to the editors of numerous newspapers who mis-reported the trial results (the way in which ME/CFS patients have been vilified in the press has just been put before Lord Leveson’s Inquiry by the Neuroimmune Alliance), and to the Royal Statistical Society, who enthusiastically requested an analysis of the statistics but then refused to publish it but provided no reason.
All complaints and concerns were ignored or dismissed and met with determined refusal to address the issues raised.
The redoubtable Countess of Mar therefore tabled a number of questions on 9th May 2012 and replies to her questions were provided by Baroness Wilcox, Parliamentary Under-Secretary of State in the Department for Business, Innovation and Skills (the Department responsible for the MRC).
Those replies were widely regarded as being unsatisfactory and Lady Wilcox was clearly anxious about the situation; following discussions between Lady Mar and Lady Wilcox, the latter offered Lady Mar a meeting with BIS officials to discuss the concerns, which Lady Mar accepted. It was anticipated that four prominent people from the ME community would accompany Lady Mar to that meeting with officials from the MRC and BIS.
Consequently, the attached document “Briefing Notes for meeting with BIS officials about incorrect answers to Parliamentary Questions re: the MRC-funded PACE Trial and ME/CFS” was compiled.
Given that it was accepted that BIS officials would have little knowledge of the situation, the document was written in plain English and was in five easy-to-understand sections: (i) Objectives of meeting with officials from the Department for Business, Innovation and Skills (BIS); (ii) Essential background information about ME/CFS (iii) Problems with the replies to Parliamentary Questions (PQs) of the Parliamentary Under Secretary of State in the Department for Business, Innovation and Skills (Baroness Wilcox); (iv) Problems with the Medical Research Council’s role in the PACE Trial and its repeated denial of accountability and (v) Failure of the PACE Trial Principal Investigators (PIs) to report primary outcome measures as set out in the Trial Protocol; evidence of misrepresentation of the data and evidence of unacceptable selectivity in the results of the trial published in The Lancet.
However, on 21st June 2012, Lady Wilcox informed Lady Mar that the BIS officials were “concerned” about Lady Mar’s “associations” and suggested that she and Lady Mar should have tea together to enable her to get a handle on the problem before the full meeting with BIS officials.
Before any such discussions could take place, on 25th June 2012 Jamie Ballantyre, Assistant Private Secretary to Baroness Wilcox, wrote to the Countess of Mar asking if she would be available to meet Baroness Wilcox, Officials from the MRC and Officials from BIS at 16.30 on 12th July 2012. The letter was specific: “This meeting would be to discuss MRC funding opportunities for CFS/ME and the actions the MRC has taken to try to build capacity in this area…It will not be possible for the technical details of the trial or interpretation of the data to be discussed”. The reasons given were (i) “The research itself has been conducted independently of government and results of the trial, following peer review, have been published (ii) Although funding for the trial was provided by the MRC, this decision to fund would have been made based on peer review. In line with the Haldane Principle of scientific independence, the department would not and should not have any influence on this process (iii) None of the officials who will be present have the relevant technical expertise and knowledge”. The letter continued: “It is important to note that there are channels for challenging scientific results through journals and other publications, speaking at conferences, further research work etc”.
Not only was the DWP a co-funder of the PACE Trial (so it cannot be argued that the trial was independent of government, which has a strong vested interest in getting people with ME/CFS off state benefits and back to work, this being the non-clinical rationale for the “clinical” trial), it has been acknowledged that the PACE Trial protocol that was published in BMC Neurology on 8th March 2007 (BMC Neurology 2007, 7:6 doi:10.1186/1471-2377-7-6) was not peer-reviewed by the journal before publication (“This study protocol was not peer reviewed by the journal because it had already received ‘ethical’ and funding approval by the time it was submitted….We strongly advise readers to contact the authors or compare with any published results article(s) to ensure that no deviations from the protocol occurred during the study” -- Editor’s comment 31st January 2007: http://www.biomedcentral.com/imedia/2095594212130588_comment.pdf ).
In the case of the PACE Trial, The Lancet’s peer review process has patently failed, since no non-biased peer-reviewer would have approved such a significant deviation from the published Protocol including the highly selective publication of results, the abandonment of primary end-points and the shifting of goal posts such that the post-hoc “normal range” overlapped with the entry criteria.
It seems remarkable that officials from the MRC would not have the relevant knowledge to discuss very basic concerns about the methodology of one of its own clinical trials or its own role in that trial.
Moreover, the Haldane Principle (ie. that decisions about how research funds should be spent should be made by researchers and not by politicians) has nothing to do with the issues raised: the key issue is that BIS is responsible for the MRC’s failure to adhere to elementary rules of scientific procedure that occurred in the PACE Trial and as a result of the MRC’s failure, sick people continue to be put at risk of iatrogenic harm.
What is at stake here is the fact that the PACE Trial is scientifically flawed and the results have been misrepresented so NICE, insurance companies, the DWP and private companies contracted by the government (including Atos) are relying on false interpretation of the data to the serious detriment of very sick people.
Lady Mar’s response was that such a meeting would “get us nowhere” and that she would meet Baroness Wilcox privately and explain the situation to her.
A private meeting between the Countess of Mar and Baroness Wilcox took place on 11th July 2012, at which Lady Wilcox agreed that the best way to deal with the situation would be a debate in the House of Lords; this will be arranged for October when the House returns after the recess.
At the conclusion of that meeting, Lady Mar gave Lady Wilcox a copy of the attached Briefing Notes, so there can be no argument that she did not receive a copy.
These facts are being made publicly known in accordance with the government’s keenness for transparency and accountability.
Briefing Notes for meeting with BIS officials about incorrect answers to Parliamentary Questions re: the MRC-funded PACE Trial and ME/CFS
Professor Malcolm Hooper
This document is in five parts: (i) Objectives of meeting with officials from the Department for Business, Innovation and Skills (BIS); (ii) Essential background information (iii) Problems with the replies to Parliamentary Questions (PQs) of the Parliamentary Under Secretary of State in the Department for Business, Innovation and Skills (Baroness Wilcox); (iv) Problems with the Medical Research Council’s role in the PACE Trial and its repeated denial of accountability and (v) Failure of the PACE Trial Principal Investigators (PIs) to report primary outcome measures as set out in the Trial Protocol; evidence of misrepresentation of the data and evidence of unacceptable selectivity in the results of the trial published in The Lancet.
The selectively reported results of the PACE Trial have serious implications for people with ME (myalgic encephalomyelitis) and erroneous replies by Baroness Wilcox further compound the problem.
Whilst the PACE Trial report published in The Lancet in March 2011 remains widely acclaimed by the authors and the press on the basis that 30% of participants achieved a “normal range” of functioning after undergoing directive psycho-behavioural therapy, it should be noted that 70% cannot be claimed to have achieved any improvement. If such statistics involved a drug, then the drug would not be promoted nationwide as the treatment of choice.
The post-intervention improvements achieved by participants were so small on the six minute walking test distance that the scores were worse than the activity levels of those considered seriously diseased (eg. people with heart disease or those awaiting lung transplant).
It is unclear what disorder was being investigated and the trial is replete with significant changes in benchmarks, making meaningful interpretation impossible.
There is misuse of terminology such that terms used by the investigators have been misconstrued, which has had the effect of augmenting the claimed success of the trial and misrepresenting the results.
The interventions promoted by the study can cause serious harm to ME patients and the article in The Lancet should be withdrawn, since both the investigators and the publishers failed to meet research standards requirements.
Following what were widely regarded as unsatisfactory answers to written parliamentary questions (PQs) about the PACE Trial on “CFS/ME” tabled by the Countess of Mar on 9th May 2012, Baroness Wilcox offered the Countess of Mar a meeting with officials to discuss the continuing concerns about the PACE Trial, as BIS is responsible for the MRC which, together with the Department of Health, the Department for Work and Pensions and the Scottish Chief Scientist’s Office, was a co-funder of the PACE Trial.
The primary aim of the meeting with BIS officials must be to bring to their attention not only the evidence that Baroness Wilcox’s replies to the Countess of Mar were misguided, evasive and in some cases erroneous, but also the many serious problems enveloping the PACE Trial and the unresolved concerns about the MRC’s role in continuing to support the published outcome of that trial. Unless BIS is aware of these substantial problems, then its officials will not be in a position to implement effective intervention in order to prevent further iatrogenic harm caused by the failure of the MRC to fulfil its obligations.
In its Terms and Conditions relating to its grants, MRC-funded authors have a responsibility to report accurately and without obfuscation, and the MRC requires grant-holders to adhere to its policy on data-sharing which is built on the OECD report “Promoting Access to Public Research Data for Scientific, Economic and Social Development”. That report identified that publicly-funded research data are “a public good, produced in the public interest and should be openly available to the maximum extent possible”. The MRC specifically states that it expects “valuable data arising from MRC-funded research to be made available to the scientific community with as few restrictions as possible so as to maximise the value of the data for research and for eventual patient and public benefit” and that such data “must be shared in a timely and responsible manner”. It also states: “Our data-sharing policy applies to all MRC-funded research”; and it requires that results from this data-sharing “should meet the high standards of all MRC research regarding scientific quality, ethical requirements and value for money”.
In the case of the PACE Trial, substantial evidence exists that the MRC has approved and condoned serious breaches by the Principal Investigators (PIs) of its own Terms and Conditions relating to its grants, as there are numerous examples of significant and serious mis-reporting and non-reporting of the PACE Trial results in The Lancet (PD White et al. Lancet 2011:377:823-836).
In the interests of the UK’s international reputation for scientific research and the safety of patients with ME/CFS, the PACE Trial article published in The Lancet should be withdrawn and the raw data re-analysed by independent (ie. non-MRC) statisticians.
Because the MRC has repeatedly declined to heed the many concerns about the PACE trial submitted by Parliamentarians, clinicians, medical scientists, ME charities and patients alike, it is the responsibility and the duty of BIS to address these issues with expediency.
(ii) Essential background information
Some of the proven and published organic abnormalities in ME/CFS include evidence of:
There is irrefutable evidence that ME/CFS is not “medically unexplained fatigue” that is perpetuated by aberrant illness beliefs, pervasive inactivity, membership of a self-help group, hypervigilance to normal bodily sensations or being in receipt of disability benefits, as claimed by the Wessely School.
Because it misrepresented the disease ME/CFS, the whole PACE Trial was based on the myth that it is a behavioural disorder, and this myth continues to be condoned by the MRC.
There was never any realistic hope that the PACE Trial could be successful in restoring those with ME/CFS to health and employment: the FINE Trial (Fatigue Intervention by Nurse Evaluation, a sibling of the PACE Trial that was wholly funded by the MRC in which nurses delivered pragmatic rehabilitation and supportive listening to severely affected home-bound patients) also failed.
Indeed, on 18th April 2011 one of the PACE Trial PIs (Professor Michael Sharpe) conceded in an ABC (Australian) broadcast: “What this trial wasn’t able to answer is how much better are these treatments than really not having very much treatment at all”.
This is a view that the Chief PI, Professor Peter White, seems not to share, since at the BACME conference on 14th/15th March 2012 he claimed that the PACE Trial was a “magnificent achievement”.
Despite Professor White’s inflated claim, the international evidence is that the premise upon which the PACE Trial was predicated is scientifically untenable and that the directive (as opposed to supportive) psychological interventions used in the trial are potentially harmful to patients.
Notwithstanding, NICE continues to recommend those same interventions in its Clinical Guideline on “CFS/ME” and at the 2012 BACME conference, Professor White accused ME patients and charities of influencing the NHS clinics against his recommended extensive use of CBT/GET, asserting that their negative response to his PACE Trial was actively harming patients by creating a nocebo effect (nocebo meaning “I shall harm”).
The PACE Trial was not only a tragedy for patients but was a travesty of science since, with the approval of the MRC, even elementary rules of scientific procedure were abandoned wholesale (see section (iv) below).
(iii) Problems with the responses to PQs by Baroness Wilcox
Baroness Wilcox replied that there is a requirement for publicly funded clinical trials to be registered; that the MRC has been a strong supporter of trials registration for many years and has provided financial support to help set up the ISRCTN scheme and was involved in promoting its widespread adoption in the UK. She further stated that the MRC requires that all MRC-funded clinical trials comply with CONSORT (Consolidated Standards of Reporting Trials), stating that the PACE Trial would have had to meet this standard as a prerequisite for publication in The Lancet. She also said that the MRC is not responsible for assuring the quality of data in the ISRCTN and that the Government cannot comment on the completeness of the data.
The evidence is that the PACE Trial was incompletely and incorrectly registered in the ISRCTN and did not conform to CONSORT and hence is in breach of MRC requirements (as well as The Lancet’s own requirement about correct registration).
As stated by Baroness Wilcox, CONSORT “offers a standard way for authors to prepare reports of a trial findings, facilitating their complete and transparent reporting, and aiding their critical appraisal and interpretation”. However, the PACE Trial failed to comply with CONSORT, so such critical appraisal and interpretation is not possible.
It is a matter of record, confirmed by The Lancet, that the editor responsible for publication of the PACE Trial article took Professor White’s article “on trust”; furthermore it has been confirmed by the editor responsible for publication that the article was not sent to The Lancet’s statisticians for checking before publication (for the importance of which, see below).
The Lancet may have assumed that as the MRC “...has been a strong supporter of trials registration for many years and has provided financial support to help set up the ISRCTN scheme and was involved in promoting its widespread adoption in the UK” any MRC-funded PACE Trial complied with both ISRCTN and CONSORT and, further, as it is an MRC requirement for its research to comply with ISRCTN and CONSORT, that the MRC must have a mechanism in place to ensure conformity before allowing any MRC-funded paper to be put forward for publication (otherwise what is the point of the MRC’s requirement for conformity?), yet in the case of the PACE Trial, neither of these conditions was fulfilled by the PACE Trial Investigators.
Equally, clinicians and medical researchers around the world may assume that MRC-funded research complies with its published requirements, but in the case of the PACE Trial this is an erroneous assumption.
In 2004 the New England Journal of Medicine published an Editorial (co-authored by The Lancet’s editor-in-chief Richard Horton) which addressed the need for registration of clinical trials: “In return for the altruism and trust that make clinical research possible, the research enterprise has an obligation to conduct research ethically and to report it honestly….Unfortunately, selective reporting of trials does occur….The case against selective reporting is particularly compelling for research that tests interventions that could enter mainstream clinical practice….If all trials are registered in a public repository at their inception, every trial’s existence is part of the public record….The International Committee of Medical Journal Editors (ICMJE) will require, as a condition of consideration for publication, (comprehensive) registration in a public trials registry….The registry must be accessible to the public at no charge….There must be a mechanism to ensure the validity of the registration data….Registration is only part of the means to an end; that end is full transparency with respect to performance and reporting of clinical details” (NEJM 2004:351:12:1250-1251).
On 3rd October 2009 Dr Ben Goldacre commented in The Guardian on a paper published in the Journal of the American Medical Association (JAMA 2009:302(9):977-984) about the need for adequate registration of clinical trials: “…in the absence of this full information, people are subjected unnecessarily to side effects….We also know that researchers can change their stated goal, or ‘primary outcome’, after their trial has finished….These problems are supposed to have been fixed by clinical trials registers: before you start your trial you publish the protocol, saying what your primary outcome is, how many people are in your trial…and so on. Then, by looking at the protocol and the finished paper…people can see…if you have misled them by changing your primary outcome. This only works if it is enforced….They…found repeated discrepancies between the outcomes stated at registration and the outcomes published in the final paper…In almost all papers where it was possible to assess a switch, a duff outcome was switched out in favour of one that showed a positive finding….Our failure to ensure full, undistorted publication of all trial data is the single most important issue in medicine today, because this is the only way we can know whether a treatment does good, or harm”.
Baroness Wilcox failed to hold the MRC (which she claims is a “strong supporter” of registration) to account for condoning false and incomplete registration in the PACE Trial, which entirely vitiates the purpose of the ISRCTN clinical trials register (ie. to protect vulnerable patients from inappropriate or harmful interventions).
Question 2 HL/69 (this question was initially omitted by the Clerk so had to be re-tabled). The Countess of Mar asked which disease or condition was being studied in the PACE Trial in the light of the statement made by the Chief Principal Investigator, Professor Peter White, that the PACE Trial did not purport to be studying CFS/ME but only fatigue, when all the trial documentation, ethical approval and funding refer to CFS/ME.
Baroness Wilcox entirely failed to answer this question, saying simply: “The criteria for the PACE study were published in the trial protocol and are also addressed in the main findings published in The Lancet”.
The whole point of the question was that, following publication of selective results from the PACE Trial in The Lancet, Professor White denied in writing what had been published in both the Trial Protocol and in The Lancet; he wrote to the editor-in-chief of The Lancet stating: “The PACE trial paper refers to chronic fatigue syndrome (CFS)…it does not purport to be studying CFS/ME but CFS defined simply as a principal complaint of fatigue that is disabling, having lasted six months, with no alternative medical explanation (Oxford criteria)”. Since this is at variance with the trial documentation and with what was published in The Lancet, it was this issue that Baroness Wilcox was required to address.
Since the entry criteria (the Oxford) were the Investigators’ own and by definition excluded people with neurological diseases such as ME, it is impossible to know how many participants fulfilled the criteria for ME and how many suffered from a principal complaint of chronic fatigue. This means that contrary to the claims of the PIs, the results cannot be extrapolated to those with the neurological disorder ME, yet this is exactly what is happening, to the detriment of people who have genuine ME.
Given that the MRC sees nothing wrong in this situation, as the MRC is accountable to BIS, it now falls to BIS to address this significant disparity concerning an MRC-funded clinical trial that cost taxpayers £5 million but which does not clarify which disorder the PIs were studying or to which patients CBT and GET may be safely applied.
Question 3 (22nd May 2012/HL44) The Countess of Mar asked when and why the PACE Trial sponsor was changed during the course of the trial and why these changes were not recorded in the ISRCTN.
Baroness Wilcox replied that “Queen Mary, University of London, has been identified as the formal Sponsor of the PACE trial throughout the duration of the study”.
This answer does not accord with the original registration details which are still accessible via electronic archive tracking, nor with the removal of previous entries from the ISRCTN, which likewise are accessible via electronic archive tracking. There are no records of the original sponsor (the MRC Clinical Trials Unit), nor of the change of sponsor from the MRC Clinical Trial Unit to the MRC itself before being changed to Queen Mary, London, on the current entry, which should include these changes and the reasons for them.
Baroness Wilcox’s reply does not accord with the fact that the sponsors of the PACE Trial are referred to in The Lancet article as being the Medical Research Council, the Scottish Chief Scientist’s Office, the Department of Health in England and Wales and the Department for Work and Pensions.
The reason for this question was because the WHO International Clinical Trials Registry Platform (whose Scientific Advisory Group includes The Lancet’s Dr Richard Horton) that is linked to the major ISRCTN Register requires that those responsible for completing the Register “must not have conflicts of interest over which trials or trial information to register” and must “collect full Trial Registration Data Set”) and the change of sponsor may have been related to a conflict of interest on the part of one of the UK Directors of the ISRCTN responsible for the registration and tracking of changes to the trials register (Dr Chris Watkins of the MRC Neurosciences and Mental Health Board, of which Professor Simon Wessely was a member and on which two of the PACE Trial PIs, Professors Peter White and Trudie Chalder also served).
Watkins (whose title was MRC Programme Manager for Research on Mental Illness and Drug Addiction) authored the 2003 MRC “CFS/ME Research Strategy” by its Research Advisory Group (RAG), which stated: “The remit of the Research Advisory Group was not to review the existing body of knowledge but …(to) propose a research strategy….The Research Advisory Group acknowledges that the descriptive term ‘CFS/ME’ does not refer to a specific diagnosis….The Research Advisory Group considers that…it is not an essential prerequisite to identify…causal pathways in order to undertake research on CFS/ME. The MRC Research Advisory Group considers it appropriate to explore potential interventions for CFS/ME in the absence of knowledge of causation….The Research Advisory Group has not undertaken a full literature or systematic review of the published literature on CFS/ME. Such an undertaking would have been a substantial undertaking that would have taken a significant length of time…It is the firmly held belief of the Group that psychiatric illnesses are no less real… than neurological illnesses….Many reported findings in the area of pathophysiology are not published in the peer-reviewed literature….The MRC Research Advisory Group has not undertaken a detailed review of the current level of scientific knowledge..of CFS/ME….The MRC CFS/ME Research Advisory Group has… chosen to consider how the evidence-base for potentially effective management options can be strengthened…Two specific strategies were identified of potential benefit for CFS/ME, graded exercise therapy (GET) and cognitive behavioural therapy (CBT)”.
Commenting on this MRC report, Hooper et al noted that it is the duty of the MRC to identify causal pathways (the MRC is a Council for “medical research”); that RAG members had ignored elementary rules of procedure concerning the prerequisite for awareness of the knowledge already established about the disorder; that by December 2002, 92 mainstream peer-reviewed journals had published pathophysiological findings on ME/CFS, and that the MRC’s report seemed to be an attempt to curtail the advancement of medical science (Hooper M et al. Ignoring the Evidence? 5th May 2003 www.meactionuk.org.uk/ignoring_the_evidence.htm ).
Clearly Watkins had a significant conflict of interest as Director of the UK ISRCTN Register as his support for the PIs and for psychologically-based research in ME/CFS is a matter of record.
The PACE Trial Steering Committee Minutes of 22nd April 2004 record: “The MRC’s change of policy regarding trial sponsorship was noted… It was noted that the MRC will no longer be the sponsor of the trial and that this needed to be clarified”.
Baroness Wilcox’s reply fails to address any of these important issues that may have a significant bearing on the PACE Trial and on the MRC’s unquestioning support for it.
Question 4 (22nd May 2012/HL45) The Countess of Mar asked why the recovery statistics and other outcomes as defined in the PACE Trial Protocol have not been published as required.
Baroness Wilcox replied that it is the responsibility of the investigators and the relevant journals to determine how and when the findings are published. She went on to state: “The MRC understands that further publications are planned, one of which will address the issue of recovery”.
This is an unacceptable answer. The whole point of trial registration is that accurate and complete information as set out in the Trial Protocol (especially the primary outcome measures) is available to interested parties and in the case of the PACE trial, it is not available in the clinical trials register nor has it been published as required. It is the duty of BIS to hold the MRC to account for permitting such laxity in an MRC-funded trial.
Question 5 (22nd May 2012/HL46) The Countess of Mar asked why the “normal range” for the two primary outcomes (fatigue and physical function) were re-defined so that it was possible for a participant to deteriorate on both measures during the trial yet still fall within the Chief Principal Investigator’s re-defined “normal range”, and what impact HMG considers this re-definition had on the validity of the trial.
Such an outcome of an MRC-funded trial is illogical, but Baroness Wilcox entirely failed to answer this question.
Question 6 (22nd May 2012/HL47) The Countess of Mar asked in the light of the non-conformity of the PACE Trial with the ISRCTN Register requirements, what is the position of the MRC as co-funder of the trial regarding the subsequent reliance by NICE and the DWP on the outcome as reported in The Lancet.
Baroness Wilcox replied that the study “aimed to evaluate treatments that were already in use, and for which there was insufficiently strong evidence to support their effectiveness” and that the MRC does not have a position on how the outcome of MRC-funded studies are interpreted and used by regulators or policy makers. She went on to state: “The MRC strongly supports the publication of the findings of all MRC funded research….(and) it supports prompt publication of its research findings”.
This is an unacceptable response: the first part is wrong, since Adaptive Pacing Therapy (one of the arms of the trial) was completely new and the protocols for CBT and GET were written specifically for the PACE Trial and were designed to disabuse participants’ (correct) belief that they suffer from an organic neuroimmune disorder in that the cognitive behavioural therapy involved was directive (and was intentionally not supportive, as is the CBT offered to help people cope with serious illness).
On the one hand Professor White was strongly promoting CBT/GET in his submissions to NICE in support of its Guideline that recommended only CBT/GET as interventions in CFS/ME because he asserted that there was sufficient evidence of their efficacy for their implementation across the nation, yet at the same time he was receiving £5 million of taxpayers’ money to carry out his PACE Trial because he claimed that there was not sufficient evidence of the efficacy of the same interventions.
The second part of her reply (that the MRC does not have a position on how the outcome of MRC-funded studies are interpreted and used by regulators or policy makers) essentially means that investigators can mis-report their findings and withhold their data at will because the MRC takes no responsibility for checking, and such misrpresentation of the data can then be relied upon and used with impunity by Government bodies.
If this is so, who is responsible for ensuring that selective and misleading data is not published, as is the case with the PACE Trial? The fact that the MRC Biostatistics Unit at Cambridge is a prestigious unit ought not to mean that its analysis of the data cannot be scrutinised.
In The Lancet article, the PIs presented their selected data with such complexity and in such a convoluted way that it may be construed as their attempt to hide the fact that the PACE Trial results were disappointing. One can only conclude that Professor White endeavoured to disguise the poor results in the wealth of data presented, because the detail in the published figures serves to obscure the fact that the reported “improvements” are miniscule, and for those results to have been widely proclaimed as even “moderately successful” must border on impropriety.
Baroness Wilcox seems to be missing the whole point of the question: according to its own Terms and Conditions for grant-holders, the MRC has a responsibility to ensure that the research it funds achieves the required standards so that clinicians and policy makers have accurate data to use safely for the benefit of patients and to ensure that the data is widely available, no matter how disappointing the outcome.
Organisations like NICE and the DWP, and indeed researchers and organisations from around the world, will assume that PACE does comply with these MRC requirements and will be acting on that basis, making decisions on further ME research, treatments, disability benefits and insurance matters accordingly, decisions which have a profound effect on people’s health and lives, when in the case of the PACE Trial, the PIs did not comply with MRC requirements.
The third part of Baroness Wilcox’s reply does not accord with her reply to HL/45 (“The MRC understands that further publications are planned, one of which will address the issue of recovery”). For the recovery statistics not to have been published over one year after publication of some of the results in The Lancet is not in accordance with “prompt” publication of research (the MRC “supports prompt publication of its research findings” – see HL47 above) and leads to speculation that no-one recovered: if people had indeed recovered, then it is reasonable to expect those statistics to have been up-front in The Lancet article that claimed success of the PACE Trial.
Since £5 million of taxpayers’ money is involved, clearly HMG should have a position on how the outcome of an MRC-funded study is used by other Government agencies.
The PQs were structured with precision and the lack of credible answers is evidence of HMG’s non-compliance with its own statutory requirements.
(iv) Problems with the MRC’s role in the PACE Trial
The MRC is charged with requiring high standards and scientific excellence in the projects it funds and it is the duty of BIS to hold the MRC to account for its demonstrable failure to adhere to even the most elementary rules of procedure regarding the PACE Trial.
The PACE Trial was flawed at every stage
Before considering the misrepresentation of the PACE Trial results and the inconsistencies in the selective published outcome of the PACE Trial, it necessary to consider some of the problems inherent in the trial itself, such as the flawed methodology, to which the MRC ought to have objected, for example:
1. The Investigators failed to take account of the extant literature about the disorder in question -- a serious issue in a clinical trial and in clear breach of the Declaration of Helsinki: B11 states: “Medical research involving human subjects must conform to generally accepted scientific principles (and) be based on a thorough knowledge of the scientific literature”. By allowing their own bias and beliefs to underpin the whole trial and by their ignoring of the extant biomedical evidence, the PIs failed to incorporate safeguards such as pre-exercise and post-exercise objective checking of immune parameters, maximal oxygen uptake, cardiovascular screening, respiratory function tests etc, all of which have been shown to be abnormal in ME/CFS. This is particularly disturbing, given that in 2004 the Chief PI himself published the following: “Immunological abnormalities are commonly observed in CFS…Concentrations of plasma transforming growth factor-beta (TGF-b) (anti-inflammatory) and tumour necrosis factor-alpha (TNF-a) (pro-inflammatory) have both been shown to be raised….Abnormal regulation of cytokines may both reflect and cause altered function across a broad range of cell types…..Altered cytokine levels, whatever their origin, could modify muscle and or neuronal function….Concentrations of TGF-b1 were significantly elevated in CFS patients at all times before and after exercise testing….We found that exercise induced a sustained elevation in the concentration of TNF-a which was still present three days later, and this only occurred in the CFS patients….TGF-b was grossly elevated when compared to controls before exercise (and) showed an increase in response to the exercise entailed in getting to the study centre….These data replicate three out of four previous studies finding elevated TGF-b in subjects with CFS….The pro-inflammatory cytokine TNF-a is known to be a cause of acute sickness behaviour, characterised by reduced activity related to ‘weakness, malaise, listlessness and inability to concentrate’, symptoms also notable in CFS….These preliminary data suggest that ‘ordinary’ activity (ie. that involved in getting up and travelling some distance) may induce anti-inflammatory cytokine release (TGFb), whereas more intense exercise may induce pro-inflammatory cytokine release (TNF-a) in patients with CFS” (emphasis added; Immunological changes after both exercise and activity in chronic fatigue syndrome: a pilot study. White PD, KE Nye, AJ Pinching et al. JCFS 2004:12 (2):51-66). No such post-exercise changes were assessed in the PACE Trial.
2. Entry criteria were chosen that did not define the population allegedly being studied, namely the PIs’ own criteria (the 1991 Oxford criteria, which were financially supported by the PACE Trial’s Chief Principal Investigator) that select only chronically fatigued patients and exclude those with neurological disorders. The Trial Identifier stated that the Oxford criteria were intentionally chosen in order to catch as many people with “chronic fatigue” as possible, but the Oxford criteria have never been adopted internationally; there is no consensus about them; they are used only in Britain and only by the Wessely School; they lack diagnostic specificity, have been shown to have no predictive validity, and select a widely heterogeneous patient population. It is virtually unheard of for studies to use criteria that have been superseded (one of the PIs, Professor Michael Sharpe – who was lead author of the Oxford criteria -- stated in 1997 that the Oxford criteria “have been superseded by international consensus”. Particularly notable is the fact that the Oxford criteria do not even mention, let alone require, the cardinal feature of ME (post-exertional physiological exhaustion and malaise) to be present in participants in a trial that purported to be studying that disorder, yet in his letter to The Lancet published on 28th May 2011, the Chief PI stated that post-exertional fatigue is characteristic of “CFS”. The fact that a group of psychiatrists who do consultancy work for the permanent health insurance industry does not accept that ME is a neurological disorder is insufficient reason for them to use superseded criteria as entry for an MRC-funded trial, or for the MRC to acquiesce in this departure from good scientific practice.
3. Whilst declining to carry out any subgrouping of “CFS/ME” (which would not accord with their intention to include as heterogeneous a “fatigued” population as possible), the PIs claimed to have carried out a secondary analysis of the data by using criteria that do not officially exist (their own modification of the “London” criteria which, like the Oxford criteria, do not require the cardinal feature of ME to be present in patients in a trial which purported to be studying that disorder). Had the trial entry criteria been rigorously applied, then no amount of “secondary analysis” would reveal those with ME.
4. The PACE Trial intentionally included participants who did not suffer from the disorder supposedly being investigated: this was confirmed on 12th May 2004 by Parliamentary Under Secretary of State at the Department of Health, Dr Stephen Ladyman, at an All Party Parliamentary Group on Fibromyalgia (FM), who announced that doctors were being offered financial inducements to persuade patients with FM to attend a “CFS” Clinic to aid recruitment to the PACE Trial (the PIs were granted more money and more time to achieve the set recruitment levels, and an additional Trial Centre had to be opened at Frenchay Hospital, Bristol, which began recruiting in April 2007).
5. The instrument chosen to measure fatigue (the much-criticised Chalder/Wessely Fatigue Questionnaire) was problematic because its inability to measure deterioration in this patient cohort biased the PACE Trial in favour of finding “improvements” in fatigue since exercise-induced relapses cannot be recorded. The Chalder Fatigue Questionnaire does not measure the key symptom of ME/CFS (post-exertional exhaustion and malaise). In simple terms, if a participant already has a maximum score at the start of an intervention (such as graded exercise), then even if the participant feels worse and is actually worse at the end of the intervention, their total score on the Chalder Fatigue scale cannot increase, so there is no evidence that they have been made worse by the intervention. In other words, people cannot be shown to “get worse” on the Chalder Fatigue Scale even if they feel -- and are -- worse. To many people, it is incomprehensible how such a method of assessment could be deemed scientific when assessing those with ME/CFS, but the MRC Data Monitoring and Ethics Committee apparently had no problem agreeing to its use in the PACE Trial.
6. It is a basic rule of any clinical trial that participants are not told during the trial how effective is the intervention that they are receiving. It should never be suggested to trial participants that the intervention they are undertaking is a cure unless it is certain that it is indeed curative, in which case there would be no need for a clinical trial to prove the efficacy of the intervention. Whilst the PACE Trial was in progress, the Chief PI published his beliefs about the interventions being used: “recovery from CFS is possible following CBT….Significant improvement following CBT is probable and a full recovery is possible” (PD White; Psychother Psychosom 2007:76(3):171-176). To mislead participants by suggesting that a cure can be expected when there is no such certainty is in breach of the General Medical Council Regulations as set out in “Good Medical Practice” (2006). To advise trial participants that they can recover from ME/CFS if they would only follow the psychiatrists’ regime of CBT/GET offers false hope: the recovery statistics for ME/CFS simply do not support such a belief and, as noted above, at least 20 international experts (many of professorial status) have expressed their written concern about Professor White’s pronouncements. Moreover, by overtly favouring the CBT/GET arms of the trial but not the other two arms (adaptive pacing and standard medical care), the PIs may have introduced bias and may have invoked the placebo response by putting pressure on participants to report a positive outcome.
7. During the trial, participants were actively encouraged to give glowing reports about the PACE Trial to their friends and contacts in order to encourage those friends and contacts also to enter the trial. To do so is unethical, but that is what happened in the PACE Trial and it was condoned by the MRC, whose Head of Corporate Governance and Policy, Dr Frances Rawle, stated in correspondence: “I should make it clear that MRC considers it good practice for researchers to engage with trial participants”.
8. The Investigators diluted the entry criteria after the PACE Trial had commenced by moving the SF-36 (Short-Form Physical Function Subscale) threshold and by including people who had previously undergone CBT/GET and who had initially been rejected as PACE Trial participants. This undermines the reliability of all conclusions to be drawn from the data, not least because the first tranche of participants met different entry criteria from those who were recruited later. Because the entry criteria had been diluted, people in subsequent tranches were less ill and were thus more likely to respond favourably to the interventions ie. some participants would have higher physical function scores and lower fatigue scores on entry than others.
9. The Investigators mis-portrayed ME/CFS as a dysfunctional belief instead of a chronic inflammatory neuroimmune disorder, which is both scientifically invalid and also unethical. In his letter published in The Lancet on 28th May 2011, Professor Peter White gave his reason for ignoring the existing biomarkers for ME/CFS: “Possible biomarker data were not ignored but were irrelevant to the main aims of the trial since knowledge of their reported association with CFS did not alter the need to do the trial”. Here is an admission that the existence of biomarkers which disprove the Investigators’ beliefs about the disorder allegedly being studied were “irrelevant to the main aims of the trial”. Had these biomarkers been heeded, the Investigators would not have been able to proceed on their assumption that there is no underlying organic pathology and that it was safe for participants in the CBT and GET arms of the PACE Trial to be instructed to ignore any exacerbation of symptoms arising during the Trial (as advised in the Trial Manuals). From the Chief PI’s perspective, the existence of biomarkers was indeed irrelevant to the application of CBT and GET to correct what he and his colleagues assert are reversible wrong illness beliefs and deconditioning. However, they were not irrelevant to what should be the primary aim of any clinical trial, namely scientific integrity: to base research on a falsehood and to disregard the existing biomedical evidence to suit a desired outcome and personal beliefs is scientifically and morally inexcusable.
10. Even though they acknowledged they did not know what causes “CFS/ME”, in the CBT and GET arms of the trial the PIs assumed that participants had no physical disease but did not inform participants of their own conviction that they did not have a physical disease, or of their own assumption that CBT and GET do not work from a pathological perspective, but only from a psychiatric perspective. The PIs portrayed their own assumptions as established facts, thereby deliberately misleading participants, which is deceitful and unethical. This could mean that participants were not in a position to provide fully informed consent.
11. The Investigators chose a single six minute walking test as “an objective outcome measure of physical capacity”. The reference provided by the PIs for this draws attention to the difficulty of achieving reproducible results with such a test and it cannot be considered truly objective (see below for further consideration of this issue). The Chief Principal Investigator himself, Professor Peter White, has published evidence supporting the need for serial post-exercise testing (Immunological changes after both exercise and activity in chronic fatigue syndrome: a pilot study. White PD, KE Nye, AJ Pinching et al. JCFS 2004:12 (2):51-66 ) but none was carried out in the PACE Trial.
12. The Investigators originally intended to obtain a non-invasive objective measure of outcome using post-treatment actigraphy but abandoned this on the spurious grounds that wearing such a monitor for one week would be too great a burden at the end of the trial. Therefore, after spending millions of pounds of public money and involving hundreds of people in an intensive regime, the PIs completely failed to obtain objective measurements that would reveal whether or not the interventions were successful in the chosen cohort (who may not necessarily have ME/CFS, since the Oxford entry criteria exclude those with neurological disorders). The MRC found the non-obtaining of objective evidence to be perfectly acceptable, yet objective measurement is of the essence of science.
13. Apart from the incorrectly carried out six minute walking test, the PACE Trial results were based only on participants’ subjective responses to questionnaires, which other investigators have demonstrated do not relate well to actual activity (and the PIs themselves deem ME/CFS patients to misperceive their illness so their perceptions are not to be trusted). This is of particular concern when two of the interventions being tested (CBT and GET) specifically encouraged participants to re-interpret their symptoms as not resulting from disease but as normal responses to exercise in deconditioned people.
14. The trial therapists were trained to provide participants with misinformation; they were also trained to advise participants to ignore symptoms arising from the interventions, a situation that may in some cases result in death.
15. The Investigators may not have achieved the required clinical equipoise of the trial because they had already formed their opinion that “CFS/ME” is a somatoform disorder, that CBT and GET are successful and that the other active arm of the PACE Trial (adaptive pacing therapy) would be unsuccessful.
16. The Investigators and some members of the Trial Steering Committee initially failed to declare significant financial conflicts of interest: at the Trial Steering Committee meeting on 22nd April 2004, all members present were asked to declare any conflict of interest. No financial conflicts of interest were declared and it was agreed that no-one present had any other substantial or material conflict relevant to their work on the PACE Trial, yet among those present were Professors Peter White, Michael Sharpe and Trudie Chalder, who all work for the insurance industry and who thus have considerable financial interests which should have been declared and minuted.
17. Because of recruitment difficulties, patients at one particular ME/CFS clinic felt they were coerced into entering the PACE Trial by virtue of being told they would be discharged from the clinic and would lose access to a consultant (essential for supporting claims for State benefits) unless they agreed to undergo CBT and GET; coercion such as this is in breach of the Declaration of Helsinki yet patients and participants have asserted that coercion was used (breaching A8, B20 and B22).
18. Patients’ confidential data was not kept securely and was stolen, which is also in breach of the Declaration of Helsinki (breaching B21). The crime number is 3010018-06/ 22nd March 2006.
19. Although described in the Trial Protocol and in the trial literature as a “randomised controlled trial”, it was not a controlled trial: it was described in The Lancet as a “randomised trial”.
On numerous counts, the PACE Trial lacked scientific rigour and it is a matter of concern that the MRC continues to condone such a lack of scientific exactitude which, according to it own policy, would not have been condoned in any other classified neurological disorder apart from ME/CFS (which the MRC regarded a mental disorder; the MRC Portfolio in Mental Health Research states: “Mental health in this instance covers…CFS/ME”: Neurosciences Mental Health Board Strategy and Portfolio Overview Group Scoping Study, January 2005).
When asked why a WHO-classified neurological disorder was designated by the MRC as a mental health disorder, on 6th December 2005 Dr Rob Buckle from the MRC Neurosciences and Mental Health Board replied: “The Mental Health Scoping Study included the PACE and FINE trials on the basis of the type of intervention being assessed, namely psychological interventions, which best fitted…research…under the umbrella of the mental health programme manager”.
Although it was nominally a clinical trial, the PACE Trial had an underlying non-clinical purpose, namely the politically generated aim of removing people from State benefits (ie. the use of psycho-behavioural therapy to achieve the intended result of the cessation of State benefits for patients with “CFS/ME”).
The PACE Trial is the only clinical trial ever to have been (co)funded by the Department for Work and Pensions and it did so on the proviso that the interventions CBT and GET would successfully remove people with ME/CFS from State benefits.
No meaningful analysis of a trial with such a heterogeneous cohort is possible, especially given that it is impossible to know how many people with true ME/CFS (as distinct from those suffering from chronic tiredness) were amongst the participants.
Furthermore, the results of an intervention in any trial cannot be “evidence-based” for efficacy in a disorder when those most severely affected by that disorder were excluded from the outset.
The results of the PACE trial can do little for people with ME/CFS because the trial was based on a myth about ME/CFS that was allowed by the MRC to masquerade as science.
In the PACE Trial Protocol, the authors stated their intention to comply with certain codes of practice:
“The trial will be conducted in compliance with the Declaration of Helsinki, the trial protocol, MRC Good Clinical Practice (GCP) guidance, the Data Protection Act (1998), the Multi-centre Research Ethics Committee (MREC) and Local Research Ethics Committees (LREC) approvals and other regulatory requirements, as appropriate. The final trial publication will include all items recommended under CONSORT (Consolidated Standards of Reporting Trials)”.
There appear to have been some notable failures in this regard: the PACE Trial did not conform to the Declaration of Helsinki; it did not conform to the Trial Protocol in that the PIs failed to report the primary outcome measures (this being the whole point of a clinical trial); it did not conform to the MRC’s Good Clinical Practice guidance; it did not conform to the Data Protection Act (1998); the Chief PI did not keep his promise to the West Midlands Multicentre Research Ethics Committee (MREC) and the trial did not conform to CONSORT.
These are all serious matters which the MRC ought not to tolerate, but correspondence shows that the MRC sees no problem with such disregard by the PACE PIs of the scientific rules of procedure.
Referenced evidence of this lack of required conformity is set out in detail in “Magical Medicine: How to Make a Disease Disappear” (http://www.meactionuk.org.uk/magical-medicine.htm ); in “Professor Malcolm Hooper’s Further Concerns about the PACE Trial article published in The Lancet” (http://www.meactionuk.org.uk/Normal-fatigue.htm) and in an article entitled “Statistics and ME” (http://www.investinme.org/Article435%20Statistics%20and%20ME.htm) which was commissioned by The Royal Statistical Society.
Furthermore, the PIs did not conform to the ISRCTN Register requirements for transparency and completeness (the PACE Register number is ISRCTN54285094). Numerous and substantial changes were made to the PACE Trial between the publication of the Trial Protocol and the publication of selective results in The Lancet, for example: changes to the disorder being studied; changes to the entry criteria thresholds; changes to the method of scoring; changes to the PIs’ own definition of the “normal range” and changes to the measurement of outcomes, all of which have important implications for the analysis and the applicability of the results but not all are recorded as required.
Remarkably, in view of the tortuous complexity of much of the analysis presented in The Lancet article, the PACE PIs have stated: “Changes to the original published protocol were made to improve either recruitment or interpretability” (Lancet: doi:10.1016/SO140-6736)11)60651-X).
Rather than improving interpretability, the changes have in fact had the opposite effect.
The implemented changes had the effect of making it easier for the PIs to claim success for their PACE Trial, as well as making it harder for participants to report being made worse, for example, using the threshold set out in the Trial Protocol would have substantially reduced the number of participants meeting the PIs’ post-hoc “normal range” as described in The Lancet article. Furthermore, if researchers can modify a Trial Protocol more-or-less as they choose, they may consciously or otherwise construct one that gives them the results they desire. This defeats the purpose of having a protocol which, by definition, is designed to be adhered to both for the protection of participants and also in order to provide public confidence in the scientific endeavour.
Given the substantial number and nature of the changes made between publication of the Trial Protocol and the published article in The Lancet and the significant effect these changes had on the interpretation of the data, as well as the considerable public import that the PACE Trial results would have on influencing national and international healthcare guidelines for people with ME/CFS, provision of patient care, medical insurance coverage and State benefit entitlement, it is important to know why the MRC approved public funding for such changes.
Not only did the PIs change the primary outcome measures for physical function and fatigue set out in the Trial Protocol (these being scores that were to define a “positive outcome”, specifically a score of 75 or more or a 50% increase from baseline on the SF-36 Physical Function Subscale and a score of 3 or less on the Chalder Fatigue Scale), they defined new primary outcome measures with considerably lower thresholds (ie. an SF-36 physical function score of 60 or more and a Chalder Fatigue score of 18 or less using Likert scoring), and then referred to this as the “normal range”. This “normal range” threshold was both 15 points lower than the result that was to be regarded as a “positive outcome” and 25 points lower than the pre-defined “recovery” outcome of an SF-36 physical function score equal to or above 85 that was set out in the Trial Protocol. It was also 10 points less than the score which the MRC Trial Steering Committee considered to represent a “trivial” improvement and it was 5 points less than the score necessary to enter the PACE Trial.
These changes had a huge effect on how the study was (mis)reported.
Changes to the disorder being studied:
The PACE Trial purported to be studying the disorder “CFS/ME” (which the PIs stated in the trial literature is the same as ME/CFS, CFS or ME).
However, not only did the PACE PIs intentionally include those with fibromyalgia, a biochemically and taxonomically different disorder (classified in ICD-10 at M79), but because of recruitment difficulties, on 14th July 2006 Professor Peter White (the Chief PI) sought approval from the West Midlands Multicentre Research Ethics Committee to advertise his PACE Trial to doctors and to ask them to refer anyone “whose main complaint is fatigue (or a synonym)” to enter the trial.
Such heterogeneity severely undermines the conclusions of a trial that purported to be studying “CFS/ME” patients.
Moreover, after publication of The Lancet article the Chief PI confirmed in writing: ““The PACE trial paper …. does not purport to be studying CFS/ME but CFS defined simply as a principal complaint of fatigue that is disabling, having lasted six months, with no alternative medical explanation (Oxford criteria)”.
The Lancet editor responsible for publishing the PACE trial article has confirmed that Professor White specifically asked for his paper to be fast-tracked: for an article to be fast-tracked, it must be judged to warrant fast dissemination and to have a major effect on human health, so why would The Lancet fast-track an article concerning a disorder defined “simply as a principal complaint of fatigue”?
More importantly, the PACE Trial documentation refers to “CFS/ME” and both ethical approval and funding were granted on the basis that the disorder being studied was “CFS/ME”, not “fatigue or a synonym”.
Furthermore, the PACE Trial Protocol states that the main aim of the trial was to “provide high quality evidence to inform choices made by patients, patient organisations, health services and health professionals about the relative benefits, cost-effectiveness, and cost-utility, as well as adverse effects, of the most widely advocated treatments for CFS/ME”.
Given such confusion, there needs to be an immediate, high-profile clarification specifying which disorder was being studied in the PACE Trial and to which patients CBT and GET can be legitimately and safely applied, since ME/CFS is a completely different disorder from “a principal complaint of fatigue”.
Changes to the entry criteria threshold: as noted in section (iv) above, these included changes to the physical function entry threshold (SF-36 Physical Function Subscale, where a lower score means poorer physical functioning); this was initially set at less than 75 out of 100 in the Trial Identifier; it was changed to less than or equal to 60, but was then changed again to less than or equal to 65 to aid recruitment.
Because the SF-36 threshold was amended, a PACE participant could be enrolled with a score of 65, deteriorate to a score of 60 during the trial, yet the interventions could still be declared a success because an SF-36 score greater than or equal to 60 was counted as being within the PIs’ re-defined “normal range” (see below).
This means that the Chief PI did not keep his promise to the West Midlands Multicentre Research Ethics Committee made on 9th February 2006: requesting permission for one of many substantial amendments, he wrote: “This would mean that the entry criterion on this measure was only 5 points less than the categorical positive outcome of 70 on this scale. We therefore propose an increase of the categorical positive outcome from 70 to 75, reasserting a ten point score gap between entry criterion and positive outcome”. In fact there was a 15 point difference but the proposed ten point score gap between entry criterion and positive outcome became a negative five point score gap in the published article, meaning that a participant could deteriorate during the course of the trial and leave the trial more disabled than before treatment, yet still fall within the PIs’ re-definition of the “normal range”, ie. attainment of the “normal range” was set lower than the entry criteria.
Changes to the method of scoring: the Trial Protocol stated that “The 11 item Chalder Fatigue Questionnaire measures the severity of symptomatic fatigue…we will use the [bimodal scoring system] to allow a possible score of between 0 [perfectly healthy] and 11. A positive outcome will be a 50% reduction in fatigue score, or a score of 3 or less, this threshold having been previously shown to indicate normal fatigue”.
The bimodal system is a method which is easy to score but is less sensitive than Likert scoring because the respondent has only two choices of answer, eg. whether a symptom is present or absent, whereas Likert scoring is a labour-intensive psychometric instrument that uses continuous scoring which gives a respondent the chance to score each symptom from 0 - 3 to allow possible scores of the 11 item fatigue questionnaire between 0 and 33.
In reporting the results in The Lancet, the PIs did not use bimodal scoring, they used Likert scoring, stating that a Likert score of 18 or less would be counted as “normal”.
Thus different methods of scoring for entry and on completion were used and the complex and imprecise conversion is not readily understood by the average reader.
A Likert score of 18 is equal to a bimodal score in the range 4 - 9 inclusive, hence a Likert score of 18 always represents a state of abnormal fatigue according to the Trial Protocol.
Paradoxically, as with physical functioning scores, this change of scoring method allowed a participant to leave the trial with worsened fatigue than at entry but still be deemed to fall within the PIs’ re-defined “normal range”.
The rationale provided by the PIs for the change to Likert scoring in the consideration of outcomes in The Lancet article was: “Before outcome data were examined, we changed the original bimodal scoring of the Chalder fatigue questionnaire (range 0-11) to Likert scoring to more sensitively test our hypothesis of effectiveness”.
This raises the issue as to why, if they deem Likert to be a more sensitive instrument, the PIs did not use it from the beginning to the end of the trial.
Not all the changes made to their outcome measures were made before the PACE Trial PIs had seen the trial data.
Informed people believe that key changes may have been made because the FINE Trial (whose results were published a year before the PACE results) used identical outcome measures to those which the PACE Trial originally intended to use and the FINE Trial reported no statistically significant reductions in either fatigue or physical function at 70 weeks.
The FINE PI, Alison Wearden PhD, was an observer on the PACE Trial Steering Committee and when seeking one of his significant amendments, the PACE Chief PI had written to the MREC in terms: “The other advantage of changing to 75 is that it would bring the PACE trial into line with the FINE trial, an MRC funded trial for CFS/ME and the sister study to PACE”. Given the close links between the PACE and FINE Trials, it is inconceivable that the PACE Trial Investigators would have been unaware that the FINE Trial had produced such disappointing results.
However, following publication of those results, in a post-hoc analysis, the FINE Trial PI reappraised the data according to Likert scoring which produced a “clinically modest, but statistically significant effect…at both outcome points”, a fact of which the PACE Trial PIs must have been aware.
Thus one consequence of adopting a Likert approach to processing responses is that it becomes easier to demonstrate relatively small differences between the groups. The late adoption of Likert scoring by the PACE Trial PIs may have resulted in enhancement of a small level of significance.
The net results of the PIs’ changed method of analysis of the data is that identical responses could both qualify a person as sufficiently unwell to enter the PACE Trial and at completion of the trial allow them to be deemed within the “normal range”.
How both the MRC (which requires a high standard of excellence from the PIs in its funded trials) and The Lancet can accept such non-science as objective and reliable evidence of the success of the PACE Trial remains to be determined, since both have declined to address this issue that has been brought to their specific attention.
Changes to the definition of the “normal range”:
The two key indicators of therapeutic effectiveness - the primary outcomes of the trial, not to be confused with a “positive outcome” - were subjective questionnaire-based assessments of a participant's perceived physical function and level of fatigue.
The PIs reported in The Lancet that of those treated with GET and CBT, 28% and 30% respectively were "within normal ranges for both primary outcomes at 52 weeks", and in their accompanying Comment, Bleijenberg and Knoop stated: "the recovery rate of cognitive behaviour therapy and graded exercise therapy was about 30%".
However, it was not disclosed that prior to publication the PIs significantly lowered the clinical threshold that would count as a positive outcome, leading to misrepresentation of the efficacy of CBT and GET both in the lay press and The Lancet.
The PIs’ revised definition of the “normal range” of physical function, a score of 60 or greater out of 100 on the SF-36 Physical Function Subscale, is problematic not only because it is five points lower than the score required to enter the trial but because it contradicts previous publications from the same authors and has given rise to the widely-reported “30% recovery” claim.
At the post-publication press release one of the PACE PIs specifically referred to the “normal range” in terms meaning that patients were back to normal levels of health, so it is not surprising that the “normal range” has been misinterpreted by many clinicians and the media as “normal” health or “recovery”.
However, the “normal range” does not equate to normal health as widely understood in lay terms: it is a statistical analysis used by researchers (also known as a “reference range”), whereas “normal” in lay terms means high physical function with little or no impairment.
Indeed, in a post-publication letter to the Lancet, the Chief PI acknowledged that: “Being within a ‘normal range’ is not necessarily the same as being ‘recovered’ ”.
“Normal ranges” for the general adult population already exist: for example, in his SF-36 Health Survey Update, Ware states: “the physical function scale averages between 80 and 90” (SPINE 2000:25:24:3130-3139) and the Health Survey for England 1996 (included in Bowling et al, referenced in PACE) presents normative data for the SF-36 physical function scores in the general adult population as showing that 68% of the population score 75 or above.
A score of 75 or above is the same as the original score given by the PIs in the Trial Protocol before they revised it downwards.
It is notable that the SF-36 physical function score of 60 used to define the threshold of the “normal range” specifically for the PACE Trial contradicts the PIs’ previous publications about the same disorder using the same measures.
In 1997 the Chief PI published a randomised controlled trial of graded exercise in patients with “CFS” in which after 12 weeks of treatment the average post-treatment SF-36 score was 69, about which he stated that none of the measures had returned to “normal” (BMJ 1997:314:1647-1652); thus the Chief PI was clearly saying that an SF-36 score of 69 is not “normal”, yet in the PACE Trial an SF-36 score of 60 is deemed to be “normal”.
In 2007 the Chief PI stated: “A patient had to score 80 or higher to be considered as recovered” (Psychotherapy and Psychosomatics 2007:76:171-176) and in 2009 the authors of a manual on which the PACE Trial CBT manual was based asserted: “A cut-off of less than or equal to 65 was considered to reflect severe problems with physical functioning” (European Journal of Public Health 2009:20:3:251-257).
To demonstrate further how the PIs’ inappropriate use of a standard deviation has led to an unrepresentative “normal range” for SF-36 physical function scores in the PACE Trial, the scores for other disorders show that stable congestive heart failure patients have a mean SF-36 physical function score of 79.2; hepatitis C patients have a mean SF-36 physical function score of 79.3 and patients with osteoarthritis of the hip have a mean score of 62.4; thus patients with serious health conditions have mean SF-36 scores of more than 60 that was designated by the PIs as the “normal range” for healthy adults.
The result of changing the scoring instrument was that up to 30% of participants in the CBT and GET groups were able to be within the re-defined “normal range” at 52 weeks.
Not only did the PIs misrepresent the data with a flawed analysis (which should not have passed peer review), Lancet readers were misled about the PACE Trial normative comparison group which the article cited as being the UK working age population, when in fact it was the English adult population as a whole.
This is important as the actual reference group used by the PIs included elderly (ie. not limited to working age) people, which afforded a lower threshold of the “normal range”, thus boosting the proportion of PACE participants who could be deemed to have attained the PIs’ re-defined benchmark level of physical functioning, a fact which the Investigators had no option but acknowledge:
“We did however make a descriptive error in referring to the sample…as a ‘UK working age population’, whereas it should have read ‘English adult population’”.
According to the reference used by the PACE Investigators to gauge PACE participants’ outcomes, (Bowling et al), around 90% of the general population are within the “normal range”, with only 10% of the general population functioning at a lower level.
Therefore it can be seen that the 70% of PACE Trial participants who underwent CBT/GET in addition to SMC failed to reach even the PIs’ re-defined “normal range” and remained in the poorest-functioning 10% of the general population.
To recapitulate, discounting the 70%, it is the remaining 30% statistic that has been repeatedly but erroneously quoted as evidence that around one third of PACE participants “recovered” or “returned to normal” with CBT and GET.
In their reply published in The Lancet to complaints about the PACE Trial, the PIs have made numerous conflicting statements about the changes they introduced during the trial, for example: “All these decisions and plans were fully approved by the (MRC) Trial Steering Committee, were fully reported in our paper, and were made before examining outcome data to avoid outcome reporting bias”.
However, it must be questioned why the MRC Trial Steering Committee would give permission to alter the “normal range” threshold to a point which was well below a point that they had already deemed to represent a “trivial” improvement, as recorded in the Minutes of the TSC meeting held on 22nd April 2004 (“The outcome measures were discussed. It was noted that there may need to be an adjustment of the threshold needed for entry to ensure improvements were more than trivial”).
Modification of the benchmarks used to recruit participants and to judge whether or not they fell within the PIs’ own definition of the “normal range” at conclusion of the trial has produced an untenable situation whereby the same requirement for admission to the trial is deemed by the PIs to denote success at the end of the trial and this is being “spun” as meaning normal health and as a most important result that justifies using CBT/GET in ME/CFS nationwide.
According to the PIs and their colleagues Bleijenberg and Knoop, not only is it possible to deteriorate and still fall within their revised “normal range”, it is possible to recover but still have severe problems with physical functioning.
Changes to the measurement of outcomes
As noted in section (iv) above, the PIs originally intended to obtain a non-invasive, objective, primary measure of outcome using post-treatment actigraphy (and obtained ethical approval and funding on this basis) but during the trial the Chief PI abandoned his intention for participants to wear an actometer for one week at the end of the trial. In view of the fact that the subjective questionnaires demonstrated poor results, it is reasonable to surmise that objective actigraphy may have demonstrated even worse results.
A secondary outcome measure was the six minute walking distance test, but the capacity of such a test to assess ability in ME/CFS is highly debatable and the reference cited in The Lancet article states that it needs to be carried out twice to achieve reproducible results. Unless this protocol is followed (and it was not followed in the PACE Trial, as it was not carried out twice), then the test is invalid according to the reference cited by the PIs.
Moreover, the Chief Principal Investigator himself, Professor Peter White, has published evidence supporting the need for serial post-exercise testing in “CFS” patients (JCFS 2004:12:(2):51-66).
Furthermore, the assessors knew to which of the intervention groups the participants had been allocated in the trial, such masking being deemed “impractical” by the PIs.
The mean distance record by PACE participants who had undergone CBT was 354 metres (a 1.5 metre decrease compared with the SMC control group), meaning that CBT was ineffective.
Significantly, the CBT group managed less increase in walking distance than those who received nothing more than SMC (standard medical care).
CBT failed to improve average six minute walking distances and participants in all the intervention groups had, on average, significant disability at the end of the PACE Trial.
For those who had undergone GET, the mean distance was 379 metres (an increase of 67 metres from baseline).
In the six minute walking test, a normal healthy walking score is 500 metres; on brisk walking the average score is 650 metres, and on fast walking the score is 800 – 1,000 metres. The mean walking distance for healthy people aged 50 to 85 years is 631 metres (a score of 518 metres is deemed abnormally low for healthy but elderly people).
Patients with chronic obstructive pulmonary disorder (including those needing supplemental oxygen) are able to walk on average 60 metres further during the 6 minute walking test compared with those in the PACE Trial who had received GET plus SMC.
On average, PACE participants were able to walk less distance during the 6 minute walking test than people with traumatic brain injury.
PACE participants’ average 6 minute walking distance test scores were also lower than scores documented in many other serious diseases such as those awaiting lung transplantation, where a six minute walking test of less than 400 metres is regarded as a marker for placing a patient on the transplant list, and those in chronic heart failure (whose mean score is 682 metres), those in heart failure class II (mean score 558 metres) and those in heart failure class III, whose mean score is 402 metres in six minutes.
After CBT or GET, PACE Trial participants (whose average age was under 40) did not even achieve a six minute walking distance of 518 metres that is lower than average scores for healthy people aged 50-85 years.
Moreover, data on the six minute walking test was available for only 69%-76% of participants, a completion figure roughly 20% lower than for other secondary outcome measures, for which the PIs offer no explanation, but if participants dropped out because of ill-health, then the results are skewed in favour of the best-scoring participants.
None of the groups in the PACE Trial (which excluded the severely affected) came anywhere near to recording a healthy average walking score for the six minute walking test at 52 weeks.
The PIs had determined that a “clinically useful difference” (CUD) for the two primary outcomes to be an improvement of 2 points on the Chalder Fatigue scale (Likert scoring 0 – 33) and 8 points on the SF-36 physical function scale.
On CBT and physical function: the CBT group failed to achieve a clinically useful difference relative to SMC; the statistics were 7.1 points, which is below the PIs’ clinically useful difference threshold of 8 on a scale of 0 - 100.
On CBT and fatigue, the mean difference from SMC was a marginal 1.4 points above the PIs’ designated clinically useful difference of 2 points on a scale of 0 -33.
On GET and physical function, the mean difference was also a marginal 1.4 points above the PIs’ designated clinically useful difference of 8 points on a scale of 0 - 100.
On GET and fatigue the mean difference was 1.2 better than the PIs’ clinically useful difference of 2 points on a scale of 0 - 33.
Therefore, compared with SMC, CBT was shown to be marginally more effective in reducing self-reported fatigue, but did not achieve a clinically useful difference in physical function.
Equally, compared with SMC, GET was shown to be only marginally more effective at reducing fatigue and improving physical function.
In practice, the clinically useful difference is so small as to be imperceptible in general living and it is questionable as to whether it does in fact represent a “clinically useful” difference.
Furthermore, results on other measures were similarly under-whelming: for example, out of the reports submitted on the participant-rated CGI (clinical global impression) of change in overall health at the end of the PACE Trial, 60% of those in the GET group and 58% of those in the CBT group reported negative or minimal change.
Given that the “number needed to treat” was 1 in 7 (ie. it is necessary to treat seven patients in order for one person to improve to a “clinically useful” degree), this means that about 87% of patients were shown not to benefit to a clinically useful degree from CBT /GET.
NICE, however, announced on 14th March 2011 that there will be no review of Clinical Guideline 53 until 2013: even though some stakeholders requested a review on the grounds that the interventions recommended in CG53 should be driven by the scientific biomedical evidence (ie. not the PACE PIs’ assumptions of reversibility with cognitive restructuring), NICE remained intransigent:
“…interventions recommended in the original guideline, such as CBT and GET, were described as the interventions for which there is the clearest evidence-base of benefit. This is supported by the recently published PACE trial….The results of the study are in line with current NICE guideline recommendations on the management of CFS/ME….There are no factors…which would invalidate or change the direction of the current guideline recommendations. The CFS/ME guideline should not be updated at this time”.
The PACE Trial results challenge the PIs’ assertions that psychological interventions should be the primary management strategy for patients with ME/CFS, as according to the published results, only about 13% of secondary care patients achieved a minimal improvement after the interventions CBT/GET.
Notwithstanding, at the European Association for Consultation Liaison Psychiatry and Psychosomatics (EACLPP) Conference to be held in Denmark on 27th-30th June 2012, the Chief PI will be speaking: according to the published abstract he will say about the PACE Trial: “We found that CBT and GET were more effective than APT (adaptive pacing therapy) and SMC”, that “These results support individually delivered CBT and GET as moderately effective” and that “testing the limits of the illness is more effective than staying within them”.
Such assertions by the Chief PI are not supported by the figures.
Data not reported as required
1. Recovery statistics: The Trial Protocol stated: “‘Recovery’ will be defined by meeting all four of the following criteria: (i) a Chalder Fatigue Questionnaire score of 3 or less, (ii) SF physical function score of 85 or above, (iii) a CGI (clinical global impression) score of 1 and (iv) the participant no longer meets Oxford criteria for CFS, CDC criteria for CFS, or the London criteria for ME”.
Sixteen months after publication of selective results, no recovery data have been published, yet none of those associated with the PACE Trial has corrected the exaggerated media reports of “recovery”, for example: “Got ME? Fatigued patients who go out and exercise have best hope of recovery, finds study”; “scientists have found encouraging people with ME to push themselves to their limits gives best hope of recovery” [Daily Mail, 18 February 2011]; “The biggest-ever study of treatments for ME…has found that more people recover if they are helped to try and do more than they think they can” [Guardian 18 February 2011]; “About 30 per cent of patients given cognitive behavioural therapy or graded exercise made a full recovery to normal levels of activity, the study found” [The Times 18 February 2011]; “30 per cent recovered sufficiently to resume normal lives” [Independent 18 February 2011].
Of more concern is the fact that, as noted above, The Lancet published a Comment that misrepresented the PACE trial results, claiming “The recovery rate of cognitive behaviour therapy and graded exercise therapy was about 30%…..PACE used a strict criterion for recovery”. The Senior Editor of The Lancet promised to remove this erroneous claim; this has not been done and it remains on the record to be cited uncritically by others.
Even more perplexing is the fact that, despite no recovery statistics having been published, a consultant paediatrician who specialises in CFS, Dr Esther Crawley, has augmented the 30% recovery figure to 40% recovery: “Evidence of a recent evidence (sic) trial of cognitive behaviour therapy and graded exercise indicated a recovery rate of 30-40% one year after treatment” (Collin et al. BMC Health Services Research 2011, 11:217)
CONSORT and the ISRCTN require that the outcomes contained in the Trial Protocol be reported, whether the interventions being studied are successful or not, so by not publishing the recovery figures, the PACE Trial PIs have not acted in accordance with CONSORT or the ISRCTN.
2. Positive outcomes: The “positive outcomes” data as defined in the Trial Protocol have not been published.
3. Rates of deterioration: To understand how effective and safe the interventions were, the rates of deterioration should be published alongside the rates of improvement, with the equivalent measure of using the clinically useful difference to determine the rates of both improvement and deterioration, but the data for deterioration has not been released.
4. Economic analysis/ cost-effectiveness: The PACE Trial Protocol stated that an economic analysis would be presented and the cost effectiveness of the interventions would be evaluated. No such data has been published.
5. Numbers who returned to work/study: An important secondary outcome was to be the number of participants who returned to gainful employment/study (“The Client Service Receipt Inventory (CSRI), adapted for use in CFS/ME, will measure hours of employment/study, wages and benefits received, allowing another more objective measure of function”).
On the advice of (now) Professor Sir Mansel Aylward, former Chief Medical Advisor to the Department for Work and Pensions, the DWP co-funded the PACE Trial because it wanted a therapy that would get people with ME/CFS off State benefits and back to work. By letter dated 17th March 2011, the DWP Central Freedom of Information Team (re)confirmed that the PACE Trial was the only clinical trial funded by the DWP and supplied the reason for doing so: “The funding was agreed by a previous Departmental Chief Medical Adviser, who supported PACE due to his combined expertise and academic interest in this area of work. In his role as Chief Medical Adviser he felt it reasonable to support this trial, particularly as when the trial was initially being developed, consideration was given to exploring the use of a five point measure of work and social adjustment, which would look at employment and social outcomes for people taking part in the trial”.
By letter dated 21st February 2011, Dr Frances Rawle, Head of Corporate Governance and Policy at the MRC, provided more information about the involvement of the DWP:
“You ask why questions relating to participants’ financial situation were included…We accept that this is unusual in a clinical trial but…being in receipt of a disability pension was amongst a group of factors found in previous work (ie. a “finding” made only by the Wessely School) to be potential predictors of a negative outcome to treatment… The other reason to include financial questions was to be able to measure how treatments affected both healthcare costs and costs to society”.
Professor Peter White collected the data but has not delivered what was required as he has not published the number of participants who were able to return to gainful employment or study at the conclusion of the PACE Trial.
6. Impact of Clinician Expectations:
Responding to on-line questions regarding possible bias arising from the known affiliations of the PACE Trial Investigators, Professor White stated: “To measure any bias consequent upon individual expectations, all staff involved in the PACE trial recorded their expectations as to which intervention would be most efficacious before their participation, and we will publish these data after the end of the trial”.
Sixteen months after incomplete results were published in The Lancet, this has still not been done.
The PIs themselves concede that: “Our trial had limitations. We excluded patients unable to attend hospital”; that “Results cannot be extrapolated to those who are severely affected”; that “primary outcomes were subjective” and that “What this trial isn’t able to answer is how much better are these treatments than really not having very much treatment at all”.
What the PIs failed to acknowledge was that their ignoring of the biomedical evidence about the disorder they were supposedly studying (breaching the Declaration of Helsinki B11) invalidated the entire trial in that it was not grounded on the existing evidence-base and thus contravened the most basic principle of scientific research.
After a trial lasting nine years and costing £5 million, the PACE Trial has not taken us forward: not only have the results been misrepresented, but safe guidance on management options must address the needs of all patients with ME/CFS and it is not the case that the PACE Trial results are generalisable to all people with the disorder as claimed by the PIs.
The problematic analysis and selective presentation of data means that the PACE Trial has failed to provide “high quality evidence”, which is an unacceptable outcome: patients, clinicians and tax-payers have a right to expect higher scientific standards from the MRC.
The PACE Trial failed on a fundamental aspect of clinical research in that the benchmarks used to judge suitability for entry to the trial and successful outcomes are patently contradictory.
The need for independent statistical re-evaluation of the raw data is overwhelming as, without such an independent assessment, doubts over the veracity of the claims made by Professor White et al cannot be resolved.
Furthermore, the post-publication admission by the Chief Principal Investigator that the study was ‘not purporting to be studying CFS/ME’ invalidates the whole study which claimed to be addressing CFS/ME.
Given (i) the inability of the recruitment criteria to distinguish between ME/CFS and psychogenic fatigue, (ii) the illogical overlap of the entry criteria with “the normal range”, (iii) the failure of CBT to achieve a clinically useful difference for one of the primary outcomes and the trivial improvement produced by GET, (iv) the failure to recognise that an “averaged” improvement often masks very different responses to an intervention, and (v) the fact that around two thirds of participants who received CBT/GET remained in the lowest functioning 10% of the general population, the international ME community wonders why the PACE Trial is being hailed as a “gold standard” study which demonstrated the efficacy of CBT and GET for ME/CFS patients (as noted above, although the Protocol refers to it as an RCT [randomised controlled trial], The Lancet paper at no point describes PACE as a controlled trial, yet it was described in the press release as “the highest grade of clinical evidence” and as “extremely rigorous (and) carefully conducted”), which by any standards is risible.
Despite the irrefutably poor results of the trial, CBT and GET are being actively and inappropriately applied to people with ME/CFS; the PACE press release stated that the results suggest: “everyone with the condition should be offered the treatment” and that every patient “who wishes to be helped” should be willing to take part in such regimes. Non-compliance (for example, if a person has already found that exercise exacerbates their condition) is deemed to demonstrate lack of desire to recover, which in some instances has already led to the withdrawal of state and/or insurance benefits.
The PACE Trial was not a scientific study and defies reason but, given the considerable investment (Governmental, financial and professional) in its success, it could not be allowed to fail.
Since the MRC has declined to address the many public concerns that have been brought to its attention, it falls to BIS to do so.
Professor Hooper gratefully acknowledges the assistance of members of the international ME community.