We are getting to this story a bit late, but perhaps can provide some new insight.
In 2006, an article that supported the value of CT scans to screen for lung cancer caused quite a stir. [The International Early Lung Cancer Action Program Investigators. Survival of patients with stage I lung cancer detected on CT screening. N Engl J Med 2006; 355: 1763-1771. Link
here.]
The article reported a case series of over 30,000 patients who were entered into a program using annual CT scans to screen for lung cancer. Over 10 years of enrollment, 484 patients were found to have cancer. The investigators estimated the10-year lung-cancer specific survival of these patients was 80%, concluding "annual spiral CT screening can detect lung cancer that is curable."
A few days later, the article's lead author, Dr Claudia Henschke, was quoted
in the New York Times as not understanding why CT screening was not the standard of care, "'I don�t get what the resistance is,' Dr. Henschke said. To her, it is a matter of simple logic:
the earlier cancer is found, the better the odds of a cure. CT finds lung cancer early. So why not use it?"
In line with that sentiment, advocacy for widespread screening using CT scanning has continued under the auspices of the
Lung Cancer Alliance. (See
their news release, "Lung Cancer Alliance stands behind CT screening for lung cancer" from last fall.) The LCA asserted, "the mounting evidence which continues to mature from the largest international, protocol-driven screening effort (I-ELCAP) [is] showing that CT screening in a high risk population has the potential to reverse the current 15% five year survival rate." Furthermore, as reported in the
May 4, 2007 edition of the Cancer Letter, legislation providing funding for screening using I-ELCAP protocols has been introduced in several state legislatures.
Of course, this seems to be another example of the introduction of an expensive, high-technology intervention that could save lives, but once again driving up the already staggering cost of health care. But these sorts of advances seem inevitable, don't they?
Maybe not. Dr Henschke's enthusiasm for screening seemed to go well beyond the data her study provided. In fact, the multiple flaws of the study were nicely summarized in a special article in the Archives of Internal Medicine [Welch HG, Woloshin S, Schwartz LM et al. Overstating the evidence for lung cancer screening: the International Early Lung Cancer Action Program (I-ELCAP) study. Arch Intern Med 2007; 167: 2289-2295. Link
here.]
- A Case-Series, not a Randomized Controlled Trial - Because the study did not contain a control group, it is impossible to tell whether its apparently favorable results were due to the intervention, to the selection of patients with unusually good prognoses, or to chance. As Welch et al put it, "The I-ELCAP study design is a case series. Because there was no control group, readers cannot know what would have been observed in the absence of screening."
- Lead-Time Bias - The major result of the NEJM study was that patients with cancer seemed to live longer after their diagnosis than experience suggests is the norm for lung cancer patients. However, this could have been due to lead-time bias. Per Welch et al, "because survival is measured from the time of diagnosis, any screening test that advances the time of diagnosis will bias the measure of survival�the lead-time bias." For example, consider a person who developed cancer at age 60, was diagnosed at age 65, and died at age 66. His survival after diagnosis was one year. But if he had been diagnosed at age 60, and died at age 66, his survival after diagnosis would have been six years, but his life-span would not have been altered.
- "Overdiagnosis Bias" - This means that more intensive efforts at diagnosis may reveal cases of disease with more favorable prognoses. For example, it is well known that some patients who appear to have early prostate cancer will never have progressive disease. But detecting such patients by screening will not improve their survival. Welch et al noted that previous studies that used chest x-rays to screen for lung cancer demonstrated improved survival after diagnosis, presumably as a result of combined lead-time and overdiagnosis biases, since overall survival was not improved in patients who were screened compared to controls.
- Failure to Assess Harms of Screening and Subsequent Treatment - Welch et al noted problems with how the I-ELCAP study assessed deaths due to surgical treatment of cancer. The study also failed to report adverse effects of invasive diagnostic efforts for patients who had positive CT scans, most of whom did not prove to have cancer. For example, needle biopsy of lesions found by CT scan can cause pneumothorax (lung collapse), but these events were not reported in the NEJM article.
Also, a series of letters to the editor of the NEJM brought up similar concerns, plus additional flaws, e.g., although the I-ELCAP study reported 10-year disease specific survival, it only followed patients for a median of 40 months, and less than one-fifth of subjects were followed for five years or more. A response by lead author Henschke and two other authors failed to reassure. It first seemed to simply deny the importance of lead-time bias,
Screening for cancer is supposed to provide for lead time in diagnosis and treatment. A bias is introduced when one compares relatively short-term survival rates as of diagnosis to assess the effectiveness of treatment with lead time relative to treatment without lead time. We did not do this.
Henschke et al also seemed to misunderstand overdiagnosis bias. They responded to the relevant critique by asserting that all cases of cancer were confirmed by pathology. This begged the question, because pathological diagnosis does not necessarily determine whether a tumor will progress.
Finally, they seemed to deny that randomized controlled trials are necessary to assess screening programs.
Berg and Aberle remark that our study had no comparison group, but unlike interventional research, diagnostic research does not inherently require a comparison, much less a comparison group. The attainable frequency of diagnoses of stage I tumors can be assessed only in the framework of screening, and our principal diagnostic result was the 85% frequency of diagnoses of clinical stage I tumors among all diagnoses. The principal interventional result was the 92% cause-specific 10-year survival rate after prompt treatment of stage I tumors. The relevant comparison group would consist of patients who received early diagnoses but were treated late, principally to learn about the timing of deaths from lung cancer.
Every scientist tends to be an enthusiast about his or her own work, but I was struck by how Henschke et al seemed to shrug off their study's obvious methodologic problems.
(It might seem to be beating a dead horse, but I found even more problems with the I-ELCAP article beyond those noted above: failure to describe the clinical characteristics of the patient population; not blinding assessment of whether deaths were cancer-related; failure to have a credible method to verify that patients who did not have a positive CT scan did not have cancer; and failure to report loss to follow-up.)
More concerns about the origins of the I-ELCAP investigators' enthusiasm, which seems to have infected such groups as the Lung Cancer Alliance, were raised by a new report in the Cancer Letter, (from Jan 18, 2007, but not available without a subscription, and just made available to me last week),
The leaders of a research consortium that advocates computed tomography scanning to detect early-stage lung cancer have built an estate of 27 patent applications worldwide covering technologies and methods of screening.
According to publicly available databases, the leaders of the International
Early Lung Cancer Action Project are listed as inventors on patent applications
and one issued U.S. patent that cover innovations in lung cancer screening,
from clinical trial methodology, to software for interpretation of scans, to
technology of biopsy needles.
The first of these patent applications was submitted on April 11, 2000. In the U.S., group leaders are listed as inventors on one patent and 10 published applications. At least one of the inventions has been licensed by General Electric, a maker of CT scanners, and, in another instance, patent rights were reportedly exchanged for stock of a start-up company that markets lung biopsy devices.
Disclosure statements that accompanied the publications by the group�s leaders didn�t reflect these activities as potential conflict of interest, a literature
search shows.
Furthermore,
Both GE and Cornell declined to disclose the details of a licensing agreement covering Henschke�s and Yankelevitz�s inventions.
'We do have a license agreement with the Cornell Research Foundation for certain CT lung cancer patents,' Corey Miller, a spokesman for GE Healthcare Americas,
said to The Cancer Letter.
'The researchers�Henschke and Yankelevitz� have received some royalty payments from GE for a computer algorithm equation that they developed to detect lung cancer on diagnostic scans,' Miller said. 'That is, as far as I know, the extent of our relationship with those two.'
Note that the 2006 NEJM article included no disclosures of conflicts of interest.
Finally, the Cancer Letter noted that I-ELCAP investigators have worked actively with the Lung Cancer Alliance to promote screening.
As they spar with skeptics, I-ELCAP scientists work closely with LCA, a group that advocates for screening former and current smokers in accordance with I-ELCAP protocols.
So here we go again. Flawed research that supports an expensive, high-technology intervention for which millions of people would be eligible appears in a major journal. The intervention is hyped in news articles, and by a grass roots patient advocacy group. Politicians soon jump on the bandwagon. Citing the potential for saving lives, enthusiasts ignore the flaws in the data that started it all. Only much later do we find out that the researchers who wrote the flawed study may stand to make money if the high-technology intervention is widely adopted.
At the minimum, this case suggests that academic medicine's current method of managing conflicts of interest through disclosure is not working well. Further, in the absence of such disclosure, it seems that individual clinical decisions and public policy may too often be based on apparently unbiased research that in fact is being purveyed not merely by enthusiasts, but by people who may make more money when the results turn out a certain way.
People have been wringing their hands about ever rising health care costs for a long time. Expensive high-technology interventions are often cited as responsible, but not further addressed, apparently due to the assumption that their costs are an inevitable reflection of their advantages. Yet maybe this is not so inevitable. We may have placed to much credence in research that exaggerated the value of high tech gizmos because it was done by by people who stood to gain from the sale of such gizmos.
At the very least, we need to make sure we know when clinical research to evaluate screening methods, diagnostic tests, drugs, or devices was done by people with financial interests in their success. It might be, however, that the only way to get unbiased evaluations of drugs and devices is to have the evaluation done by people who have no financial interest in the results turning out one way or the other.
See
this editorial in the Boston Globe by Dr Jerry Kassirer, and
this post in the WSJ Health Blog for earlier discussion of what the Cancer Letter revealed.
ADDENDUM (11 March, 2008) - See also
previous comments after publication of article by Welch et al by Merrill Goozner in GoozNews blog.
