Sunday, August 1, 2010

Evidence-based Medicine in Disguise: Beware the Surrogate!

In this post, I will give ordinary folks a ‘peek behind the academic curtain’. I am not an academic physician, but a mere practicing gastroenterologist who spends my days ‘enlightening’ Cleveland colons. Why do some medical studies, which achieve breaking news status, often fall so short of our expectations? Physicians are cynical about these medical milestones, since they are often short-lived. Today’s cure may become tomorrow’s disease.

A common practice and serious flaw in medical research is to rely upon a surrogate marker when studying a disease. Let me explain. If you endure the following few paragraphs of literary driftwood, you will understand press reporting of medical studies on a deeper level.  This could directly affect your medical care and generate some interesting conversations during your next doctor visit.

A surrogate marker is an event or a laboratory value that researchers hope can serve as a reliable substitute for an actual disease. A common example of this is blood cholesterol levels. These levels are surrogates, or substitutes, for heart disease. If a medical study demonstrates that a medication can lower cholesterol level 10%, then we assume that this will also lower the risk of cardiovascular disease. Why doesn’t this same study determine if an anti-cholesterol drug decreases heart attack rates directly? After all, most folks would rather be spared a heart attack than have a silent decrease in their blood cholesterol levels.

Why are Surrogates Used?

It is much easier and cheaper for researchers to measure surrogates than actual disease events. What could be simpler than measuring blood cholesterol levels? In contrast, it would be a very tough slog to show that a cholesterol-lowering drug reduced heart failure or mortality rates. With a surrogate, medical studies can be completed much more rapidly, in contrast to studying actual diseases, which can take a decade or more to complete. By then, the findings may no longer be relevant. Surrogate research is also much less expensive to perform.

Surrogate results have flashy marketing appeal because their findings can be expressed in catchy headlines that extrapolate the actual conclusions. Haven’t we all read headlines like this?

Research shows that new blood test can prevent cancer

Surrogate research is valid if the surrogate truly represents the disease. Often, this assumption is questionable or outright false. Not long ago, cardiologists were going gaga over the CRP (C-reactive protein) blood test as a surrogate marker for cardiac disease. This was great news for the ‘statin’ drug manufacturers who claimed victory when their medications reduced CRP levels, but did this really prevent heart disease? I wonder. Surrogates often take on a life of their own, far removed from the actual disease they represent. Patients shouldn’t care if their ‘surrogates’ are improving; their objective should be to prevent disease, feel better and live longer. Yet, we physicians have often convinced our patients that surrogate improvement means better health. Monitoring cancer blood tests called tumor markers illustrates this point well.

“Great news Mrs. Bedridden. Your cancer blood test improved 10 points!”

“Thank you doctor, but I still can’t walk.”

In my own field, gastroenterologists remove colon polyps with enthusiasm and zeal. Polyps are not diseases. They are surrogates for colon cancer. We hope and believe that when we remove pre-cancerous polyps that we are reducing your risk of colon cancer. Interestingly, there is no double-blind placebo controlled trial (the gold standard of medical research) that establishes that colonoscopy reduces colon cancer. Just because it sounds logical, doesn’t mean that it’s true. There have been medical studies, for example, that have described treatments that reduce arterial plaque, which is a surrogate marker and not a symptomatic disease. Most of us would welcome a treatment that reduces plaque, because we would assume that this would benefit us by preventing heart attacks and strokes. Assumption, however, is not science.

Those of us who have been reading journals for some time are skeptical before we celebrate the medical breakthrough of the day or week. What is sound medical dogma today may fade over time and become junk science. This is particularly true of surrogate studies, which are indirect by definition.

Those who earn their living and their reputations from medical research may have a different view on surrogacy than I do. Indeed, surrogate research is an important research tool, that can open important therapeutic avenues and stimulate additional research. We must be mindful, however, how easy it is to exaggerate their conclusions beyond the data. The public needs understand this issue. Think about this the next time you read a news flash that promises a medical miracle. Chances are that the miracle is a mirage.

14 comments:

DrJohnM said...

Very important information. Nicely said.

Also problematic is the use of combination surrogate endpoints. This statistical slight of hand is frequently done in cardiology. Can't show that a treatment reaches statistical significance in one thing, then use a combination, and the likelihood of significance increases.

JMM

Duval & Stachenfeld said...

Most labs I know who take medical students in over the summer due it because they believe it will be good for the student, not because of the good it will do for the lab. Students are there for 8 short weeks and during that time they require significant training and constant oversight. If anything, they often slow the other PhD students, Post Docs, or the PI herself down from getting work done efficiently because of the training they require.

Elaine Schattner, M.D. said...

Michael,
Thank you for this excellent post on an important topic that's perhaps too dry for most newspapers and blogs and, even more, for your persistent skepticism. I wish all docs were thinking about what they read, rather than just taking it in -

Gary Schwitzer, Publisher, HealthNewsReview.org said...

Great piece.

For more than four years, my HealthNewsReview.org project has graded health news stories that include claims of efficacy in medical interventions. We apply ten standardized criteria to the review of each story, and have now reviewed more than 1,100 stories, emailing journalists whenever their story has been reviewed.

One criterion evaluates whether the story grasped the quality of the evidence. Surrogate markers and intermediate endpoints become huge traps for journalists - and for consumers, as you point out. We grade a story unsatisfactory on that one criterion if it doesn't seem to grasp the concept or fails to emphasize the possible limitations of surrogates in the story.

Your column was a terrific explainer. Thanks for the contribution.

Gary Schwitzer
Publisher
HealthNewsReview.org

Jackie Fox said...

Great post and how awesome is it that Gary Schwitzer reported/linked to it on Healthnewsreview.org/tweeted it. I'm sure I will be one of many retweeting it.

Dan said...

It's true the lay press often picks up on these surrogate marker studies and prematurely touts some medical breakthrough. However, the original journal article always includes limitations of the study. This might be difficult for lay people to understand fully, however, so it's the job of a science reporter to translate the article for the masses. I think they (real science reporters, specifically) generally do an OK job. Editors are usually the ones who sensationalize the article with some fantastic headline.

John said...

I promised not to bother you any more but I'm back to say congratulations. Getting good comments from Gary Schwitzer above is like getting the Good Housekeeping seal of approval. He linked your post with glowing terms.
Keep up the good work.

Michael Kirsch, M.D. said...

Thanks, John. Were you testing to see if I am old enough to know what the Good Housekeeping Seal is? I am!

John said...

That old, huh?
No wonder you're against reform.
(...only joking)

A. Bailey said...

I dunno.

I'm a simple man, but I'm not sure I agree with your post.

Somewhere in it is lurking a failure of distinction, I think.

Perhaps there needs to be a distinction between surrogates, precursors, and prerequisites.

For example, studies have shown that obesity is a "risk factor" for colon cancer. That seems like a classic "surrogate", as a direct causal link is tenuous at best, and it certainly isn't clear that modifiying the "surrogate" would modify your rick of disease. I'm with you on that one.

On the other hand, except in certain disease states such as ulcerative colitis, adenomatous polyps would appear to be precursors, perhaps even prerequisites of colon cancer. When I remove a polyp, I believe I'm interrupting the natural history of a disease, not treating a "surrogate". I think the science backs me up on that one.

If you truly believed that going on a "search and destroy mission" for colonic adenomas was a meaningless task, wouldn't that have certain implications on your ethical practice of medicine?

I say this as a devil's advocate in search of clarity, not as a personal attack. I think it's an interesting issue.

Michael Kirsch, M.D. said...

@A Bailey, thanks for comment. Adenomas are believed to be precursors to colon cancer, as you noted. In medical studies, however, they are regarded as surrogates for colon cancer. It remains an assumption, unproven by rigorous double blind controlled studies, that polyp removal prevents cancer. Remvoing polyps is ethical mainstream medical practice. We practice based on our knowledge of the time, which will change with experience and knowledge aquisition.

Ronnie Horesh said...

HI Michael, and thanks for your post. Sorry for the lateness of this query in response to your comment of 7 August: when you say "unproven by rigorous double blind controlled studies", do you mean that these studies have been conducted and proved inconclusive, or that they haven't been done at all?

Michael Kirsch, M.D. said...

@Ronnie, to my knowledge these studies have not been conducted.

Lindasy Rosenwald said...

Nice blogging, My review is very good example.
Lindsay Rosenwald http://www.lindsayrosenwald.biz/lindsay-biotechnology/dr-lindsay-rosenwald-leader-drug-development/ Dr. Lindsay Rosenwald- A Leader of Drug Development

Add this