Evidence-Based Medicine

Special Bonus for Subscribers to the John R. Lee, M.D. Medical Letter October 1998
A Commentary On The New Buzzword of Conventional Medicine:
by John R. Lee, M.D.

Evidence-based medicine is the new buzzword in medical literature. It refers to the desirability of basing one's medical decisions on adequate, good quality research. This, of course, is a position against which no one would argue. However, it is being used in some circles to denigrate observational research and, particularly, to stem the present flow of patients from conventional to alternative physicians and to steer people away from over-the-counter nutritional therapy back to prescription drugs. The presumption is that conventional physicians use drugs that are backed by good research whereas health food store items lack proper research. Is this presumption correct? Let us look at some recent evidence.

I. The Tamoxifen Question

Conventional medicine supports the notion that tamoxifen prevents breast cancer, presumably by binding to estrogen receptors in breast tissue to reduce the cancer risk of one's endogenous estrogen. Historically, tamoxifen's role has been confined to use in women who have had mastectomy for breast cancer. It is now being proposed that all women at risk of breast cancer should be put on prophylactic tamoxifen. Is this proposal based on good evidence? The 11 July 1998 issue of Lancet contains reports of two studies that cast grave doubt on this proposal. While the US study (BCPT) was of 24 month's duration, the duration of studies reported in Lancet were 46 months and 79 months. Both studies found no difference in breast cancer frequency relative to controls. They did find significant increases in the incidence of endometrial cancer, vascular (venous thromboses) events, and hypertriglyceridemia.

References:

1. Veronesi U, Maisonneuve P, Costa A, et al., Prevention of breast cancer with tamoxifen: preliminary findings from the Italian randomised trial among hysterectomised women. Lancet 1998; 352: 93-97.

2. Powles T, Eeles R, Ashley S, et al., Interim analysis of the incidence of breast cancer in the Royal Marsden Hospital tamoxifen randomized chemoprevention trial. Ibid, pgs 98-101.

3. Commentary by Pritchard K, "Is tamoxifen effective in prevention of breast cancer? Ibid, pg 80.

II. HRT and Prevention of Coronary Heart Disease (CHD)

It is widely advertised that conventional HRT should be given to all postmenopausal women to prevent coronary heart disease. But is there good evidence for this claim? The reference most widely used to claim a heart benefit from HRT is the Nurses Questionnaire Study, which was neither randomized, nor placebo controlled. A critique of this study appears in chapter 13 of my book, What Your Doctor May Not Tell You About Menopause. Recently JAMA published a randomized blinded study of 2,763 postmenopausal women followed for five years. One half of the women received Premarin 0.625 mg and Provera 2.5 mg daily; and the other half received placebo. After five years of follow-up, there was no evidence of a cardiac benefit from hormone therapy. There was, however, a three-fold increase in the incidence of thromboembolic events [blood clots in the legs and lungs] and a significant increase in gallbladder disease in the treatment group compared to controls.

Similarly, the long-running Framingham study has compared the incidence of coronary heart disease with postmenopausal hormone replacement therapy and found no correlation.

If evidence-based medicine is your guide, the claim that HRT prevents CHD must be discarded.

References:

1. Hulley S, Grady D, Bust T, et al., Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA 1998; 280: 606-613.

2. Editorial by Petitti D in the same journal.

III. Estrogen Supplementation of Postmenopausal Women for Osteoporosis

It is widely taught that osteoporosis is due to estrogen deficiency, and that estrogen essentially falls to zero at menopause. These two hypotheses lead, of course, to the conclusion that postmenopausal women should receive estrogen supplementation to prevent and treat osteoporosis. A recent study reported in the New England Journal of Medicine casts great doubt on both hypotheses. This study compared the risk of hip and vertebral fractures in older women (age 65 and older) to serum concentrations of estradiol, sex hormone binding globulin (SHBG), and 1,2-dihydroxy-vitamin D. None of the women used supplemental estrogen (ERT or HRT). It was found that women with undetectable serum estradiol concentrations (<5 picograms per milliliter [pg/ml]) had a relative risk of 2.5 for hip and vertebral fracture compared to women with detectable serum estradiol concentrations. However, the relative risk did not decrease further with progressive increases of endogenous estradiol concentrations (from 5 to >10 pg/ml). Only 33% of the women had serum estradiol concentrations less than 5 pg/ml. Thus, the majority of women over age 65 continue to make estradiol in sufficient amount to create optimal estrogen-related bone benefit (it restrains osteoclast-mediated bone resorption).

The great majority of women continue to make estrone after menopause by aromatization of andro-stenedione in body fat. Estrone is inter-converted to estradiol in the liver and the gut. Thus, it is ignorant nonsense to assume that estrogen levels fall to zero at menopause.

Osteoporosis is a multi-factorial disease. Despite adequate estradiol, osteoporosis occurs. This proves that factors other than estradiol deficiency cause osteoporosis. One of the major factors is progesterone deficiency, as I describe in What Your Doctor May Not Tell You About Menopause. This, however, was not considered in this study in NEJM.

The study also showed that SHBG reduces the bioavailability of estradiol. Those women with more SHBG showed an increased risk of fracture despite having adequate serum estradiol.

Both the authors of the study and the editorial that accompanied the article lamented the fact that there was no simple test to measure bioavailable estradiol. It is rather strange and a sad commentary on the knowledge of presumed experts that they did not know that saliva hormone assay is readily available and measures only the non-protein bound steroid hormones, i.e., the bioavailable hormone.

Conclusion: The evidence clearly indicates that the two assumptions on which estrogen supplementation for prevention and treatment of osteoporosis are untrue in the majority of postmenopausal women. Evidence-based medical decisions should recognize that the majority of women 65 years or older have sufficient endogenous estrogen for bones. Further, conventional ERT and HRT dosages that raise serum estradiol levels to >40 pg/ml clearly are excessive and unwarranted.

Reference:

Cummings SR, Browner WS, Bauer D, et al., Endogenous hormones and the risk of hip and vertebral fractures among older women. NEJM 1998; 339; 733-738.

The Problem With "Scientific" Experiments such as Randomized, Double Blind, Placebo-Controlled Testing

"Scientific" testing is based on the belief that the treatment group differs from the control group only in the factor being tested. While this works well in testing of inanimate objects as in the physical sciences, it is far from clear that such testing works all that well for health matters. Living organisms are simply too complex and illness is often so multifactorial that parity of the treatment group and the control group can not be presumed.

In the case of hormones and osteoporosis, operative factors include sex hormone binding protein (SHBG) and cortisol binding protein (CBG), genetic traits, exercise, sun exposure, digestive differences, stress, program compliance, adiposity, diet, nutrients such as magnesium, calcium, silicon, and boron, or vitamins such as C, A, E, D, and K, as well as the influence of other hormones such as androgens, growth hormone, and thyroid. Also involved are numerous drugs and medications used by the test subjects. While some of these factors (such as height, weight, adiposity, and medications) are easily "controlled" (i.e., accounted for), many of them are not.

Obviously, the more confounding factors there are, the more difficult it is assure parity between the treatment and the control groups. Medical history is studded with examples of spurious conclusions resulting from uncontrolled or unrecognized confounding factors.

For example, the current vogue of calcium supplementation at 1500-2000 mg/day resulted from studies of a generation ago when women subjects were given a diet containing excessive protein. At that time it was not recognized that excess dietary protein induced urinary calcium loss and a negative calcium balance. Instead of the 4-5 ounces of protein (equivalent to 16-20 ounces of steak) consumed each day by the subjects of these earlier tests, we now know that 1.5 ounces of protein daily is a more suitable intake for the average healthy adult. Under those conditions, total daily calcium intake is optimal at 600-800 mg/day.

It is argued that, with sufficient number of subjects, randomization will correct for unrecognized confounding factors. When the number of confounding factors are great, the number of subjects required becomes unmanageable.

Randomization and placebo control do not adequately control for each subject's individuality. When individuality is ignored, test results are confounded by unknown variables. And statistical calculations can lead to false conclusions. For example, if a given populations experiences 50% mortality by age 1, and the rest all live to age 80, the average (mean) life span in this population is 40. Yet, no one dies at 40. In another population, children do not die but most of the population does die at 40. The average life span for this population is also 40. Yet the ages of death are totally different between the two groups.

Similarly, a river of average depth of 2 feet will have channels deep enough for people to drown.

Given the above, a more logical experiment would be to test BMD change serially in women in whom the only difference to their treatment is the addition of progesterone. If those women whose decline of BMD is well recorded and the addition of physiologic doses of progesterone results in BMD improvement over the next few years, it is good evidence that progesterone does indeed enhance bone building. That is the sort of experiment I conducted with patients of average age 65. If more physicians would do as I did, the more they would realize the importance of progesterone not only in treating osteoporosis but also for all the other progesterone benefits their patients will demonstrate.

John R. Lee, MD
Editor, The John Lee Medical Letter

Junk Science: Garbage In, Garbage Out

The Exalted Peer Review
An Essay from John R. Lee, M.D.

The advance of medicine, looked at historically, has occurred most commonly by intelligent observation and serendipity. The insightful observation coming at a propitious time in medicine leads to new directions that eventually find fulfillment in a change of medical practice. Older erroneous conceptions are, however, surprisingly difficult to overthrow. Change in medicine is sometimes glacial despite overwhelming evidence for it.

Bleeding therapy, for example, was the standard of care for many years, having been strongly advocated (but never justified) by the eminent Benjamin Rush, MD, in the late 1700's. George Washington's demise was undoubtedly hastened by multiple bleedings. Publication in established medical journals is one way new concepts become known. The implication is that, if some alternative concept is worth reading, it will be accepted for publication. Conversely, if the alternative concept is not accepted for publication, the implication is that the paper is probably not worth reading. Since most established medical journals receive far too many papers for publication than can be accommodated, the question becomes - who decides what is published? The principal mechanism for this these days is peer review.

The idea of peer review, on the surface, is a good one. Let reviewers with experience and education comparable to the author's read and judge the credibility of the paper being presented for publication. But if the paper's concept is innovative and challenging, who is the author's peer? If the review panel represents established orthodoxy, might there not be a bias against the unorthodox viewpoint? Peer review in grant applications raises the same concerns. This was recently examined by Simon Wessely of the Department of Psychological Medicine, King's College School of Medicine, in The Lancet, 25 July 1998, 352: 301-305. The answer is that good answers are hard to come by. Those that "win" think the process is fine; those that are denied find the process unfair. Case in point: A well-done study showing prostate cancer cell growth driven by estrogen and blocked by progesterone is turned down by a major cancer journal. The peer reviewers had no criticism of the study but denied publication with the excuse that it would have been accepted if the scientist authors had used a progestin instead of a progesterone in their research.

It is of some interest that The Lancet published the Wessely report at this particular time since it has been taking some flak for a major article it published in its March 14, 1998, issue. That article, "Dietary sodium intake and mortality: the National Health and Nutrition Examination Survey (NHANES I)" by Michael H. Alderman, Hillel Cohen, and Shantha Madhavan, was reviewed and given poor marks by the Nutrition Action Health Letter, June 1998. The study is based on baseline dietary information acquired during 1971- 75 in a sample of 20,729 adults (aged 25-75). Eventually, 11,346 participants underwent medical examination and nutrition examination, and their "vital status" was obtained by June 1992. Of this group, 3,923 of the participants had died. The authors then compared the sodium and calorie intakes of those who had died with that of the survivors. Multiple regression analyses were done to assess the relations with mortality. The authors claim they found that the death rate was slightly higher among people who reported eating the least sodium, and this finding was statistically significant. This is contrary to the general wisdom. The authors, however, must have felt a little uneasy with this finding since, in their conclusion, they state that "This observation does not justify any particular dietary recommendation"; "nor [does it] justify advice to increase salt intake or to decrease its concentration in the diet." Despite spending God knows how much money from the National Center for Health Statistics of the Center for Disease Control and Prevention, the National Institute on Aging, the National Cancer Institute, and other institutes of the National Institutes of Health, and all their multiple regression analyses, they are uncertain whether their years of work mean anything at all.

Let us take a closer look at the study. ( Their nutritional examination, by which they determined each participant's twenty years of nutritional intake, consisted of a single 24 hr recall. Even if accurate, why would anyone believe that one day's food intake is representative of 20 years of food intake? As far as that goes, I'm not sure I can accurately remember everything I ate yesterday. ( Those who reported lower sodium intake were more likely to have high blood pressure which would have raised their risk of dying. Whether they actually ate less salt or whether merely reported eating less salt (to indicate they followed doctor's orders) is not important; by being patients with high blood pressure their risk of dying was higher than participants without high blood pressure. (Those who reported low sodium intake also reported implausibly low calorie intake. But they weighed no less than the average participant. Thus it is clear that their food intake reports were inaccurate, or possibly falsified. Thus their low sodium intake is equally likely to be inaccurate or falsified. The phrase for this is - garbage in, garbage out.

When looking at the ratio of sodium per 1,000 calories, those with a higher concentration of sodium in the diet had a higher risk of dying. The authors' conclusion of an inverse association of sodium to risk of dying is false by their own data. (The fact that lead author Michael Alderman had been consultant to the industry's Salt Institute should probably be overlooked.) The Nutrition Action Health Letter is not alone in condemning the study. Dr. Lawrence Appel of Johns Hopkins University states, "No good researcher would even [want to] publish the unadjusted data. I don't know how this study got published. It's outrageous." My concern is not about salt in the diet; it is about the belief that peer review establishes good science. This example points out that good medical journals with a strong tradition of peer review can publish papers that lack merit. The reality is that publishing in a good orthodox journal is not a guarantee of good science; nor is it true that publishing in an alternative journal means less good science. Wherever the paper is published it must stand on its own merits. If a study clearly reports its methods and findings, it is up to the individual reader to judge its contents and merit. The fact of peer review does not remove this obligation.

John R. Lee, M.D.

Rhubarb of the Month*

Keeping an eye on the medical literature is almost an exercise in futility. First of all, the literature is now so vast that, like the tower of Babel, it is beyond comprehension. Second, no matter what the topic, you can be sure that various references on the same subject will report contradictory results. And third, summaries are simply too treacherous for judging the validity of the reference's claims: you have to read the entire study. Serendipity is another factor: sometimes the accidental timely coincidence of contradictory articles will produce insights that otherwise would have never occurred.

This month's example involves two articles on postmenopausal osteoporosis, one from JAMA (Dec 2, 1998) and the other a 1990 study from the J. Bone Mineral Research sent to me by a keen-eyed friend, both of them by coincidence emanating from Chicago. The 1990 study involved oophorectomy-induced bone loss in mature rats. The bone loss was evident 20 weeks after oophorectomy, at which point some rats were put on estradiol and a second group was put on progesterone in physiologic doses. Twenty weeks later, the bone mineral density (BMD) of the E2-treated rats showed stabilization of bone loss, whereas the progesterone-treated rats showed a statistically significant improvement of BMD. Not only had the BMD increased to pre-oophorectomy levels, but so also did the calcium content and the femur strength, indicating a reversal of the oophorectomy-induced bone loss. This report was printed not only in the J. Bone Mineral Research but also in something called Cell and Materials Supplement 1, 1991, (pg 108) in Scanning Microscopy International, Chicago, IL 606663. Thus, there exists a well-done animal study confirming that progesterone, even in the absence of estrogen, does indeed promote new bone formation and reversal of osteoporosis. This is strong evidence for including progesterone in the management and treatment of postmenopausal osteoporosis.

The JAMA report4 is entitled "Primary Prevention of Postmenopausal Osteoporosis" with the subheading, "Updates Linking Evidence and Experience." It is a remarkably short article, considering the complexities of bone physiology, only 2 pages, buttressed with (I kid you not) only 17 references. The report promotes the uses of calcium and vitamin D (all in one paragraph), hormone replacement therapy (HRT) (without defining it), and bisphosphonates. The bisphosphonate portion describes their rather minimal benefit of 0.7% increase in BMD overall and 3.5% increase in lumbar BMD after two years of treatment, comparing that to HRT which showed a 1%-2% greater response than 5 mg of alendronate. The authors never mentioned the bone effect of diet, nutrition, exercise, progesterone or testosterone. They never mentioned the fact that the estrogen effect was dependent on the sex hormone binding globulin (SHBG), the protein to which estrogen is bound and which blocks estrogen bioavailability. They did not reference the recent NEJM study5 that showed SHBG concentrations were more important (in a negative way) than estradiol concentrations, per se. Nor did they mention that estradiol levels sufficient for optimal bone protection are present in two-thirds of women aged 65-80 without any estrogen supplementation. Nor that the estradiol supplement dosage necessary to restore estradiol concentrations to optimal bone-sparing levels in estradiol-deficient women is only one-tenth of that used in conventional HRT.

It should be obvious to knowledgeable physicians that this JAMA article is a totally inadequate and unconvincing piece of work, feebly supported by a highly selective and minimal list of 17 references (out of perhaps 1700 appropriate references). This has no place in a journal such as the JAMA that purports to advance the education of its readers. One wonders how it came about. The clue for its publication, I believe, lies in its subtitle concerning the new buzz-word, "evidence." Conventional medical journals are touting the claim that conventional medicine is evidence-based, and alternative medicine is not. If this is their proof of "evidence", the alternative physicians have nothing to fear.

by John R. Lee, M.D.

References:

*1. Am. Heritage Dictionary: rhubarb (3); heated discussion, quarrel, or fight

2. Barengolts EI, Gajardo HF, Rosol TJ, et al. Effects of progesterone on postovariectomy bone loss in aged rats. J. Bone Miner. Res. 1990; 5(11): 1143-7/

3. Barengolts EI, et al. Comparison of the effects of progesterone and estrogen on established bone loss in ovariectomized rats. Cell and Materials Supplement 1, 1991, pg 108, in Scanning Microscopy International, Chicago IL 60666.

4. Levinson W, Altkorn D. Primary prevention of postmenopausal osteoporosis. JAMA 1998; 280: 1821-2.

5. Cummings SR, Browner WS, Bauer D, et al. Endogenous hormones and the risk of hip and veterbal fractures among older women. NEJM 1998; 339: 733-8.

Mystery Solved

Editorial
By John R. Lee, M.D.

Solving a mystery is often fun. Recently, an article I came across in reading Chemical Abstracts, Vol. 128, No. 8, 1998, resolved a mystery that had been bugging me for some time. It started out in November, 1997, when I received a letter from a Prof. David Purdie of Hull Royal Infirmary, Hull, UK, challenging me to debate him on the subject of using transdermal progesterone for the prevention and treatment of osteoporosis. Given the topic, it was difficult to turn down such a challenge.

The debate was held on April 27, 1998, at the Royal College of Physicians, in Regents Park, London. The terms of the debate required me to lead off with a 20-minute argument in favor of using transdermal progesterone for osteoporosis, after which Prof. Purdie would have 20 minutes to argue that transdermal progesterone has no place in osteoporosis prevention or treatment. Prior to the debate, I wondered why Prof. Purdie, an OB/GYN academician, would feel it necessary to challenge me about using progesterone for osteoporosis. To my knowledge, he had not published any research concerning progesterone or osteoporosis.

The mystery deepened the day before the debate when I discovered that Drs. Cooper and Whitehead had written a letter to be published that weekend in Lancet, claiming that they had proved that progesterone is not absorbed well enough through the skin to have any therapeutic effect. Further, they had managed to have a press release released to the British media announcing this "fact" prior to the actual publishing date, a practice not usually allowed by Lancet. Drs. Cooper and Whitehead are not only leading hormone researchers in England but are friends and associates of Prof. Purdie. The timing of their letter and the press release on the same day as the debate was certainly a curious coincidence.

Friends of mine further discovered that the claim of progesterone's poor topical or transdermal absorption was based on serum or plasma tests. With this information I was able to include in my 20-minute talk the information that serum or plasma tests measure non-bioavailable protein-bound progesterone and that it is not a good test for active hormone in the blood. I explained the progesterone is absorbed without becoming protein-bound and thereby the bioavailable portion of the hormone, being fat-soluble and hydrophobic, would not be adequately measured by serum or plasma tests. Further, I explained the superiority of saliva hormone assay in this regard. My point was that, if Prof. Purdie based his argument on a serum or plasma test, he was merely misinformed. And I was still able to present my experience showing that progesterone did indeed have a benefit in maintaining and improving bone mineral density, and preventing fracture.

This put Prof. Purdie in an uncomfortable position since he had based his argument on the presumption of progesterone's poor absorption, an argument I had just demolished. He was left with the argument that progesterone's role in osteoporosis had not, as yet, been studied by the double blind, placebo controlled, randomized tests that Prof. Purdie preferred.

In the Q A session that followed, Prof. Purdie was forced to admit that many advances in medicine come about from clinical experience and not from tests academicians such as Prof. Purdie might prefer. Further he admitted he had never used topical progesterone for any medical condition, let alone osteoporosis, nor had he ever used saliva assay for measuring bioavailable hormone concentrations, nor was he aware of any study that disproved progesterone's role in bone or in the prevention or treatment of osteoporosis.

A further benefit of the debate was the publication in Lancet 1998; Vol 352, of my letter explaining the error of Drs. Cooper and Whitehead in presuming that plasma or serum accurately reflects the true bioavailable concentrations of progesterone circulating in blood. The letter included the 1997 comparison by Dollbaum CM and Duwe GF of serum and saliva concentrations one finds when testing the absorption of progesterone after topical application.

But a mystery remained. What had motivated Drs. Cooper and Whitehead, and their stalking horse, Prof. Purdie, to purposely try to damage the credibility of readily available transdermal progesterone? The answer has now become clear. These doctors are involved in research and promotion of a newly-patented bioadhesive vaginal gel containing, guess what?, natural progesterone. The stated purpose of this vaginal natural progesterone product is to prevent estrogen-induced endometrial hyperplasia in estrogen-treated postmenopausal women. In their paper [Am J Obstet Gynecol 1997; 177(4): 937-941], they report that vaginal doses of progesterone in doses of both 45 mg every 2 days or 90 mg every 2 days prevents endometrial hyperplasia.

Why are we not surprised to find this result? The same effect is obtained by using any transdermal progesterone applied in doses of 20-30 mg daily. It is no secret that unopposed estrogen is the cause of endometrial hyperplasia.

Clearly, the April 1998 debate was a charade. Prof. Purdie was not interested in debating the merits of progesterone for osteoporosis, his purpose was to publicly discredit existing topical progesterone products in order to secure a market for their patented bioadhesive gel product. Their claim that topical progesterone is poorly absorbed is false and they know it. They used their considerable reputations to trick Lancet into publishing their irresponsible letter. The fact that they failed to reveal their involvement with the promotion of the vaginal cream at the time of the debate is further evidence of their connivance in this matter. In their research involving the vaginal progesterone, they could have added a test group using transdermal progesterone, but they did not. When we asked the manufacturer if they plan to compare their endometrial results with transdermal progesterone, the answer was "No." The only purpose of the bioadhesive gel, which is esthetically unpleasant to users, is that the gel can be patented. This is blatant commercialism at its worst.

This whole affair is an example showing that even very good medical researchers can be seduced into shady commercial exploitation. The interests of good medical practice are violated for the purpose of non-defensible profit.