The Natural Medicines Handbook: The Truth about the Most Effective Herbs, Vitamins, and Supplements for Common Conditions
If you want to read their other submissions, please click the links.
Chapter 1 – Natural Medicines Are Popular
It was an embarrassing night! I was hosting the first of what would turn out to be 854 episodes of the prime-time cable TV show, Ask the Family Doctor. The show ran for five years, starting on America’s Health Network, which two years later became Fox’s The Health Network. I had never done television before but was fortunate to be chosen from scores of far-more-experienced TV doctors to be the host for this nightly national live television show.
Before our inaugural broadcast on March 25, 1995, each of the show hosts had received extensive training. There were sixteen of us: the eight primary hosts, like me, worked Monday through Friday. We each had a live two-hour program, and the first hour of each broadcast was rerun during the overnight hours. We each had a substitute to cover the Saturday and Sunday shows. We were each prepared based upon this training and our practice experience to answer common questions about common maladies – along with interviewing experts on a wide variety of topics of interest to our viewers. However, I was not prepared for the first questions that night:
- Dr. Larimore, does vitamin C work for colds?
- What multivitamin should I take?
- What supplement could I use for brain health and memory?
Three strikes. I didn’t have a clue about any of these questions. The good news was that we were trained to explain that if we didn’t know the answer to an inquiry, we would say that, promise we’d research it, and answer the inquiry on the next edition of the program.
What I quickly learned those first few weeks of my program was:
- the public had an insatiable interest in natural medicines and alternative medicine,
- I knew very little about either,
- I needed a crash course in the truth, whatever it was, about complementary and alternative medicine (CAM), and
- the term “natural medicines” included herbal products, vitamins, and dietary supplements.
My expertise grew by leaps and bounds as I attended conferences and participated in distance learning with scores of experts. This allowed me to begin incorporating evidence-based information about natural medicines and CAM into my patient care, speaking, writing, and radio and TV programs.
My growing expertise led to two best-selling books that I co-wrote with Dónal O’Mathúna, PhD: Alternative Medicine: The Christian Handbook (2001), which was updated and revised to Alternative Medicine: The Options, the Claims, the Evidence, How to Choose Wisely (2008). Both books did very well because the use of natural medications was hugely popular. It still is and is approximately a $30 billion industry in the U.S., with over 100,000 products on the market in 2018. In 2020, Natural Medicines™ listed over 185,000 products – including discontinued products.
A 2019 consumer survey by the Council for Responsible Nutrition (CRN) announced, “Dietary supplement use reaches an all-time high.” CRN added that “the use of dietary supplements among U.S. adults increased ten percent over the past decade,” and “77 percent of U.S. adults take dietary supplements, as opposed to just 65 percent in 2009.” The CRN survey revealed, “Among all the age groups, adults between the ages 35-54 have the highest usage of dietary supplements at 81 percent.” It was 79 percent for U.S. adults aged 55 or older and a striking 70 percent for those aged 18-34.
Additionally, CRN found, “While vitamin/mineral supplements remain the most popular category among supplement users, the overall use of herbals/botanicals has significantly increased in the past five years. In 2019, 41 percent of supplement users reported they had taken herbals/botanicals in the past twelve months – up 13 percentage points from 2013.”
Another 2019 national survey reported 86 percent of Americans take natural medicines, especially vitamins and dietary supplements – with 48 percent of adults taking vitamins and 39 percent taking supplements – most to “maintain health and prevent disease.” Another national survey showed that fifty-two percent of U.S. adults reported the use of two natural medications a day while ten percent reported using at least four such products daily.Although these percentages vary from source to source, depending upon who and how they surveyed, the point is that natural medicines are very popular.
What are the most popular natural medicines? In its “2020 Survey of Supplement Users,” ConsumerLab.com reported, “Despite a slight decrease, vitamin D (-0.4 pts) remained the most popular supplement, purchased by 66 percent of respondents. With its continued rise in popularity, magnesium secured its place as the second most popular supplement and is used by 53.5 percent of respondents, surpassing [third-place] fish oil. CoQ10 remained in fourth place this year, followed by multivitamins (42.4 percent, -1.3 pts), probiotics (38.9 percent, -2.7 pts), curcumin and turmeric (34.8 percent, -9.9 pts), vitamin C (including rose hips, 34.5 percent, -1.0 pt), B-complexes (31.2 percent, -1.1 pts), and vitamin B12 (cobalamin, 30.3 percent, +2.2 pts). Among the top 50 supplements, 30 declined in popularity, while only 19 showed an increase.”
CRN reported that among the U.S. adults they surveyed, multivitamins were the most popular dietary supplement (58 percent) followed by vitamin D (31 percent per), vitamin C (28 percent), protein (21 percent), calcium and vitamin B or B complex (20 percent each), and omega 3 fatty acids/fish oil (16 percent).
Again, the disparate results reflect different populations surveyed and different survey tools. For example, ConsumerLab surveyed 9,782 U.S. adult subscribers (with over 82 percent taking at least four different supplements daily). In contrast, the CRN survey included a U.S. sample of 2,006 adults, of whom only 1,529 considered themselves supplement users.
- According to ConsumerLab’s 2020 survey, the top-rated supplement brands based upon overall consumer satisfaction:
- Catalog/Internet Brand: Andrew Lessman’s ProCaps Laboratories
- Direct Selling (MLM) Brand: Young Living™
- Discount/Warehouse Brand: Kirkland Signature™ (Costco®)
- Food/Drug/Mass: Broad Product Line: Nature Made®
- Food/Drug/Mass: Narrow Product Line: HPF Cholestene™
- Grocery Store Brand: Trader Darwin™ (Trader Joe’s®)
- Healthcare Practitioner Brand: Standard Process®
- Health Food Store Brand: Sports Research™
- Pharmacy Brand: CVS
- Vitamin Store Brand: Vitamin Shoppe®
- The top-rated supplement merchants based on overall consumer satisfaction were:
- Catalog/Internet: Life Extension®
- Direct sales (MLM): doTerra®
- Grocery Store: Natural Grocers®
- Mass Market: Target
- Online Multi-Category Retailer: Amazon.com®
- Online Supplement Retailer: Vitacost®
- Pharmacy: Walgreens
- Vitamin Store: The Vitamin Shoppe®
- Warehouse Store: Costco®
- Top-rated Supplement Brands on Consumer Satisfaction with Specific Types of Supplements:
- Calcium: Brand: Kirkland Signature™ (Costco®)
- CBD/Hemp Extract: Bluebird Botanicals
- CoQ10: Member’s Mark (Sam’s Club®)
- Joint Health: Doctor's Best
- Melatonin: Life Extension®
- Multivitamin: Life Extension®
- Omega-3s: Life Extension®
- Probiotic: Swanson®
- Vitamin D: Kirkland Signature™ (Costco®)
The Nation Center for Complementary and Integrative Health (NCCIH) writes, “A lot of people believe that when it comes to medicine, ‘natural’ is better, healthier, and safer than ‘unnatural’ or synthetic drugs.” They add, “Some people also believe that ‘natural’ products are safe because they believe these medicines are free of chemicals.” Around the world, people seem sold on the marketing claims that these products will allow them to avoid the “toxic, unnatural drugs” foisted upon them by “the evils of Big Pharma” that “seeks enormous profits over the health and well-being of the humans it serves.”
An article in Science-Based Medicine adds, “Supplements are marketed as safe, natural, and effective, and there is no question that messaging has been effective.” CRN proclaims, “Consumer confidence in products and trust in industry remain strong. The majority of U.S. adults – 87 percent – have overall confidence in the safety, quality, and effectiveness of dietary supplements. … Additionally, … 78 percent of Americans perceive the dietary supplement industry as being trustworthy.”
THE BOTTOM LINE? It appears consumers of all ages in the Asia-Pacific, North American, and European regions have come to believe natural medicines are risk-free magic bullets for health and wellness, as well as to treat every common malady. Globally, people are in love with natural medicines and often use them across all age groups and for a wide variety of reasons. The popularity and use of these products have been increasing every year for decades, and most believe they are both safe and effective. Unfortunately, false advertising and inflated claims are the norm. Shockingly, there are dangers that have been withheld from you. In the next chapter, I share the truth with you and reveal the secrets that some natural medicine marketers are hoping you won’t ever learn.
Chapter 2 – Natural Medicines Are A Problematic
I find most of my patients are surprised to learn that, as JoAnn Manson, MD, chief of the Division of Preventive Medicine at Brigham and Women’s Hospital in Boston, writes, “Most supplements on the market are not tested for either efficacy or safety.” Natural Medicines™ explains, “Most dietary supplements cannot claim to treat, cure, or prevent a disease. Doing so makes them legally unapproved drugs.”
Unfortunately, the FDA and the Federal Trade Commission (FTC) often have to issue warnings letters to companies whose “advertisements and labels tout the supplements’ ability to treat or cure Alzheimer’s, cancer, and myriad other diseases. The warning letters specifically call out the manufacturers for making unapproved drug claims and deceptive statements.” Natural Medicines explains that “many supplement companies bank on the ignorance of consumers and continue to skate this line. It can sway those with serious medical conditions away from using proven therapies.”
“Even food products that are allowed to make specific health claims have not been ‘approved’ by the FDA the way that drugs are,” says Meredith Worthington, PhD, Director and Senior Editor at Natural Medicines. “These health claims indicate that the FDA ‘acknowledges’ that the food product(s) may have evidence of benefit, but this process is not the same as the approval process for drugs.”
A 2018 editorial in the Journal of the American Medical Association (JAMA) reported, “Clinicians and patients should be aware that the U.S. FDA is not authorized to review dietary supplements for safety and efficacy prior to marketing.” This is unfortunate because, like any medication, natural medications have the potential to cause harm. I often say, “If a substance has no side effects, then you can be 100 percent sure it has no effects.”
Besides the fact that “… makers of dietary supplements are not required to prove effectiveness, safety, or quality of a product before marketing their products,” even more disturbing is that the manufacturers of natural medicines “… are not required to report [the] discovery of adverse effects to the FDA.”
In testimony before the U.S. Congress in 2010, Tod Cooperman, MD, the President and Founder of ConsumerLab.com, said, “Based on tests of over 2,000 dietary supplements representing over 300 different brands, we find that one out of four has a quality problem. Problems have been found in products from every size of manufacturer and are most common in herbal supplements, multivitamins, and products with ingredients that are newer to the market.”
Cooperman added, “Our most recent tests of herbal supplements show that 46 percent contained less than their expected amounts of key compounds.” Of course, the problem with this is that if you don’t get enough of the natural medicine that you might need, then it’s not likely to work, and you’ve just wasted your money. Imagine buying an antibiotic, an anti-seizure drug, or a heart drug that you thought contained a particular amount of medicine, but it did not? You’d be justified to be outraged! But that’s not the only issue with these products. ConsumerLab cites these common troubles with natural medicines sold in America:
- Tablets that won’t break apart properly to release all of their ingredients and therefore are much less likely to be absorbed.
- A lack of proper labeling because they had too few or too many nutrients.
- Deceptive labeling suggesting more ingredients than provided or components that are not even in the product at all.
- Products containing potentially dangerous, even cancer-causing or heavy metal contaminants,
- A lack of voluntary warnings which could help consumers avoid potential problems, such as ingredients in amounts above known tolerable levels, and a lack of public access to adverse event reports filed by manufacturers with the FDA.
- Faulty products left on the market due to inaction by manufacturers or “quiet” recalls announced to retailers but not to the public.
- The illegal addition of prescription or investigational drugs, particularly those sold for “erectile dysfunction,” “bodybuilding,” “brain health,” and “weight loss.”
So, in 2010, based upon ConsumerLab tests, if you purchased a natural medicine in the U.S., you had between a one-in-two and a one-in-four chance of buying a product that did not meet even minimum standards for safety. You’d assume things would be better ten years later, but you’d be wrong. In early 2020, ConsumerLab reported that about one in five supplements in the U.S. and Canada that they tested did not meet one or more quality parameters. This is true even though ConsumerLab’s testing tends to focus on the more popular and established brands that would be expected to have the best findings,
Also, ConsumerLab reported in early 2020 on information from the FDA revealing that 51 percent of 598 U.S. dietary supplement manufacturing facilities audited from October 1, 2018, through September 30, 2019, failed inspections and received letters of non-compliance with good manufacturing practices (GMP). ConsumerLab.com received the results under the Freedom of Information Act.
The good news is that the high-water mark in the failed inspections in U.S. facilities was 70 percent in 2012. “Although an improvement over past years,” says Tod Cooperman, MD, “the most recent FDA audits show that, overall, dietary supplement manufacturers need to do a lot better.” He added, "Consumers often express concern about supplement ingredients from China, but the U.S. is doing no better than China based on this limited sampling. People may also be surprised that four out of five facilities were noncompliant in Germany, where many herbal supplements have drug status. However, our tests have shown issues with some products from Germany.”
The FDA noted a median of four infractions at facilities that received notices but did not release a list of the specific violations of each facility. Nevertheless, the most common breaches, each observed at more than 15 percent of non-compliant facilities, included:
- not establishing product specifications for the identity, purity, strength, or composition of the finished dietary supplement (25.25 percent),
- not establishing or following written procedures for quality control operations (21.64 percent),
- not producing batch records, which include the complete information relating to the production and control of each batch (15.08 percent).
If you don’t think the situation is problematic, this may make the hair on the back of your neck stand up: A 2015 study by the New York Attorney General concluded, “DNA testing … shows that, overall, just twenty-one percent of the test results from store brand herbal supplements verified DNA from the plants listed on the products’ labels – with 79 percent coming up empty for DNA related to the labeled content or verifying contamination with other plant material.” In other words, they found that if you purchase one of the tested herbal supplements, you had a four-in-five chance of being duped.
In February 2019, FDA Commissioner Dr. Scott Gottlieb said, “The growth in the number of adulterated and misbranded products – including those spiked with drug ingredients not declared on their labels, misleading claims, and other risks – creates new potential dangers.”
Science-writer Markham Heid reported, “Despite recent studies that find supplements are frequently contaminated or that the best way to get nutrients is through food, Americans’ interest in supplements is only growing. And experts say many supplement users don’t recognize or appreciate the risks that accompany the use of these products.” Dr. Manson compares “the current regulatory environment to ‘the Wild West.’” In 2018, a study in JAMA reported, “From 2007 through 2016, 776 adulterated dietary supplements were identified by the FDA, and 146 different dietary supplement companies were implicated.”
A perfect recent example surrounds the growing popularity of cannabidiol, called CBD. The market size is soaring at about 35 percent CAGR and could reach $1.3 billion USD by 2024. The New York Times reports, “We are bombarded by a dizzying variety of CBD-infused products: beers, gummies, chocolates and marshmallows; lotions to rub on aching joints; oils to swallow; vaginal suppositories for ‘soothing,’ in one company’s words, ‘the area that needs it most.’
Even CVS and Walgreens have announced plans to sell CBD products in certain states.” The Times adds, “Many of these products are vague about what exactly CBD can do. … Yet promises abound on the Internet, where numerous articles and testimonials suggest that CBD can effectively treat not just epilepsy but also anxiety, pain, sleeplessness, Crohn’s disease, arthritis, and even anger.” The FDA sent out a flurry of letters warning companies not to make medical claims.
In 2015, the FDA reported it found that many CBD-labeled products contained very little CBD. Two years later, another study published in JAMA found that in eighty-four CBD products sold online, forty-three (fifty-one percent) had more CBD than advertised, and twenty-six (thirty-one percent) had less. Also, eighteen of the eighty-four products (twenty-one percent) contained THC (the psychoactive component of marijuana), with none listed on the label.
I bet you’re finding these facts almost unbelievable. “After all,” you might argue, “People around the world respect the FDA for its pharmaceutical policies. No other country is as rigorous in protecting the public. We can rely on our prescriptions and over the counter medications to be high-quality, safe, and effective.” And you are 100 percent correct.
Unfortunately, the same is not true for natural medicines. Maybe you would now agree with Dr. Manson when she compares “the current regulatory environment to ‘the Wild West.’” How did this ever come to be? Scott Gavura, writing for Science-Based Medicine, explains why:
It’s a consequence of legislation deliberately designed to weaken the FDA’s ability to regulate and provide oversight of supplements. The Dietary Supplement Health and Education Act of 1994 (DSHEA) was an amendment to the U.S. Federal Food, Drug, and Cosmetic Act that established the American regulatory framework for dietary supplements. It effectively excludes manufacturers of these products from many of the requirements that are in place for prescription and over-the-counter drugs. Amazingly, it puts the requirement to demonstrate harm on the FDA rather than the onus on the manufacturer to show a product is safe and effective.
Natural Medicines adds:
One of the biggest differences between drug regulations and dietary supplement regulations is the approval process. There is NO approval process for dietary supplements. As long as supplements contain dietary ingredients that were either already used in supplements before DSHEA (1994), or established as reasonably safe since then, they can be sold to consumers lawfully.
This is very different from drugs, which must go through an extensive approval process BEFORE entering the market. Basically, drugs are assumed unsafe until proven safe, whereas supplements are assumed safe until proven otherwise.
Besides the many regulatory difficulties with natural medications, there are many other potential dangers many consumers are not aware of. For example: “Although many people don’t realize it, dietary supplements can cause serious health concerns when taken incorrectly. They are a common reason for calls to U.S. Poison Control Centers, and dietary-supplement-related calls are on the rise,” says a report from Natural Medicines.
In 2017, researchers published data showing that in just over ten years, “U.S. Poison Control Centers received nearly 275,000 calls related to dietary supplements.” That’s one call every 24 minutes! Natural Medicines writes, “The majority of the calls involved miscellaneous dietary supplements, followed by botanicals and hormonal products. Of these, seventy percent related to dietary supplement use in children younger than six years old. These exposures were mostly accidental, so it’s important to store dietary supplements out of the reach of children.”
Another critical issue is the possibility of the interactions and direct toxicities of natural medicines. The NCCIH warns, “Although there is a widespread public perception that herbs and botanical products in dietary supplements are safe, research has demonstrated that these products carry the same dangers as other pharmacologically active compounds. Interactions may occur between prescription drugs, over the counter drugs, dietary supplements, and even small molecules in food – making it a daunting challenge to identify all interactions that are of clinical concern.”
Now, I’m not a guy that’s for big government and lots of regulations. But in healthcare, I think they are often necessary and helpful. And in the case of natural medicines, the way DSHEA weakened the rules to protect health consumers could have been predicted to increase the risks of manufacturers taking shortcuts, which could result in inconsistent product quality and even irresponsible behavior from some manufacturers. Unfortunately, that’s precisely what we find when we look at supplement sales today.
Worse yet, it has led to far too many catastrophic harms suffered by consumers. In 2019, researchers from Harvard’s T.H. Chan School of Public Health found that of the single-supplement-related adverse event reports in children and young adults, “approximately 40 percent involved severe medical outcomes, including death and hospitalization. Supplements sold for weight loss, muscle building, and energy were associated with almost three times the risk for severe medical outcomes compared to vitamins. Supplements sold for sexual function and colon cleanse were associated with approximately two times the risk for severe medical outcomes compared to vitamins.”
There’s also the issue of effectiveness. Despite ubiquitous advertising to the contrary, for natural medicines as a broad category, there is very little reliable evidence to support most of the claims. “With a few exceptions,” Gavura writes, “the research done on dietary supplements is unconvincing and largely negative.” The National Institute on Aging (NAI) adds, “Some ads for dietary supplements in magazines, online, or on TV seem to promise that these supplements will make you feel better, keep you from getting sick, or even help you live longer. Often, there is little, if any, good scientific research supporting these claims. Supplements may cost a lot, could be harmful, or simply might not be helpful.”
By now, you may be thinking that I’m against natural medicines. Nothing could be further from the truth! I’ve recommended and promoted them for over two decades; however, I use the same standards for my natural medication recommendations as I do for my pharmaceutical and over-the-counter medication recommendations: I only prescribe or suggest products that are both safe and effective.
I have yet to find a patient or a consumer who doesn’t want all of their medicines, prescription, over the counter, or natural, to be manufactured to the highest quality standards and to have accurate labeling. I would think that we all could agree that the product label should accurately describe what’s in the bottle, and there should be no exceptions to this. And yes, there should be no substituted ingredients and absolutely no contaminants at all. Again, no exceptions.
Unless Congress repeals or revises the DSHEA Act, the only hope for the average health professional, patient, or consumer considering a natural medication is to ask the questions I’ve adapted from PBS’ 2016 Frontline special, “Supplements and Safety:”
- Has the product triggered any health warnings or sanctions? Searching the FDA’s website for a list of all recalls (tinyurl.com/y5kxnwbu) will identify if the product in question has problems. ConsumerLab.com notifies its subscribers of significant warnings and recalls. Nevertheless, given the few resources put into monitoring supplement safety issues, a lack of notices is no assurance of quality or safety.
- Is the product too good to be true? While supplement makers cannot legally make unfounded claims of efficacy, it’s not hard to find these claims on products or the Internet. Again, it’s important to note that there are no magic bullets out there. Outlandish claims should not be believed.
- Is there evidence that the supplement does what it promises? Look to reputable sources that summarize the evidence without bias. If there’s no good evidence supporting any efficacy claims, it’s a giant red flag that the product is probably bogus. As I’ve mentioned, my favorite non-biased source for whether a natural medicine is safe and effective or not is Natural Medicines™.
Frontline recommends you do a little research before taking any natural medicine. I would wholeheartedly agree. However, I must warn you that, in general, the Internet is an unreliable place to do your research. One study reported that among health websites, retail websites presenting information on products they were selling had the lowest level of medical accuracy (only nine percent). On the other hand, government websites (.gov) and websites of national organizations (.org) had the highest level of accuracy (eighty-one and seventy-three percent, respectively). Note that even these “reputable” sources weren’t close to 100 percent accurate!
Shockingly, the same study reported that educational websites (.edu sites, e-books, peer-reviewed articles) only had 50 percent accurate medical information. The majority of the books found by search engines either provided outdated or irrelevant information. Blogs and websites of individuals are even worse, having low rates of accuracy (twenty-six and thirty percent, respectively).
However, there are several trustworthy, medically accurate, unbiased, and evidence-based sources of information I highly recommend:
- The National Institutes of Health (NIH) has summaries about the most commonly consumed supplements — vitamins and minerals — in a series of fact sheets.
- The U.S. National Library of Medicine’s MedlinePlus has similar information about drugs, herbs, and supplements. For a deeper dive into the science behind a specific supplement, explore the Library of Medicine’s PubMed Dietary Supplement Subset.
- Resources are also available for specific groups: For example, the Department of Defense offers information about the safety of specific supplements for service members. Older adults can find resources specifically designed for them by the FDA, FTC, and NIA.
- The National Institutes of Health (NIH) contains the National Center for Complementary and Integrative Health (NCCIH) provides information about complementary health products and practices – including many natural medicines.
These sites can give you general information on natural medicines but are of very little help when it comes to choosing specific products. Therefore, Frontline’s number one recommendation, and mine also, is that before you purchase any vitamin, herb, or supplement, you ask the question, “Has the product been tested by an independent lab that makes the data available to the public?”
This is critical and is the only real protection health professionals, and consumers have from unscrupulous manufacturers, untrained salespeople, and fraudulent, contaminated, or dangerous products. The independent quality testing labs (such as ConsumerLab.com, the National Sanitation Foundation [NSF® International], Labdoor, and the United States Pharmacopeia [USP®]) purchase these products from the same places you and I would (big box stores, health food stores, pharmacies, the Internet, etc.) and test them or have them tested for quality.
My two go-to organizations for evidence-based information on natural medicines, as well as which specific products to recommend, are ConsumerLab.com and Natural Medicines™. ConsumerLab uses a variety of labs for its tests to “Approve” particular products, while Natural Medicines highlights products that are USP®-“Verified.” They may also, in the future, include other certifications in their commercial product database. Both organizations are independent of the natural medicine industrial and sales complex and make their findings and recommendations publicly available via Internet subscription.
You can also get online information from NSF®, USP®, and Labdoor about the products they have evaluated.
THE BOTTOM LINE? I’ve adapted the cautions NCCIH gives health consumers:
- The amount of scientific evidence we have on dietary supplements varies widely – we have a lot of information on some and very little on others.
- Some dietary supplements can be good for your health; however, most have insufficient evidence that they are safe and effective.
- Legitimate peer-reviewed studies of many supplements frequently do not support claims made about them.
- Supplements you buy from stores or online often differ in important ways from products tested in studies.
- Some products marketed as dietary supplements may contain prescription drugs not allowed in dietary supplements or other ingredients not listed on the label. Some of these ingredients may be unsafe.
- The FDA does not have the resources to test all products marketed as natural medicines.
- And most shockingly of all, you cannot trust what’s many labels say.
With these cautions understood, let’s unravel the mysteries of what, if any, natural medicines should you consider for the following common conditions. Let’s start by learning about the most effective ways to increase the quantity and quality of your remaining years.
THE EXTENSIVE CITATIONS WERE NOT INCLUDED IN THIS SAMPEl