Using Personal Experience As Evidence

In this blog, Elaine Finucane who works with iHealthFacts – an online resource where the public can check the reliability of a health claim circulated by social media – explains why anecdotes are unreliable evidence. This is the fifth blog of our special series on Evidently Cochrane: "Oh, really?" 12 things to assist you lot question health communication.

Page originally published: 25 September 2020. Revised and republished: 09 June 2022 to include updated testify.

Take-home points

'It ain't and so much the things nosotros don't know that go united states of america into problem. Information technology's the things nosotros know that but own't so.' – Anonymous

We have all grown up with erstwhile wives' tales advising u.s.a. to use specific treatments to cure common and not then common ailments. Stories and treatments that have been passed down through centuries and generations.

How many of us take rubbed dock leaves on nettle stings to relieve the painful sting? Or, maybe drank flat soda to assistance relieve tummy pain?

We now live in a guild where health claims, some reliable and some unreliable, are created and spread at the touch of a push. During the COVID-19 pandemic, we have experienced a rapid increment in these wellness claims, with The World Health Organisation labelling it an 'infodemic' of informationⁱ (where we are overwhelmed with data – some accurate and some not).

In response, iHealthFacts, an online resource where the public can quickly and hands check the reliability of a health claim, has been 'fact checking' claims submitted by the public. These claims have varied wildly, from advising members of the public to have the drug Chloroquine to forbid or care for COVID-nineteen, to a claim telling the public that being unable to hold i's breath for 10 seconds, without cough, is a manner of diagnosing COVID-xix². Often, these claims were supported by personal stories and individual experiences – just were not supported past evidence. While it can be very tempting to believe these claims, especially during these uncertain times, it is important to make well-informed decisions almost your wellness.

So what? What's wrong with assertive what nosotros are told?

If your friend Tom tells you, that after existence stung past a nettle, he rubbed a dock leaf on his skin and the pain vanished, it must be truthful, correct? Indeed if information technology worked for Tom, it has to work for you?

Well… not necessarily.

If dock leaves were a reliable treatment for nettle stings, then we would expect Tom'south feel to exist typical, merely is Tom's experience on its own enough bear witness to support this claim?²

Only considering Tom felt better after using a dock leaf does non mean that the dock leaf is responsible for Tom feeling better. Rubbing dock leaves onto the nettle sting may take distracted Tom from the pain of the sting. A placebo upshot may be responsible, where Tom believed the dock leaf would help, and this belief provided a beneficial upshotⁱⁱⁱ. Would he accept felt better if he did zero?

In other words, personal experiences – or even a series of personal experiences – can sometimes exist misleading. Sir Francis Galton coined the term 'regression to the mean' when he noted that extreme outcomes Outcomes are measures of health (for example quality of life, pain, blood sugar levels) that can be used to appraise the effectiveness and safety of a treatment or other intervention (for example a drug, surgery, or exercise). In inquiry, the outcomes considered about important are 'primary outcomes' and those considered less of import are 'secondary outcomes'. tend to be followed by a more normal or average one^four. Similar tossing a coin several times in a row and it always lands on heads – it is unusual and would be very difficult to repeat⁵. When 'regression to the mean' is practical to symptoms such as Tom'south pain, it tells us that the initial extreme pain he felt may have improved without treatment Something washed with the aim of improving wellness or relieving suffering. For case, medicines, surgery, psychological and physical therapies, diet and exercise changes. .

The power of many

When talking about dock leaves and nettle stings, it may seem harmless to apply personal experiences or stories to inform your healthcare decisions. Even so, when you apply the same logic to the claim that the drug chloroquine prevents or treats COVID-xix, a drug used to treat malaria and rheumatoid weather, such as arthritis, with its potentially serious side effects, the stakes become higher^six.

Rather than relying on Tom's personal experience, or a series of personal experiences to tell you whether dock leaves relieve nettle stings, a ameliorate option would be to expect at all the bachelor show on this intervention A handling, procedure or programme of health care that has the potential to modify the class of events of a healthcare condition. Examples include a drug, surgery, practise or counselling. . To examine the experiences of people of unlike ages, gender and ethnicities to see what issue, if any, dock leaves had on their nettle stings.

Nosotros call this process a systematic review of the evidence⁷. A systematic review looks at all the electric current evidence available to answer a question. Combining all the evidence, systematically, is likely to provide a more reliable, robust answer on which to base of operations your healthcare decisions.

For case, a Cochrane Review Cochrane Reviews are systematic reviews. In systematic reviews we search for and summarize studies that answer a specific enquiry question (east.one thousand. is paracetamol effective and safe for treating dorsum hurting?). The studies are identified, assessed, and summarized by using a systematic and predefined arroyo. They inform recommendations for healthcare and inquiry. (updated in May 2022), which included bear witness from 42 studies and 52,608 participants, examined signs and symptoms to decide if a patient presenting in primary care or hospital outpatient settings has COVID‐19 illness⁸. Despite online claims that beingness unable to hold one's breath for ten seconds, without coughing, is a mode of diagnosing COVID-nineteen, a claim supported by personal experiences, this review constitute that a single symptom or sign, including a cough or respiratory symptom, cannot accurately diagnose COVID‐xix.

Retrieve critically about wellness claims!

It is unwise to base healthcare decisions on personal experiences – or even a series of personal experiences. Personal experiences or anecdotes do not provide robust bear witness. Systematic reviews In systematic reviews we search for and summarize studies that answer a specific research question (e.g. is paracetamol constructive and prophylactic for treating back pain?). The studies are identified, assessed, and summarized past using a systematic and predefined approach. They inform recommendations for healthcare and research. of all the available evidence on an intervention help minimise bias Whatsoever factor, recognised or non, that distorts the findings of a study. For case, reporting bias is a type of bias that occurs when researchers, or others (due east.1000. drug companies) choose not report or publish the results of a study, or practise not provide full data nigh a study. and produce more reliable evidence to support informed healthcare decisions.

Join in the conversation on Twitter with @CochraneUK, @iHealthFacts1 and @elainefin4 or exit a comment on the blog. Please note, nosotros cannot requite medical advice and we will not publish comments that link to commercial sites or appear to endorse commercial products.

References (pdf)

Visit the iHealthFacts website to submit a health merits to be fact checked, or search for previously answered questions.

Visit the Teachers of Testify-Based Wellness Care website, where you can find resources which explain and illustrate why anecdotes are unreliable evidence.

This series of blogs is inspired by a list of 'Fundamental Concepts' developed by the Informed Health Choices.

Elaine Finucane reports that she is a Inquiry Associate with the HRB-Trials Methodology Research Network, NUI Galway and an evidence researcher for iHealthFacts. iHealthFacts (www.ihealthfacts.ie) is funded past the Health Research Board, Ireland.