16 points to consider when reading medical research.

I love the picture above. It sums up everything that irks me with media reporting of medical information and research studies. How many times have you seen the headline "New hope for cancer sufferers. Coffee the key to cure. Full story at 9." Everyone gets excited. Family members run to text patients. News feeds fill with retweets and shares. It's then picked up by multiple news outlets who lead their news hour with the same headline. Only to find with a few moments Googling, that this headline is based on a pilot study involving 3 rats at an obscure lab in Dusseldorf, that the researchers are still trying to work out exactly what levels of coffee are required for the effect, if the coffee brand they fed their tiny white cohort was what caused the death of the other 2 rats, and if any other intervening factors affected their results. In the meantime, desperate patients are chugging down their double espressos with wild abandon, thanks to a shoddy news report.

If you want to take control of your health and you want the best care for your condition you need to be able to sort through all this information and determine what is reputable, what is flawed, or in some cases, what is totally fraudulent. Being an informed patient increases your odds of a better health outcome. And, although often a dry and arduous task, understanding the information available about our varied disorders is worth the effort. The knowledge and skills required to interpret this information is what is known as health literacy.

Interpreting medical information, in all its forms, is not something we are all equipped to do. But in this, the first of 2 posts on the topic, I will give you some basic tips you can use to begin navigating the quagmire that is medical research and reporting.

1. Check your own biases: this is probably not the point you expected me to start with, but it is very important all the same. We all have biases. Some we are well aware of, and others that are more unconscious. Some have an inherent distrust of pharmaceutical companies or the medical establishment. Some have a unassailable belief in a particular treatment. Yet others have an unwavering confidence in a particular researcher, or research organisation. Rightly or wrongly, we need to acknowledge these before we can objectively interpret any medical information from the news media, websites, or well known medical journals. Time and again I've seen people read the same information, or watch the same documentary, and come out enthused by the support for their particular position. Positions which just happen to be the polar opposite of each other. As human beings we naturally drift towards information that is salient to us and our belief system. And we are more inclined to pick out that which reinforces our existing position. If we don't check these natural instincts, we may be doing ourselves, and our health, a disservice. 
2. Be sceptical: a healthy dose of scepticism is good for everyone. If something sounds too good to be true it probably is. If someone is touting a cure, especially one that only they know of, but they are willing to share with you for the price of your left kidney or your first born, run and run fast. If someone has a product or treatment that cures everything from halitosis to Alzheimer's disease, run even faster. There are also less than reputable medical, health, and alternative practitioners, who add in just enough medical and authoritative jargon to their claims, that it can make it hard to discern fact from fiction. Even if the information comes from a well known and reputable medical source be sceptical until they can show you the evidence that backs their claims. Remember that a good researcher will be his or her best critic, because they can see the limitations of their work and the areas requiring further research. 
3. Not all research gets published: Research studies that find no evidence for a particular theory or are neutral, rarely get published. For example, a recent BMJ article found that 29% of registered trials completed prior to 2009 were never published, with a higher rate (32%) of unpublished trials in industry funded research. With 78% of the unpublished trials not having their data available on ClinicalTrials.com. For every paper that finds support for particular theory/treatment, there may be a dozen with no or neutral findings that will never see the light of day, something that became readily apparent in the high pressure, research machine of my doctoral days. As such, we are already consuming inherently biased information.
4. Patient testimonies and anecdotal evidence: Often we hear personal testimonies/anecdotal evidence regarding particular treatments. These can be interesting from a patient perspective; I know I love to hear real experiences from other patients. However, alone they cannot be used to prove or disprove a claim, and are often more reflective of what is salient to that patient, rather than an objective review of the treatment. Clinics who use this type of information to sell their theory or treatment, are highly unlikely to put up negative testimonials on their sites. Thus once again you are being given biased information. Anecdotal evidence can become a motivator for needed systematic scientific studies, especially when organised patient groups can demonstrate a large volume of patients experiencing the same outcome. An interesting move of late are pharmaceutical reviews that scour the Internet (a process known as data mining) for patient reports of previously unknown side-effects. This has been successfully used for a number of medications leading to further research. 
5. Who is funding the research?: Researchers are required to disclose their funding sources and any conflicts of interest. Are they an employee, stock holder or board member of the company who owns the treatment they are researching? Does their membership of other competing organisations impact on the way they review a medication or therapy? These factors are not always an instant negative. There is nothing inherently wrong with researchers earning money from their discovery and the reality is that without pharmaceutical companies and big business, a lot of research would never be funded, but it is still an important factor to have in the back of your mind when you examine their research. Always question, does the researcher or media representative, have a vested interest in the treatment they are supporting? Are they independent and/or does the quality of their research override any potential conflicts?
6. PR machines: Some researchers know how to use PR and marketing far better than others, irrespective of their actual expertise. The same faces are seen on television or in newspapers, and they are touted as the go-to experts in their field. Patients and their families are more likely to be drawn to their research by their level of familiarity and legitimacy conferred by their frequent interviews. Others know how to use social media to sell their treatment or theory. Sheer volume of exposure will bring them to mind when you think of particular issues. There are many well respected researchers working behind the scenes, whose conclusions are just as valid or scientifically sound yet never really see the light of day except when uncovered by highly-caffinated and half-starved doctoral students doing the hard yards on PubMed late at night rather than sleeping. Even researchers associations with particular well-respected organisations will also lead to greater media exposure and give their work an inherent elevated legitimacy, regardless of the quality of their work.
7. It is possible to create the results you want: I will go into more detail about this issue in the next post, but it is an important fact to keep in mind. Researchers can, sometimes intentionally, sometimes unintentionally, set up their study in such a manner as to create the result they are looking for (eg through the inclusion/exclusion criteria they use to select participants). All studies have a theory or treatment they are wishing to test, and through a review of the available literature studies are designed in a manner to reflect this knowledge. However, depending on quality of the researcher, the design of the study may mean that only one result, the one wished for, is possible.
8. Research is rarely simple: News media, and some groups, will cherry pick the information they want to share (often to support a particular position) and share little in the way of detail (for example, the recent media reports linking salt to autoimmune conditions, many of which did not offer information on even the basics such as how excessive salt intake was defined for the purpose of the study). Often if you read the original research paper you will find that it is not as simple as first described in the media release. Life, humans and illness are complex, multi-factorial entities. Often it is more a case of, if a person has variable 1 and variable 2, and factor 3 is accounted for, then treatment A may lead to outcome B. Good researchers recognise the grey inherent in their research and spell out their constraints and need for further research. This however, does not make for a punchy sound bite. This follows onto the next point.
9. Find the original source: wherever possible find the original source article. This at least gives you the best starting point to evaluate the validity of a particular finding or claim. Do not rely on medical reporting or social media to give you an accurate portrayal of a research study. If in doubt contact the researcher. Every research paper will have contact details listed and many researchers love nothing better than to discuss their work with truly interested parties. 
10. Not all research results are generalisable to a wider patient population: For practical purposes, research is often conducted on a limited number of people, within a limited age range, often single sex, limited ethnicity etc. Time and financial constraints as well as protocols for particular forms of research eg drug trials, necessitate these limitations, but these factors can also limit what can be inferred from any results. Whilst certain research advances may be promising, if they are only based upon 10 non-smoking, non-drinking, vegan, left-handed, Caucasian women, aged 18-20, all named Bertha, with no other medical issues than the one in question, any conclusions must be tempered by the limited subset (these are often, but not always, pilot studies, which are used to test out a research study prior to the implementation of larger-scale studies). This is particularly important when a patient population is extensive. A subset of 10 cannot be used to make sweeping generalisations about a patient population in the millions. Similarly, results from a female only study may not always be applicable to men. Always look at the characteristics of the participants involved in a study to see if the conclusions reached reflect the wider patient population or a more discreet Bertha-specific subgroup.
11. Results or treatment that are only being promoted by a single researcher: this should raise a red flag, but, and it is a big but, it is not an instant disqualification. Some now commonly acknowledged theories and treatments were widely dismissed and criticised, with the researchers involved ostracised, prior to being found to be correct. For example, the work relating to H.pylori and stomach ulcers was initially met with hostility within the medical community, but is now commonly acknowledged as correct. However, until these theories have been replicated and thoroughly investigated through longitudinal and large sample studies, by multiple research groups, it is important to keep them in mind, but with a critical eye until the evidence is in. This is particularly the case when a treatment is either risky or expensive.
12. Keep up to date: theories come and go. Scientific research is constantly questioning it's own findings and is rarely static. What may have been set in stone 10 years ago may be completely revised by current research. Yet there are many websites relying on outdated information to support their theory. A good example of this is Fructose Malabsorption. Safe and unsafe foods change frequently as research continues and more foods are tested using newer, more accurate devices. Unless a website creator is continually on top of the issue, their information may be outdated, incorrect and, in some cases, dangerous.
13. Organisational bias: there are many health sites on the web. It is important to understand and be aware of the biases that are held by those sharing the information. Be it traditional medicine or alternative therapies, there are sites which only share information that supports their position, be that a study that shows their treatment/theory is correct or that another treatment/theory are is completely incorrect or dangerous. Know your information sources.
14. Not all research is equal: The scientific method is rigorous. There are ethics committees, reviews of methodology and conclusions, and research must follow an objective stringent set of rules. It is often staged eg the phases for drug trials, and must be able to be replicated by other researchers. This is why there is often a lag, sometimes measured in decades, between the findings from those 3 over-caffeinated rats in Dusseldorf, and new therapies. Articles in peer-reviewed journals are just that, peer-reviewed. If research is shoddy it is highly unlikely to get through to publication. (I'll add a caveat here. There is some less than stellar research in some peer-reviewed journals, but the peer-review system does increase the odds of finding reputable information.) Whether you are looking at information relating to a pharmaceutical product or a natural therapy, look for appropriate, rigorous research.
15. Just because everyone is saying it, doesn't make it true: the advent of social media means that information is shared rapidly, and often without critical review. People click share or retweet reflexively and soon what was seen by one person in an obscure, fringe, e-magazine from Lichtenstein, is doing the rounds of various forums. Repeated often enough, it is easy to begin to believe what you are reading is fact. Remember Point 9. Where possible, always seek out and read the original article, so you know what you are forwarding.
16. Don't Despair: All of these points can seem overwhelming, but the more you research, and the more you apply them, the more they will become commonplace. 

These points can be applied to any type of health information, be it news media, website, medical journal or research organisation. Out of all of them I would say apply Points 1 and 2 to any health information you come across, keep your biases in check and know that scepticism is healthy. Sometimes you will find your scepticism was well placed, and other times you will find that the researcher or media outlet's claims were indeed correct. But at least you have done your due diligence.

I have nothing but admiration for those doing the hard yards, in a largely unglamorous environment, with little thanks or recognition for their efforts. You don't go into medical research for the money or for the glitz. I know this from being both a research assistant and researcher. There is simply nothing glamorous about driving around town with an esky filled with saliva samples by your side, or the collecting of those samples. But you do it in the hope that your research will help others. There are unscrupulous researchers, but they are the minority, not the majority. However, it is important to be aware that even the best researchers can be foiled by the ways in which their information is disseminated through a highly competitive 24-hour news cycle, a society that demands their information in 140 characters or less, and the way others re-interpret their results to suit their particular position. 

It's your health, and your life. Understanding, and being able to critically review the vast amounts of medical information available, allows you to take control of your care.

Cheers

Michelle :)