This story was originally written in Portuguese, and published to the website of Revista Questão de Ciência. It appears here with permission.
In recent years, more and more research has pointed to the importance of the microbiome – the set of microorganisms that inhabit our bodies, such as the bacteria and fungi in our intestines that aid in digestion, or the bacteria present on our skin or saliva – in various aspects of human health, both physical and mental. With this in mind, many companies have begun offering “microbiome tests” directly to the public. Companies sell the tests by claiming that they can detect whether a customer’s microbiome is “healthy” or “dysbiosis” (out of balance) and suggest that, if the latter is the case, it may be the cause of various health problems.
However, like the genetic tests for ancestry and alleged hereditary health risks that are also sold directly to the public, these microbiome tests have no analytical validity or clinical relevance. In other words, they do not provide information that can be used correctly and responsibly in making health decisions. Therefore, there is a need for greater regulation of the sector by health authorities, as called for by researchers in the field in an article published in an issue of the prestigious journal Science earlier this year.
“Companies’ claims of being able to detect ‘abnormal’ microbiomes are not supported by research; their testing procedures lack analytical validity, and their results have no demonstrated clinical validity,” they write. “As a result, consumers may be financially exploited or harmed by the inappropriate use of test results that neither they nor their physicians understand.”
Independent assessment
The authors of the paper did not draw these conclusions out of thin air. As part of a project funded by the US National Institutes of Health (NIH), they first conducted an online survey of companies around the world that are offering these tests, and reviewed the information they provide to the public about their services. The researchers also conducted interviews and focus groups – a qualitative research method – with microbiome researchers, physicians who treat chronic bowel and vaginal conditions, patients, and consumers who have used or have access to these types of tests. Finally, they assembled a working group of microbiome researchers, physicians, medical device and service regulatory experts, and industry and consumer representatives to discuss the regulatory challenges posed by these tests.
In all, the researchers identified 31 companies offering microbiome testing directly to consumers, 17 of which were based in the US. They found that the industry primarily attracts healthy people who are curious to learn more about their microbiome, as well as individuals suffering from chronic conditions that may be linked to it, such as Crohn’s disease, irritable bowel syndrome and bacterial vaginosis.
While most companies focus on the gut microbiome, some also offer testing of the vaginal and skin microbiomes. Again, their processes are similar to those used by genetic testing companies; consumers order a test kit, complete a health and lifestyle questionnaire, collect stool, vaginal or skin samples, and send them to a company lab for analysis.
This analysis generally describes the taxonomic composition of the microorganisms present in the samples and their relative abundance, and in some cases includes the metabolic functions present or observed of these microbes. It also includes an assessment, often using a graph, of whether the microbiome is in a “healthy” range or not, a conclusion supposedly based on comparisons with healthy people or those with chronic diseases or conditions.
And therein lies the first problem, the authors of the article point out: it is not known exactly which databases companies use to make the comparisons, or what their composition is. In some cases it may be proprietary to the company, composed of the results of all its customers, and not representative of the population at large. The company may also have purchased data from patients diagnosed with certain diseases or chronic conditions, again not based on a representative sample of the population.
Additionally, if customers’ microbiomes fall outside the “healthy” range, companies supplement the report with suggestions for dietary changes or supplements to improve the “balance” of their gut microbiome. It is therefore no surprise that the authors of the paper found nearly half of the companies that sell these at-home microbiome tests also sell the supplements those reports recommend.
Finally, companies also seek to keep these customers coming back by advising them to repeat the tests periodically to assess the effects of dietary changes and/or supplements. Some companies even offer subscriptions and nutritionist services, the authors of the paper note.
No scientific validity
The researchers then highlight that the three fundamental characteristics of the suitability and benefit of a test, in the context of human health, are: the analytical validity of the test itself, its clinical validity and its usefulness. These are characteristics these “home” microbiome tests do not have, they say.
Analytical validity involves, for example, establishing false positive and false negative rates, which, in the case of microbiome tests, would involve knowing whether they overestimate or underestimate the relative abundance of microorganisms in the samples. However, they say in the case of the microbiome this is impossible to determine, since the bioinformatics tools used by companies are not capable of recognising and identifying all microorganisms in the samples, thus not allowing for knowledge of relative abundances.
Clinical validity is based on the principle that the test is capable of determining the existence of a disease, which in the case of microbiome tests, means knowing whether they can actually define a state of health or disease. This, in turn, leads to their clinical utility. That is, whether this determination can lead to a useful clinical action – an intervention that could be applied to “treat” the individual. The authors emphasise that the clinical utility of microbiome tests is questionable, since there is little or no evidence supporting the interventions generally recommended based on the results of these tests such as diets, supplements, exercise or other lifestyle changes.
“Without analytical validity, the first of these fundamental characteristics of test suitability and benefit, test results are meaningless,” the researchers state. “Furthermore, determining whether a sample is characteristic of a ‘healthy’ state or not requires comparison to a standard. And, at present, there is no consensus on what constitutes a healthy human microbiome in any population or subpopulation.”
Worse, many of these tests have proven to be unreliable in their results, with variations in the analyses of the same sample performed by different companies, and even by the same company. The authors cite as an example preliminary results from an ongoing study by researchers at the US National Institute of Standards and Technology (NIST) in which three identical samples of standardised faecal material were sent to seven different companies. Despite the identical composition of all the samples, the results were discrepant both between the different companies and in the analyses performed by the same company.
Need for regulation
Given all this, the authors of the Science article advocate for stricter regulation of the marketing and performance of these microbiome tests. They note that although many of the companies claim that their tests are not for “diagnostic” purposes, their marketing strategies often imply this, leading people to believe that they are based on scientific and clinically relevant evidence. This also leads many people to believe that these tests are overseen by health regulatory agencies, when in fact they are in the limbo of unregulated health and wellness products, like dietary supplements.
“The lack of regulation of these tests may put consumers at risk of harm when they rely on inaccurate results and follow unproven nutritional or dietary supplement recommendations,” they say. “These harms may include erroneous self-diagnoses, delays in seeking medical treatment, and replacing prescription drugs with non-drug supplements.”
Another concern is for people who already suffer from diagnosed diseases or health conditions, another large audience for these tests, and who may see them and their results as a treatment alternative:
“Many individuals seeking these tests have chronic illnesses and are desperate to try anything to alleviate their pain and suffering. Because these tests are largely unregulated, they are not subject to reporting of adverse effects, but we have had cases reported by gastroenterologists of harm. In one of the focus groups we hosted, a paediatric gastroenterologist reported patients who developed dietary restriction disorders after following the microbiome testing companies’ recommendations to avoid certain foods.”
