Warning: DNA test results may not be as reliable as they appear

This article was originally on a blog post platform and may be missing photos, graphics or links. See About archive blog posts.

“Understanding of the genetic contribution to human disease is far from complete.”

This statement, by DNA decoder J. Craig Venter and three colleagues, is undeniably true. But it probably would come as a surprise to much of the general public.

A host of genetic testing companies have cropped up in recent years that offer to scan your DNA and calculate your risk of developing a host of diseases. It’s no wonder that customers are under the impression that their medical destiny can be read in their genes.

Venter – a key figure in the massive effort to sequence the human genome – and his colleagues have burst this bubble with a clever experiment, reported online today in the journal Nature.

They took DNA samples from five people and sent them to two prominent genetic testing companies in Silicon Valley, Navigenics Inc. of Foster City and 23andMe Inc. of Mountain View. Both companies sell testing kits online ($399 for 23andMe, $999 for Navigenics). Customers return their vials of saliva, and within a few weeks their test results are available over the Web.

If this were a perfect science, those five people would receive identical test results from both companies. And indeed, the predictions for breast cancer, celiac disease, multiple sclerosis and rheumatoid arthritis were in agreement in all five cases.

But for nine other diseases, at least one of the test subjects got conflicting results. In fact, with seven conditions – Crohn’s disease, heart attack, lupus, prostate cancer, restless leg syndrome and type 2 diabetes – at least half of the subjects got different answers from the two companies.

It wasn’t because the companies did a sloppy job of reading the DNA. Consumer testing companies typically scan 500,000 to 1 million genetic variants, and in this experiment more than 99.7% of those variants were read the same way by Navigenics and 23andMe.

Discrepancies arise in the way that different companies interpret those results. Researchers conduct genome-wide association studies to compare DNA samples from patients with particular diseases to DNA samples from healthy controls. Using powerful computers, they can pick out certain variants that are more likely to occur in people with a disease, as well as calculate the extra risk (or extra protection) that comes with each of the variants.

Both Navigenics and 23andMe rely on the same studies to assess their customers’ genetic risk. But they also use their own criteria for deciding how much weight those studies deserve.

They also emphasize different components of risk, Venter and his colleagues wrote. For instance, in calculating the chance that a particular customer will get a certain disease, 23andMe takes age into consideration, since the risk of many diseases goes up as you get older. On the other hand, Navigenics factors in the customer’s gender, since some conditions are more likely to affect men and others tend to strike women.

For some diseases, the genetics are clear. A single DNA marker has been shown to increase the risk of celiac disease by a factor of 7. Both companies recognize this, and so their results for this digestive disorder are in strong agreement. (Navigenics also includes seven additional markers that play a modest role, but their combined effect isn’t big enough to trump the effect of the one big marker.)

The situation is completely different for the skin condition psoriasis. One of the subjects was told by 23andMe that his or her chances of getting the disease were four times greater than for the general public. Navigenics agreed that the subject had an increased risk, but said it was only 25% higher.

The main reason for the discrepancy was 23andMe’s decision to rely on a DNA variant that nearly triples the risk of psoriasis, according to one study. But that study didn’t meet the scientific standards of Navigenics, so that particular variant isn’t part of the company’s risk model, Venter and colleagues said.

Even if all the results were in perfect agreement, it wouldn’t necessarily make the tests more reliable, the researchers said.

In the case of celiac disease, the DNA variants that have been identified so far are estimated to account for only 35% to 40% of the disorder’s genetic basis. (And that’s not even counting whatever environmental influences play a role.) A genetic scan today could give a customer a false sense of security if he or she happens to carry some disease variants that have not yet been discovered. There could also be customers whose risks are overstated because they have an unidentified DNA variant that greatly reduces their risk.

For these reasons and more, Venter and his colleagues can’t conclude which company’s test is more accurate. Instead, they emphasize that none of the consumer-oriented testing companies is as reliable as they appear.

-- Karen Kaplan