Tag Archives: prenatal testing

Guest Post: NIPS Is Not Diagnostic – Convincing Our Patients And Convincing Ourselves

By Katie Stoll, MS

Katie Stoll is a genetic counselor in Washington State. She graduated from the Brandeis University training program in 2003 and since that time has held positions in the areas of prenatal, pediatric and cancer genetics.

A couple of years ago we were just beginning to learn about a new prenatal testing technology termed Noninvasive Prenatal Diagnosis. It was soon relabeled as Noninvasive Prenatal Testing, and now the American College of Medical Genetics and Genomics recommends this be taken one step further by terming it Noninvasive Prenatal Screening (NIPS) to highlight the limitations of this new technology.

As currently reported by labs, NIPS presents new challenges for genetic counselors. Of particular importance is figuring out how to convey to patients and healthcare providers why relying on sensitivity and specificity alone may lead to misinterpreted results. In the absence of positive and negative predictive values there may be a tendency to assume that the high sensitivity and specificity reported with NIPS means that these tests are more powerful – more diagnostic – than they actually are.  

It is imperative that we understand both what the terms mean and how they relate to a person’s likelihood of having a condition.   Sensitivity measures the true positive rate – the proportion of actual positives which are correctly identified as such (e.g., the percentage of fetuses with Down syndrome (DS) who have a positive test result). Specificity measures the true negative rate – the proportion of actual negatives which are correctly identified as such (e.g., the percentage of fetuses who do not have Down syndrome who have a negative NIPS result for DS).

A test can have both a high sensitivity and specificity without being a good predictor of whether the condition is actually present. The likelihood that a positive test is a true positive result also depends on the incidence of the condition.

Sensitivity Graph

Genetic counselors are used to thinking about aneuploidy screening in terms of PPV, as this is generally the format for reporting maternal analyte screening such as Integrated , Quad screens, etc. Analyte screening takes into account the prior probability based on maternal age and provides a PPV as the end result. For instance, an analyte screen result may be reported as Positive with a 1 in 50 chance of Down syndrome. The PPV with analyte screening lets us know how many patients with a “positive” test will actually have a pregnancy affected with the condition and reporting results this way makes it clear that this is a screening test.

Can we apply the same interpretation to NIPS results?  Some labs provide a “risk score” which appears similar to what we see with analyte screening, but I am told by the labs that the vast majority will be reported as either >99% chance or <.01% chance.  Some labs do not report a risk score, instead giving essentially a positive or negative result. But does this mean that greater than 99% of women who receive a >99% or a positive result are actually carrying a fetus with Down syndrome or other chromosome condition?

Given that women 35 year and older are a population targeted for NIPS let me work out the expected NIPS results given the approximate sensitivities and specificities reported for a hypothetical population of 100,000 thirty-five-year old women (while I cannot tell you the specific number of women age 35 who give birth per year, CDC data suggests that for the past several years about 400,000 – 500,000 women in the age 35-39 have given birth each year in the United States – so 100,00 births annually by 35-year-old mothers is probably in the ball park of the national trend.

The performance data vary significantly from lab to lab – for the purpose of this illustration, I am using sensitivity and specificity in the range of what has been reported.  The data below are based on the chance of Trisomy 21, 18 and 13 at the time of amniocentesis for a woman 35 at time of EDD1.

Down Syndrome

Trisomy 18

Trisomy 13

Incidence

1/250

1 / 2000

1 / 5000

Affected Fetuses

400

50

20

Sensitivity

99.5%

98.0%

90.0%

Specificity

99.9%

99.6%

99.8%

Total test positives

498

449

218

True test positives

398

49

18

False positives

100

400

200

Positive Predictive Value

80%

11%

8%

If we add all of the positive results together in a population of 100,000 thirty-five-year old women we see that 1165 (1.2%) have positive test results for Trisomy 21, Trisomy 18 or Trisomy 13.  Note, though, that only 465 of these results will be true positives. This indicates that the majority of the time (greater than 60% using these statistics), a positive result on NIPS for a 35-year-old woman will be a false positive - and this doesn’t even include calculations for sex chromosome aneuploidy which some NIPS labs also screen for.

Notably, the negative predictive value for NIPS is very high indicating that a negative test result is a true negative >99% of the time. But how do we reconcile that for many women, the chance of a false positive with NIPS may be higher than the chance of a true positive result when that seems to be contradicted by way the labs are reporting the results? 

In trying to explain the chance of a false positive result to patients with a “positive” test report in hand, I have found that I am met with disbelief. I can understand why – if a test says there is a>99% chance of Down syndrome and the lab says the test has >99% sensitivity and >99% specificity, how could this test be wrong?

While genetic counselors understand the limitations, the reporting practices of the labs place us in a position in which we have to work hard to convince our patients that NIPS is only a screening test.

Currently four labs offer NIPS in the U.S. and all have different strengths and weaknesses in their reporting practices. All could be improved by making the limitations of this technology more obvious.  In some cases the language used in the reports gives the appearance that NIPS is diagnostic. For example, one company’s report suggests that the healthcare provider should advise for “additional diagnostic testing”.  The labs vary in whether the need for genetic counseling following a positive result is recommended.  Additionally there is variability in the transparency of how the performance data are derived.

Given that the performance statistics vary significantly, we need to be sure to take these details into account when considering PPV. I  encourage genetic counselors and other healthcare providers to critically look at how the performance data are derived.  The sample sizes on which these numbers are based are often quite small and the confidence intervals can be broad.  I was surprised to see in the fine print of one report that the performance data “excludes cases with evidence of fetal and/or placental mosaicism.” Given that mosaicism is a known cause of false positive results and because mosaicism cannot be definitively determined through NIPS, it doesn’t seem accurate that these cases should be excluded from calculations of test performance.

The pitfalls of interpreting NIPS results is a challenge we need to address because NIPS is increasingly taking place without the involvement of genetic counselors in pretest or post-test counseling. There is real concern that patients are making pregnancy decisions based on screening tests with the misunderstanding that NIPS is diagnostic. 

I write this as call to the NIPS labs to change their reporting practices to better emphasize the screening nature of this technology. Providing some positive predictive value estimates would be tremendously helpful as we try to make sense of NIPS results for our patients. While it may be difficult to provide individualized risk assessment, a general table of how prior probability impacts individual test performance would be beneficial for interpretation. Furthermore, eliminating language from the reports that suggests these tests are diagnostic and giving more transparency to ways in which performance data are calculated would also be welcome changes.

As genetic counselors, we strive to ensure informed decision-making for the clients we see. Key to informed decision-making is an understanding of the limitations of this evolving technology. As fellow patient advocates, I hope the genetic counseling community will join me in requesting increased accountability and responsible reporting on the part of the labs regarding NIPS.

I would like to acknowledge Evan Stoll, retired GAO data analyst for his contributions to this piece.

Please Note: Authors who contribute to The DNA Exchange cannot offer medical advice. Many commenters have raised interesting and thoughtful questions about NIPS. If you have undergone NIPS and have questions, you should meet with a certified genetic counselor. To locate a genetic counselor, go to the  Find A Genetic Counselor section of  the website of  The National Society of Genetic Counselors.

  1. Hook EB. Prevalence, risks and recurrence. In: Brock DJH, Rodeck CH, Ferguson-Smith MA, editors. Prenatal Diagnosis and Screening. Edinburgh: Churchill Livingston, 1992.

 

71 Comments

Filed under Guest Blogger

VARIATIONS IN A MINOR KEY: SOME THOUGHTS ON PRENATAL TESTING IN AN ERA OF WHOLE GENOME SEQUENCING

James Watson is many things – geneticist, Nobel laureate, agent provocateur – but in the realm of psychiatry he is first and foremost the parent of a son with schizophrenia.  So when he spoke in 2007 at the World Congress of Psychiatric Genetics, it was as a family member, albeit a family member with an unusually good grasp of the science.  And it was as a family member that he exhorted the scientists in the audience to keep up the good work, so that “someday we could identify those individuals destined to suffer from mental illness in utero, and weed them out.”

How often do you hear an audible gasp in the midst of a plenary talk?  The dismay and the indignation were palpable.  Researchers throughout the day interrupted their talks on GWAS to express in the strongest possible language that the goal of their work was to understand the pathophysiology of the disease and perhaps to aid in diagnosis – not to provide pre-symptomatic risk  assessment and not – no, never – not to be used prenatally.

“But if this is what families want,” I asked one speaker later that day.  “How do you propose to restrict testing, once the means to test is available?”

“They can’t,” he replied.  “They must not.”

Ah.  Of course.  They must not – I will pass that along.

Five years later, it is not GWAS but whole exome sequencing and whole genome sequencing providing all the buzz at conferences.  Solving the diagnostic odyssey!  Revolutionizing cancer treatment!  Ushering in an era of personalized medicine!  It’s very exciting.  Prenatal testing is rarely mentioned, and then only in passing – while prognosticators sing happy songs of a not-so-far-off day when every baby will be sequenced at birth.

Sequenced at birth?  Will it even be necessary?  Maybe Mom and Dad have baby’s DNA already, on a hard drive or a memory stick or downloaded onto their cell phones along with the ultrasound pics.

This is not the genome sequencing story you are seeing in the papers or the blogs.  It’s not what researchers are excited about.  The ones we hear are all about science journalists getting their DNA decoded and setting off on odysseys of self-discovery that involve hours of consultation with clinical and academic superstars who donate their time. We hear about kids with strange constellations of symptoms finding answers after years of disappointment.  Those are heartwarming tales: anecdotal and difficult to imagine at scale, but hopeful and exciting nonetheless.  But there is another theme playing, in a minor key, and I hear it faintly, hidden beneath the violins and the trumpets.

I hear it, an unspoken question, when we debate the utility of genomic information.  What does to mean to say that information is actionable? (Prevention? Treatment? Cure?  Prenatally, there is only Yes or No.)  Can patients handle uncertainty?  (And what will we lose, when pregnancies are terminated just to be on the safe side?)  Doesn’t everyone have the right to know what is in their own DNA? (The information is available – why not use it?  What could possibly go wrong?)

Whatever tests are available postnatally will find their way into prenatal use.  The gateway technologies – PGD, cell-free fetal DNA testing – are in place. And there is no use saying, “they can’t, they won’t, they shouldn’t” because they can and they will – and sometimes they should.  There will be good uses too: success stories and disasters averted.  A blanket “no” is not an option, and granting anyone authority to pick and choose which uses are worthwhile vests altogether too much power in the hands of any one person, or profession, or bureaucratic entity.

The same tests can be done before or after birth, but the experience is entirely different.  Uncertainty after birth is an opportunity.  The least useful information is that which will absolutely come true, no matter what you do.  Uncertainty before birth is a crisis.  Anyone who has ever discussed a variant of uncertain significance with a pregnant mother can tell you that.  But what are the chances there will be developmental delay?  Are you certain that the heart will be affected? How sure are you that this means anything?  Not nearly sure enough.  Please understand that.

In general, notions of genetic determinism increase the likelihood that genomic testing will have negative consequences.  Fatalistic attitudes about the power of genes could lead people to overestimate the meaning of elevated risks and underestimate the meaning of reduced risks.  Anxiety, stress, missed mammograms – you have heard this before.  Shrug.  People are grown ups.  They will figure this out.  Information is power.

But we are in a whole new universe trying to reconcile underpowered and often misunderstood predictive testing in the context of prenatal use.  So please, in telling tales of all the wonderful things that genome sequencing will do, save space for a mention of what it cannot do.  Make sure they understand that there are great wide cracks in our crystal ball.  Do not oversell the value of genotype in the absence of phenotype.  Remember that in the end neither researchers nor physicians nor genetic counselors will dictate how this new technology will be used.  Others will make that call, and we will be in the choir, singing songs of praise laced with sorrow.

Follow me on Twitter:

https://twitter.com/laurahercher

 

4 Comments

Filed under Laura Hercher

Resistance Is Futile: A New Paradigm for Genetic Counseling?

For with this detection there arises new lines of approach in the field of preventive medicine, and the sociological consequences may be far-reaching.

- James V. Neel, from his 1948 plenary lecture, “The Detection of The Genetic Carriers of Hereditary Disease”, delivered at the first annual conference of the Human Genetics Society of America (which eventually changed its name to the American Society of Human Genetics)

The technical advances in genetic testing over the last 5 years have been stunning. Much of what I thought of as Not Going To Happen For A Long Time  has now happened yesterday. Along with these breakthroughs is the unstated but increasingly common suggestion that everyone should taste the fruit of testing in Gregor’s Genetic Garden of Eden.

In the old days (like a year or two ago) only a small portion of the patient population were thought to be candidates for genetic testing, those for whom it made medical sense and who were emotionally ready for the ramifications of the knowledge. Genetic counselors used their skills to help patients select the appropriate test and to guide them through the clinically, emotionally, and financially complicated decision-making process. Some chose to undergo testing while others delayed or declined it. We did not really care what patients chose to do; our role was to go through the wringer with them.

Now, though, this model of genetic testing only for the select few may be replaced in the near future by the idea that everyone – healthy, sick, high risk, low risk – should have genetic testing. Population scale genetic testing, with its promises of personally tailored medical care and better health outcomes, assumes that everyone – except for a handful of Luddites, people who do not own mobile phones or have Twitter accounts, Flower Children, and conspiracy theorists – will incorporate DNA into their routine medical care. Genetic testing becomes a foregone conclusion, not an ethically and emotionally weighty matter to be carefully explored and considered. If everyone has a genetic test and everyone carries gene mutations, doesn’t that make everyone a patient?

Think I am overstating my case? Perhaps. Then again, recall the many professional and popular articles you have read that are variations on this theme: The time is near when you will walk into your doctor’s office with an inexpensive DNA Chip that contains your entire genome and that will guide your doctor in choosing the best medications for you and select the most effective screening tests. You will live to be 100, enjoy a lusty sex life, and have healthy children. While the $1000 genome may not be a shining example of truth in advertising, affordable genetic testing is upon us.

A second case in point is the introduction of cheap carrier testing for a huge number of mostly obscure genetic conditions, what has come to be called Universal Carrier Screening. I will risk stating the obvious and point out that the word “universal” implies that the test is for everyone. At $99, it is hard to say no.

A third case in point is newborn screening, which is as close as it gets to universal genetic testing. The conditions screened for with those heel sticks continues to increase but the primary justification is not “treatment before symptoms develop.” Rather, testing is predicated on reducing the number of families caught in The Diagnostic Odyssey, that emotionally and financially draining parental journey to find out what medical disorder their child may have. Based on this premise, there is no logical stopping point for including disorders in a newborn panel. Every genetic disease is a potential source of a diagnostic odyssey. In fact, the rarer the syndrome, the better it is for inclusion in newborn screening since uncommon conditions are less likely to be diagnosed by most practitioners.

Another area of pervasive genetic testing is the recommendation for universal fetal aneuploidy screening during pregnancy, made even more tempting by high detection/low false positive non-invasive tests.

Genetic screening is offered to everyone prior to conception, during pregnancy, and at birth. Testing all adults allows the rest of the camel into the tent.

Genetic counselors are not the driving force behind universal genetic testing, although undoubtedly we have some complicated role. As I have discussed elsewhere, we probably have less influence on patients’ decisions than we  think. Larger social, economic, and ethical forces are at play, in much the same way that the introduction of amniocentesis, newborn and carrier screening, and the birth of the genetic counseling profession were all products of their times.

The role of genetic counseling when it comes to genetic testing, then, may no longer be primarily to help patients make decisions. Instead, genetic counselors may become Phenotype Counselors who interpret and integrate results of genetic tests that were run – and possibly chosen through online services – before patients walked into our offices.

Ilana Löwy’s book “Preventive Strikes: Women, Precancer, and Prophylactic Surgery”

If I am right, genetic counselors are likely to encounter controversies and dilemmas. Ethical values like nondirectiveness and autonomy become less forceful if individually tailored health strategies can help prevent or attenuate serious illness. Think of how many  oncologists consider their high risk cancer patients crazy for not having BRCA testing or believe that known BRCA mutation carriers are making poor choices for not undergoing risk-reducing surgeries.

Eugenic concerns, the voice and dignity of the disability community, the psychological sequelae of coping with test results, and worries about the other downsides of genetic testing may be pushed to the wayside by the power of the still unproven assumption that medical spending will become more cost-effective, clinical decisions will be wiser, and everyone will be healthier if their genomes are analyzed. In fact, people with disabilities themselves will likely see some treatment and diagnostic benefits from genomic testing. And because laboratories and lab-based counselors will likely play critical roles, defining and protecting against conflict of interest becomes even more critical and complex.

Both good and bad will come out of universal DNA testing, though it is difficult to predict what measure of each. But so much genetic information available on so many people must give one pause. The history of genetics demonstrates that every advance in genetics is fraught with social complexity and dangers. We may have a more sophisticated knowledge of genetics than our predecessors, but we are neither wiser nor more ethical.

I  close by reminding you that knowing our past helps us better understand why we are here and what may happen if we go there. To that end, let me bring to your attention two recently published books about the history of genetic counseling and the history of medical genetics: Telling Genes: The Story of Genetic Counseling in America by Alex Stern (The Johns Hopkins Univ. Press, 2012) and The Science of Human Perfection: How Genes Became the Heart of American Medicine by Nathaniel Comfort (Yale Univ. Press, 2012). The authors, my good friends and colleagues, provide an informed and critical historical understanding of  genetic counseling and genetic medicine. Everyone should read these books. It will do your souls – and your counseling philosophy – good.

4 Comments

Filed under Robert Resta