INTRODUCTION

Section:

ChooseTop of pageAbstractINTRODUCTION <<METHODSRESULTSDISCUSSIONAuthors' Disclosures of P...Author ContributionsREFERENCES

Germline aberrations in the BRCA genes are the most common cause of hereditary breast and ovarian cancer.¹ The major factor influencing penetrance is age; other factors are mutation type (eg, protein-truncating v radical-missense mutations), mutation location, modifying genes, hormonal factors, and birth-cohort effects.^2,3 Identifying deleterious mutations has monetary, social, psychological, legal, and insurance implications that challenge individuals deciding to undergo genetic testing. These implications also challenge clinicians to help patients understand their cancer risk and to offer preventive options such as prophylactic mastectomy, oophorectomy, chemoprevention, or screening.^4-7

High-quality pretest counseling, including carrier probability estimation, should be mandatory for any patient considering genetic testing for the BRCA gene. Individuals with low probabilities may be counseled not to pursue further testing unless they belong to a particularly high-risk ethnic group or have other overriding personal or psychosocial reasons to pursue testing.

Because multiple factors affect the penetrance of mutations in BRCA1 and BRCA2, various carrier-predicting mathematical models have been developed. BOADICEA (Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm)^8,9 and BRCAPRO^10,11 have been derived from and validated using data from separate populations. However, no independent, large-scale performance comparisons of these two models have yet been performed.

To address this lack, we used familial data provided by cancer genetic clinics from the Texas Cancer Genetics Consortium, a Cancer Genetics Network (CGN) regional center, to evaluate retrospectively the accuracy and reliability of the BOADICEA model in a US population by comparing its performance with that of more widely used clinical models (ie, BRCAPRO, Myriad I and II, and Couch [all of these models were run through CaGene4.0 software, The University of Texas Southwestern Medical Center, Dallas, TX; http://www8.utsouthwestern.edu/utsw/cda/dept47829/files/65844.html]) and evaluating the effects of extended family information on the accuracy of risk estimation. We also performed a model comparison analysis with the recently developed Manchester Scoring System,^12,13 which also had not previously been evaluated in a North American population.

METHODS

Section:

ChooseTop of pageAbstractINTRODUCTIONMETHODS <<RESULTSDISCUSSIONAuthors' Disclosures of P...Author ContributionsREFERENCES

We retrospectively analyzed the pedigrees of families recruited between 1996 and 2003 at high-risk cancer genetic clinics affiliated with the Texas Cancer Genetics Consortium, including The University of Texas M.D. Anderson Cancer Center at Houston, The University of Texas Southwestern Medical Center (UTSW) at Dallas, the Cancer Prevention and Risk Assessment Clinic of the Cancer Therapy and Research Center in San Antonio, and The University of Texas Health Science Center at San Antonio (UTSA; data from the latter two centers were combined as data from one center). The internal review board of each institution approved the study protocol and use of genetic data.

The pedigree data came from 472 families; 136 (29%) were from UTSW, 54 (11%) were from UTSA, and 282 (60%) were from M.D. Anderson. At UTSW, approximately 20% of probands were self-referred, and the remainder was clinician referred. At UTSA, 81% of probands were clinician referred, and the remainder was self referred (6%), referred by relatives or friends (11%), or referred by the American Cancer Society (2%). At M.D. Anderson, referral information was recorded for 174 families; 63 (36%) were self-referred, one (1%) was referred through a hospital information line, 105 (60%) were clinician referred, and five (3%) were referred by relatives or friends. Overall, 70% of probands were clinician referred. At M.D. Anderson, 26% of families were of Ashkenazi Jewish (AJ) ancestry compared with 12% at UTSW and 15% at UTSA. We had previously submitted data from 102 families at M.D. Anderson and 108 families at UTSW for a national CGN study that included a total of 1,948 families from all CGN sites; those 1,948 families were then used to estimate penetrance for the latest version of BRCAPRO, which we used in this study.

A test result was considered positive if the mutation was deleterious and resulted in a truncated protein. Variants of unspecified significance (VUS) were considered negative test results. However, because some may become characterized as deleterious later,¹⁴ we also analyzed a subset that excluded individuals with VUS (n = 53; Table 1). We also considered two A1708E BRCA1 missense mutations to be deleterious.¹⁵ We compared the areas under the receiver operating characteristic (AUROC) curves for each model using families in which one member had been tested for BRCA mutations by direct sequencing analysis or with the AJ genetic panel (BRCA1: 185delAG, 5382insC; BRCA2: 6174delT). Myriad Genetic Laboratories Inc, performed 95% of the mutation tests; the remainder was performed in-house. If more than one family member had been tested, carrier probabilities were estimated for the first member tested. We calculated AUROC curves and their corresponding asymptotic 95% CIs to assess the overall discriminative ability of the predictive test. AUROC curves were calculated using SPSS software version 12.0 (SPSS Inc, Chicago, IL). All analyses were stratified by AJ status. For missing ages of diagnosis, we used the median population age (70 and 60 years for breast and ovarian cancer, respectively) or age at death minus 1 year.

The BOADICEA model^8,9 allows for a polygenic modifier locus effect in which several low-penetrance genes have joint effects, thus accounting for the residual cancer clustering beyond the effects of BRCA1 or BRCA2. It also considers information on extended family further than second-degree relatives. For AJ families, we modified the software to accommodate the allele frequencies calculated by Hartge et al¹⁶ (ie, 0.00575 for BRCA1 and 0.00555 for BRCA2). We ran this model once with data from the complete extended family and then again with data from first- and second-degree relatives only. The BOADICEA model is available by direct request from the authors.^8,9

The BRCAPRO model^10,11 assumes autosomal dominant Mendelian inheritance of the genes, incorporates family history up to second-degree relatives only, and considers the effects of BRCA1 and BRCA2 without allowing for a residual polygenic effect. The Couch¹⁷ and Myriad I¹⁸ models use logistic regression in predicting BRCA1. The Myriad II model, which is based on two BRCA1/2 mutation–prevalence tables (one is for AJ) maintained by Myriad Genetic Laboratories Inc, was developed using methodology described by Frank et al.¹⁹ The Myriad II model does not distinguish between BRCA1 and BRCA2 but, instead, gives a composite probability. Myriad II is simple to use because it includes searchable tables of probabilities. The freeware package CaGene4.0 runs these models (including Myriad II) in a user-friendly environment. To obtain probabilities from CaGene4.0, it is necessary to input family data into the software.

The recently developed Manchester Scoring System^12,13 uses familial data on prostate, pancreatic, and male breast cancer, and scores are obtained from one table. This system is only applicable to cancer-affected probands and non-AJ families. The authors suggest a cutoff score of 10 points, corresponding to an approximate 10% probability threshold for each gene, and a combined score of 15 points, corresponding to a 10% threshold for both genes.

The American Society of Clinical Oncology had initially suggested that mutation testing be limited to individuals with a more than 10% probability of carrying a mutation.²⁰ In a later policy update,²¹ the American Society of Clinical Oncology reversed course, stating that setting numerical thresholds for recommending genetic testing was impractical. However, to allow comparison across models, we decided to use 10% as the cutoff point for the sensitivity, specificity, positive predictive value (PV), and negative PV of each model. We additionally used the Manchester Scoring System to analyze a subgroup of non-AJ pedigrees with cancer-affected probands and then compared its predictive ability to that of other models.

To explore the accuracy of BRCAPRO and the extended pedigree version of BOADICEA, we determined the proportion of positive BRCA1/2 test results and plotted them and their corresponding 95% CIs against the estimated average carrier probability as plotted at fixed midpoints of defined intervals. (The CIs were calculated by using the exact binomial-distribution method.)

RESULTS

Section:

ChooseTop of pageAbstractINTRODUCTIONMETHODSRESULTS <<DISCUSSIONAuthors' Disclosures of P...Author ContributionsREFERENCES

Fourteen (3%) of the 472 individuals tested were men (Table 1). Probands included 318 individuals (312 women and six men) with breast cancer (67%) and 45 women with ovarian cancer (10%). Of these families, 264 (56%) had three or more members with diagnosed breast and/or ovarian cancer, and 97 (21%) were of AJ descent. Fifty-seven probands (12%) had tested positive for BRCA1 mutation, and 37 (8%) had tested positive for BRCA2 mutation. One proband had a deleterious mutation of both BRCA1 and BRCA2.

The proportion of families that had three or more members with diagnosed breast and/or ovarian cancer was higher among probands who had a confirmed BRCA mutation. The mean proband age at BRCA mutation test (50 years) was similar in those with and without BRCA mutation. For probands with breast cancer, the median age at diagnosis was lowest in those with a BRCA2 mutation (40 years); for probands with ovarian cancer, the median age at diagnosis was lowest in those with a BRCA1 mutation (47 years). Bilateral breast cancer was more frequent in probands with a BRCA1 mutation (14%) than in probands with a BRCA2 mutation (3%) or no mutation (4%). Ovarian cancer was more frequent in probands with BRCA1/2 mutation than in probands with no mutation.

As shown by AUROC curves (Table 2), the BOADICEA model performed better than the other models in AJ families. In non-AJ families, the BRCAPRO model performed slightly better than the other models in predicting BRCA1 mutation but slightly more poorly than BOADICEA in predicting BRCA2 mutation. For the nonspecific prediction of BRCA1/2 mutations, the Myriad II model performed as well as BOADICEA in non-AJ families and slightly better than BRCAPRO in AJ families.

Forty-eight non-AJ and five AJ probands were excluded from analysis because they had VUS and no deleterious BRCA mutation. However, this changed the AUROC curves by less than 0.9% and did not change the rankings at all. In general, their AUROC curves increased slightly.

As shown in Table 3, the extended family version of BOADICEA had the highest sensitivity (0.87), positive PV (0.32), and negative PV (0.96) for predicting BRCA1 mutations in AJ families, whereas the restricted (second-degree relative) version had the highest specificity (0.70). For predicting BRCA2 mutations, both versions of BOADICEA and BRCAPRO were equally sensitive (0.75), whereas the restricted version had a slightly higher specificity (0.49), positive PV (0.12), and negative PV (0.96). For predicting both BRCA1 and BRCA2 mutations in AJ families, extended BOADICEA and Myriad II both had the highest sensitivity (0.87), specificity (0.35), positive PV (0.29), and negative PV (0.90). For predicting BRCA1 mutations in non-AJ families, BRCAPRO had the highest sensitivity (0.74) and negative PV (0.96), whereas restricted BOADICEA had the highest specificity (0.88) and positive PV (0.34). For predicting BRCA2 mutations in non-AJ families, extended BOADICEA had the highest sensitivity (0.55) and negative PV (0.96), restricted BOADICEA had the highest positive PV (0.23), and BRCAPRO had the highest specificity (0.90). For predicting BRCA1/2 mutations in non-AJ families, Myriad II had the highest sensitivity (0.81) and negative PV (0.94), whereas restricted BOADICEA had the highest specificity (0.77) and positive PV (0.41).

Table 4 lists the sensitivity, specificity, and PVs of the models in a subgroup of 300 non-AJ families with a cancer-affected proband. For predicting mutations in BRCA1, BRCA2, and BRCA1/2, the Manchester Scoring System had by far the highest sensitivity, the highest negative PV, the lowest specificity, and, except for the Couch model when predicting BRCA1 mutations, the lowest positive PV of all the models.

To assess the accuracy of extended BOADICEA and BRCAPRO, we graphically plotted the observed proportion of positive tests for BRCA1/2 and their corresponding 95% CIs against the estimated carrier probabilities (Fig 1). If predictive models were perfect inferential tools, then the observed proportion of positive tests would occur linearly along a diagonal. Although sampling variability explained much of the deviation in the predictions, we detected the following pattern in both models: the true proportion of positive tests was higher than expected at low estimated carrier probabilities (p < .1) and lower than expected at high probabilities (p > .5).

DISCUSSION

Section:

ChooseTop of pageAbstractINTRODUCTIONMETHODSRESULTSDISCUSSION <<Authors' Disclosures of P...Author ContributionsREFERENCES

Generally, the BOADICEA and BRCAPRO models performed equally well when applied to data from first- and second-degree relatives. The AUROC curve for BRCAPRO in predicting BRCA1/2 mutations in non-AJ families (0.804) was slightly higher than the curves reported by Euhus et al²² (0.712; range, 0.706 to 0.720) and Marroni et al²³ (0.757). Myriad II also performed comparably well in predicting nonspecific BRCA mutations in both groups. This finding is important because the simpler Myriad II model would permit faster dissemination in actual clinical settings.

The inclusion of extended family data, which is possible with the BOADICEA model, slightly improved the accuracy of risk prediction compared with risk prediction obtained when data were limited to first- and second-degree relatives. In a few cases, the extended family data helped identify cases that would have been missed without them; however, data from such families can probably be captured more effectively via screening (eg, asking about affected individuals beyond second-degree relatives). For example, for one BRCA-positive family (Fig 2), extended BOADICEA generated a carrier probability of 0.420, whereas restricted BOADICEA and BRCAPRO generated probabilities of 0.125 and 0.081, respectively. Discrepancies between the probabilities generated by BOADICEA and BRCAPRO in several cases are explained by the fact that BOADICEA does not account for male breast cancer, double primary cancers, or bilateral breast cancers and that BRCAPRO does not include data on multiple marriages or pedigrees beyond second-degree relatives.

As expected, BRCA mutation–positive pedigrees resulted in low carrier probabilities when the proband had few first- or second-degree female relatives. Probands with BRCA mutation–negative pedigrees and high carrier probabilities had no indications of any other obvious hereditary cancer syndromes such as Cowden disease or Peutz-Jeghers syndrome. However, we have not tested for p53 mutations in probands with early-onset breast cancer.

In evaluating non-AJ families, we analyzed only those families in which at least one member had been tested for BRCA1 or BRCA2 mutation by direct sequencing. The direct-sequencing approach has the advantage of identifying intraexonic and splice-site mutations.¹ However, because we included in AJ families individuals tested against the AJ Genetic Panel, we could have missed rare nonfounder mutations.^24,25

Berry et al¹¹ and Marroni et al²³ previously demonstrated that BRCAPRO may underestimate the true proportion of positive tests at low probabilities and overestimate it at high probabilities. We observed this pattern in AJ and non-AJ families and also when using BOADICEA. Exonic deletions in BRCA1 are rare (an incidence of perhaps 5%) in the populations we were studying and would not have been detected by direct sequencing. However, the mildly decreased sensitivity of direct sequencing seems unlikely to have caused the observed downward biases.

We also analyzed the estimated carrier probabilities in 17 non-AJ families in which at least one male had breast cancer. Two probands (12%) in those 17 families tested positive for BRCA1 mutation, and three (18%) tested positive for BRCA2 mutation. BRCAPRO estimated mean carrier probabilities of 0.188 for BRCA1 and 0.299 for BRCA2, whereas restricted BOADICEA, which does not model male breast cancer, estimated mean probabilities of 0.039 for BRCA1 and 0.069 for BRCA2.

Although studies have shown that families with BRCA mutations are at increased risk of pancreatic cancer,^26,27 only the Manchester Scoring System considers this cancer by increasing the probability of a BRCA2 mutation. Twenty-six of the non-AJ families had at least one first- or second-degree relative with pancreatic cancer. Seven of these families (27%) tested positive for BRCA1 mutation, and three (12%) tested positive for BRCA2 mutation; thus, 39% of non-AJ families affected by pancreatic cancer tested positive for BRCA mutations. The mean probabilities estimated by BRCAPRO and restricted BOADICEA in these families were 0.184 and 0.062, respectively, for BRCA1 mutation and 0.060 and 0.068, respectively, for BRCA2 mutation.

The penetrances used in BRCAPRO are constantly being updated, and for the present analysis, we used the most updated version available (CaGene4.0, released in September 2004). The BOADICEA model is currently being revised to incorporate other BRCA-predictive factors.⁸ None of the models consider a family history of risk-reducing surgical procedures (eg, prophylactic mastectomy or oophorectomy), which are important for more accurate risk assessment and, thus, should be thoroughly ascertained by the clinician. Although prediction models can assist in cancer risk decision making, sound clinical judgment remains essential and cannot be superseded. In fact, the AUROC values we obtained in this analysis (range, 0.67 to 0.80) suggest that these models are only moderately effective in estimating risk.

The relatively low sensitivity at the 10% cutoff point for all of the models, except for the Manchester Scoring System, suggests that the cutoff may be set too high to adequately guide clinicians. In non-AJ families, decreasing the cutoff point to 5% increased the number of women referred for testing by 50% for BOADICEA, 25% for BRCAPRO, and 86% for Myriad II. BRCA1/2 sensitivity increased to 0.814 for BOADICEA and BRCAPRO and 0.971 for Myriad II. However, lowering the cutoff point greatly increases the likelihood of testing with its accompanying costs. The median sensitivity for BRCAPRO at the 10% cutoff had been previously reported as 0.92 by Euhus et al²² and as 0.80 by Marroni et al.²³ Although the Manchester Scoring System was most sensitive in predicting mutations in both BRCA genes, its utility may be limited because it results in over-referral for genetic testing and, at present, can only be used in non-AJ families and cancer-affected probands. Compared with the next most sensitive model (Myriad II), the Manchester Scoring System resulted in 41% more individuals being referred for genetic testing.

Together, our findings demonstrate that BOADICEA, BRCAPRO, and Myriad II are all similar to each other and superior to other models in predicting BRCA mutations but only moderately effective in estimating risk. Information about relatives beyond the second degree can improve the performance of BOADICEA, but this information can be obtained just as easily by asking about additional affected third-degree or more removed relatives. Using 10% as a cutoff for BRCA testing referral yields low sensitivity in all the models we tested except the Manchester Scoring System. Reducing this threshold increases sensitivity but only at the expense of economy. As newer versions of the prediction models we tested become available, further reassessment of their prediction ability is warranted.

Authors' Disclosures of Potential Conflicts of Interest

Section:

ChooseTop of pageAbstractINTRODUCTIONMETHODSRESULTSDISCUSSIONAuthors' Disclosures of P... <<Author ContributionsREFERENCES

The authors indicated no potential conflicts of interest.