Understanding delays in cancer diagnosis requires detailed information about timely recognition and follow-up of signs and symptoms. This information has been difficult to ascertain from paper-based records. We used an integrated electronic health record (EHR) to identify characteristics and predictors of missed opportunities for earlier diagnosis of lung cancer.

Using a retrospective cohort design, we evaluated 587 patients of primary lung cancer at two tertiary care facilities. Two physicians independently reviewed each case, and disagreements were resolved by consensus. Type I missed opportunities were defined as failure to recognize predefined clinical clues (ie, no documented follow-up) within 7 days. Type II missed opportunities were defined as failure to complete a requested follow-up action within 30 days.

Reviewers identified missed opportunities in 222 (37.8%) of 587 patients. Median time to diagnosis in cases with and without missed opportunities was 132 days and 19 days, respectively (P < .001). Abnormal chest x-ray was the clue most frequently associated with type I missed opportunities (62%). Follow-up on abnormal chest x-ray (odds ratio [OR], 2.07; 95% CI, 1.04 to 4.13) and completion of first needle biopsy (OR, 3.02; 95% CI, 1.76 to 5.18) were associated with type II missed opportunities. Patient adherence contributed to 44% of patients with missed opportunities.

Preventable delays in lung cancer diagnosis arose mostly from failure to recognize documented abnormal imaging results and failure to complete key diagnostic procedures in a timely manner. Potential solutions include EHR-based strategies to improve recognition of abnormal imaging and track patients with suspected cancers.

Research Sponsor:

Missed and delayed cancer diagnoses are associated with substantial disability and costs110 and are a frequent cause for ambulatory malpractice claims.11 Lung cancer is both common and lethal and has a particularly poor prognosis if not diagnosed early.12,13 Although efforts to promote earlier diagnosis and treatment of lung cancer have not yet demonstrated improved survival outcomes, research is underway to evaluate the benefits of screening in high-risk patients.14

Early diagnosis hinges on timely recognition and action on clinical clues.1518 Although patient care-seeking delays are well documented,1923 treatment delays may also be related to the diagnostic process following the patient's first presentation with signs and symptoms.9,12,16,18,21,22,2427 Prolonged waiting times after the initial presentation are less well understood, but some contributing factors have been documented.16,18,2830 For instance, busy frontline providers might miss early signs and symptoms of lung cancer. Scheduling delays for diagnostic tests, poor communication of abnormal results, or test misinterpretation may also impede the diagnostic work-up. Finally, patients may not adhere to scheduled appointments or procedures after the initial work-up, or they may seek care in a different health system where their test results are not available. Therefore, missed opportunities for early diagnosis of lung cancer can occur due to failure to recognize potential diagnostic clues or failure to complete the diagnostic work-up in a timely manner.

Previous literature offers limited information on the nature of missed opportunities for earlier lung cancer diagnosis. Many studies have relied on information extracted from paper-based medical records, which may be difficult to evaluate for evidence of breakdowns in communication and care coordination.10,12,18,22,24,26,3133 Integrated electronic health records (EHRs), on the other hand, can provide ready access to progress notes, documentation of abnormal findings, and exchanges of information (eg, test results, referrals, and so on) among front-line primary care providers, consultants, and other diagnostic specialists.34 We hypothesized that using an EHR would provide new insights into the origin and prevention of diagnostic delays in lung cancer. Our objective was to evaluate characteristics and predictors of missed opportunities for earlier diagnosis of lung cancer in a health care system with an advanced integrated EHR.


We used a retrospective cohort design to identify and evaluate all pathologically confirmed, newly diagnosed cases of primary lung cancer at two geographically dispersed Veterans Affairs (VA) medical centers. Our sample consisted of patients diagnosed between July 2005 and June 2007 at Site A and between July 2004 and June 2007 at Site B (a longer time period at the latter site allowed for more patients). Both sites are tertiary care referral centers with on-site multispecialty ambulatory care clinics and community-based satellite clinics that provide care to urban and rural populations. All patients are assigned a primary care provider, and most patients obtain their longitudinal care within these systems from academic and nonacademic providers and resident trainees. The study was approved by the local institutional review boards.

Data Collection Procedures

We performed a detailed review of progress notes, consultations, laboratory and radiology reports, discharge summaries, and additional relevant data in the EHR to evaluate the diagnostic processes for missed opportunities. Two trained physician raters independently reviewed each case using a standardized data collection instrument adapted from our previous work in colorectal cancer diagnosis (Table 1).35 Reviewers evaluated all relevant EHR data (in most patients as far back as 5 years) for the presence of predefined clinical clues that warrant a diagnostic work-up for lung cancer. Clues were derived from current literature19,3639 followed by team consensus (Table 1). To ensure reliable and consistent data collection, the study team supervised and trained the reviewers during pilot testing, and all discordant judgments of missed opportunities were discussed to obtain consensus. Data on patient outcomes (harm, stage of diagnosis) were not collected to reduce hindsight bias.40


Table 1. Summary of Data Collection Instrument

Table 1. Summary of Data Collection Instrument

Data Category Description of Item(s) Example
Patient characteristics Age, race/ethnicity, sex, medical and psychological comorbidities, smoking 77-year-old white male with coronary artery disease
Type of clue (symptom or sign that should prompt further work-up) Presence of at least one of the following: Abnormal chest x-ray showing nodule
    Blood in sputum/hemoptysis
    Hoarseness that lasts > 2 weeks
    Recurrent bronchitis or pneumonia
    Abnormal (ie, suggestive of possible neoplastic disorder) chest x-ray
    Abnormal chest CT
    Abnormal abdomen CT
    Serial abnormal imaging
    Abnormal sputum examination/sputum cytology
    Unexplained effusion
    New onset Cushing's symptoms/syndrome
    New onset of hypercalcemia symptoms/syndrome
    New onset of syndrome of inappropriate antidiuretic hormone
    Superior vena cava obstruction
    Worsening persistent cough/bronchitis or new description of chronic cough lasting > 8 weeks
    Provider acknowledged unexplained weight loss or other unexplained weight loss > 10 lbs in addition to respiratory symptoms
    Chest pain or rib pain
    New onset/worsening pain in non-chest location
Date clue first appeared on medical record review June 5, 2005
Date next step was requested (ordered) September 9, 2005
Date next step was completed September 15, 2005
Presence of type 1 or type 2 missed opportunity? Type 1: No evaluation (or work-up) for lung cancer was initiated within 7 days of appearance of a predefined clinical clue Yes: Type 1
Type 2: Failure to complete within 30 days a diagnostic procedure or consultation or the follow-up action requested in response to a predefined clue No follow-up CT scan ordered to evaluate nodule by June 13, 2005
Contributory factors Provider, system, and/or patient Provider
Type of personnel involved Codes for personnel adapted from Gandhi et al11 Staff physician
Setting of care Codes for settings adapted from Gandhi et al11 Primary care
Date of lung cancer diagnosis by pathology September 20, 2005

Abbreviation: CT, computed tomography.

After review of the EHR, we excluded patients who had a recurrence of lung cancer within the previous 5 years. Also excluded were patients whose pathologic diagnoses were made outside the VA setting, provided that they had not presented to the VA earlier with any potentially diagnostic clues for lung cancer.

No timeliness standards for diagnosis currently exist in the United States. However, the British Thoracic Society recommends that patients with suspected lung cancer should undergo an initial evaluation within 1 week of primary care referral and should receive diagnostic tests within 2 weeks of the decision to perform a biopsy.41 Through team consensus and additional literature on test result follow-up,42 we defined two types of missed opportunities that could result in diagnostic delays: (1) type I missed opportunities, defined as episodes of care in which there was failure to recognize a predefined clinical clue (ie, no required action or work-up was initiated within 7 days of clue appearance); appropriate decisions to watch and wait were not considered missed opportunities; and (2) type II missed opportunities, defined as episodes of care in which there was failure to complete within 30 days a diagnostic procedure, consultation, or other requested follow-up action in response to a predefined clue.

We defined the first appearance of a diagnostic clue as the earliest date that the clue could have been recognized by the care providers, regardless of when the patient first started experiencing symptoms. For instance, if a patient had hemoptysis since June 1, 2006, but did not report it to the care provider until December 1, 2006, the first appearance of a diagnostic clue was dated December 1, 2006. We applied rigorous criteria to define missed opportunities that could occur along the diagnostic pathway of lung cancer (Appendix Fig A1, online only). When information in the EHR was vague or inconsistent with expected practices, we used conservative guidelines to avoid overestimating missed opportunities. For example, reviewers were instructed not to record missed opportunities if there was insufficient supporting documentation in the EHR, or when documentation supported an informed decision not to work up a particular clue. No missed opportunity was recorded when delays occurred solely in response to appropriate diagnostic attempts, such as repeated negative bronchoscopies. For each case, we collected information on provider types and specialties, types of diagnostic procedures used, and patient characteristics. We also classified contributing factors in each case into one of three categories (Table 1). “System factors” included scheduling delays, policies, and/or procedures that were judged to have contributed to a missed opportunity. “Provider factors” were attributed in situations when providers failed to recognize previously documented clues or did not follow standards of care or standard policies, resulting in a missed opportunity. Finally, we attributed missed opportunities to “patient factors” (eg, when patients did not adhere to medical advice or appointments).

Data Analysis

After evaluating reviewer agreement on the presence of missed opportunities and reaching consensus on discordant judgments, we identified two groups of patients: (1) those determined to have experienced at least one missed opportunity (of either type), and (2) those determined to have no missed opportunities. We compared these groups on demographic and clinical characteristics, location (Site A v Site B), and provider types and specialties. We then separately evaluated predictors for type I and type II missed opportunities. First, we compared the frequencies of diagnostic clues present in cases with type I missed opportunities and cases in the no missed opportunities group. We then similarly compared the frequencies of specific follow-up actions documented in cases with type II missed opportunities and cases in the no missed opportunities group. Finally, we calculated the median wait times associated with each type of diagnostic clue or follow-up action in type I and type II cases, respectively. We used Fisher's exact test for categoric variables when the assumptions for the χ2 test were not met (two-tailed) and the nonparametric Wilcoxon rank sum test to compare median times to pathological diagnosis.

Finally, we fit three multivariable logistic regression models. The first model predicted the presence of any missed opportunity from provider type and specialty. The other models tested whether particular diagnostic clues or actions were associated with increased risk for type I and type II missed opportunities, respectively. Each model was adjusted for baseline patient characteristics that were distributed unequally between cases with and without missed opportunities. Predictors entered into the initial models included variables that were statistically significant at the 0.1 level in univariate analysis. The final models included only significant predictors. We used SAS version 9.2 (SAS Institute, Cary, NC) for all analyses.

Of 633 new patient cases of lung cancer identified over the study period, 587 met inclusion criteria (Fig 1), and 222 (37.8%) were judged to have missed opportunities after consensus agreements. Before consensus, both reviewers independently agreed on the presence of at least one missed opportunity in 184 patients and on the absence of any missed opportunities in 284 patients (overall κ = 0.69).43 The median time elapsed from first appearance of a diagnostic clue to final pathologic diagnosis was 132.0 days (range, 15 to 2,445 days) in patients with at least one missed opportunity compared with 19.0 days (range, 0 to 870 days) in patients with no identified missed opportunities (P < .001). The outliers in the latter group included patients that required serial imaging and were appropriately followed up. The Venn diagram at the bottom of Fig 1 shows the distribution of provider-related, system-related, and patient-related factors in the 222 patients with at least one missed opportunity.

Type I missed opportunities were judged to occur in 148 (25.2%) of 587 included patients; among these, the median time to pathologic diagnosis was 168 days (range, 15 to 2,445 days; interquartile range, 290 days). Type II missed opportunities occurred in 121 patients (20.6%); in these patients, the median time to pathologic diagnosis was 141.5 days (range, 38 to 2,445 days; interquartile range, 224 days).

We compared baseline characteristics of patients with and without at least one missed opportunity for subsequent inclusion in adjusted predictor models. At the 0.10 level of significance, three comorbidities were more frequent in patients with missed opportunities: hypertension, chronic obstructive pulmonary disease (COPD), and antisocial personality disorder (Table 2). However, only COPD remained statistically significant in subsequent logistic regression models.


Table 2. Baseline Characteristics of Patients With and Without Missed Opportunities

Table 2. Baseline Characteristics of Patients With and Without Missed Opportunities

Characteristics Patients With at Least One Missed Opportunity (n = 222)
Patients With No Missed Opportunities (n = 365)
No. % No. %
Age, years*
    Median 67.9 67.8
    < 65 91 40.9 152 41.6
    65-74 66 29.7 107 29.3
    ≥ 75 64 28.8 106 29.0 .99
    White 168 75.7 284 78.8
    Black 42 18.9 59 16.2
    Other 10 4.5 19 5.2 .66
    Male 221 99.6 360 98.6
    Female 1 0.45 5 1.4 .42
Year of diagnosis*
    2004 5 2.2 15 4.1
    2005 57 25.7 102 28.0
    2006 108 48.6 161 44.1
    2007 50 22.5 84 23.0 .51
    Site A 158 71.2 240 65.8
    Site B 64 28.8 125 34.2 .17
Comorbid medical diseases
    Congestive heart failure 19 8.6 32 8.8 .93
    Coronary artery disease 70 31.5 107 29.3 .57
    Hypertension 152 68.5 223 61.1 .07
    Diabetes 50 22.5 70 19.2 .33
    Chronic obstructive pulmonary disease 99 44.6 121 33.2 .006
    Advanced cardiopulmonary disease with life expectancy < 1 year 2 0.9 5 1.4 .72
    Severely disabled due to medical problem 13 5.9 19 5.2 .74
    Cancer (prior to lung cancer) 58 26.1 76 20.8 .13
    HIV 2 0.9 9 2.5 .22
    Any of the above 199 89.6 314 86.0 .20
Comorbid psychiatric disorders
    Depression 42 18.9 53 14.5 .16
    Anxiety 12 5.4 21 5.8 .86
    Dementia 8 3.6 14 3.8 .89
    Post-traumatic stress disorder 6 2.7 15 4.1 .37
    Schizophrenia 3 1.4 3 0.82 .68
    Bipolar disorder 0 0.0 4 1.1 .30
    Alcohol dependence 30 13.5 52 14.2 .80
    Antisocial personality disorder 3 1.4 0 0.0 .05
    Severely disabled due to psychiatric problem 1 0.45 5 1.4 .42
    Any of the above 78 35.1 121 33.2 .60
Smoking status
    Current smoker 127 57.2 218 59.7
    Prior smoker 88 39.6 130 35.6
    Nonsmoker 7 3.2 17 4.7 .47

*Percentages may not add up to 100% due to missing data.

†Medical record documentation was used to determine comorbid conditions. Advanced cardiopulmonary disease with life expectancy < 1 year determined by documentation of either advanced stage chronic obstructive pulmonary disease (eg, Stage IV), advanced heart failure (eg, Stage IV) or inoperable coronary artery disease combined with the mention of poor prognosis in the medical record.

‡Current smoker, patients who were actively smoking at the time of lung cancer diagnosis; prior smoker, patients who had smoked anytime in the past regardless of quantity and duration; nonsmoker, patients who had never smoked in the past.

Provider characteristics (Table 3) were associated with patients with one or more missed opportunities. In the final adjusted multivariable model, trainees were less likely to be associated with patients with missed opportunities (odds ratio [OR], 0.41; 95% CI, 0.27 to 0.62; referent, staff physician). Whereas emergency medicine providers were relatively unlikely to be associated with missed opportunities (OR, 0.52; 95% CI, 0.28 to 0.96), oncology and pulmonary specialists were overrepresented in patients with missed opportunities (OR, 18.72; 95% CI, 2.30 to 152.46 and OR, 2.35; 95% CI, 1.36 to 4.08, respectively; referent, primary care). For both oncologists and pulmonologists, type I missed opportunities were more frequent. Patient factors were associated with more than half of missed opportunities associated with pulmonary (20 [54%] of 37), but were associated with only two (2 [20%] of 10) missed opportunities related to oncology. Sample sizes were insufficient to test whether these relationships differed between sites.


Table 3. Characteristics of Providers in Patients With and Without Any Missed Opportunities

Table 3. Characteristics of Providers in Patients With and Without Any Missed Opportunities

Characteristics Patients With Any Missed Opportunity (n = 222)
Patients With No Missed Opportunities (n = 365)
No. % No. %
Type of provider*
    Staff physician 119 53.6 136 37.3
    Trainee 50 22.5 145 39.7
    Nurse practitioner 14 6.3 18 4.9
    Physician assistant 32 14.4 31 8.5 < .001
    Generalist/primary care 127 57.2 208 57.0
    Oncology 10 4.5 1 0.3
    Pulmonary 37 16.7 31 8.5
    Other medical subspecialty 7 3.2 1.5 4.1
    Emergency medicine 17 7.7 53 14.5
    Surgery 15 6.8 20 5.5
    Other 0 0.0 1 0.3 < .001

NOTE. Adjustment variable, chronic obstructive pulmonary disease, was also significant (odds ratio, 1.56; 95% CI, 1.08 to 2.24).

*Percentages may not add up to 100% due to missing data.

†Other medical subspecialties include cardiology, nephrology, neurology, rheumatology/immunology, gastroenterology, dermatology, endocrinology, infectious disease, and intensive care.

‡Surgery includes general surgery, cardiothoracic surgery, orthopedic surgery, ophthalmology, otolaryngology, vascular surgery, neurosurgery, plastic surgery, and urology.

Table 4 shows χ2 comparisons of diagnostic clues in patients with type I missed opportunities and no missed opportunities. Median times to clue recognition for missed clues is also listed. An abnormal chest x-ray was the most frequently missed clue, followed by abnormal chest computed tomography scan, and new or worsening persistent cough > 8 weeks. When we relaxed the criterion for recognition from 7 days to 14 days, the total number of patients with type I missed opportunities decreased from 148 to 127. Only recurrent bronchitis was associated with type I missed opportunities in unadjusted and adjusted logistic regression models (adjusted OR, 3.31; 95% CI, 1.20 to 9.10; referent, no recurrent bronchitis). We further assessed whether nonsmokers experienced longer delays from type I missed opportunities (data not shown). We found that of 19 outlier patients,44 11 were smokers, seven were past smokers, and one had never smoked. Smoking history was not associated with outlier status.


Table 4. Diagnostic Clues and Associated Median Time to Clue Recognition in Lung Cancer Patients With and Without Missed Opportunities

Table 4. Diagnostic Clues and Associated Median Time to Clue Recognition in Lung Cancer Patients With and Without Missed Opportunities

Clues Patients With Type I Missed Opportunities (n = 148)
Patients Without Missed Opportunities* (n = 365)
Time to Clue Recognition in Type I Patients (days)
No. %
Median Range IQR No. %
Blood in sputum/hemoptysis 128.5 98.0-159.0 61.0 2 1.4 33 9.0 .09
Recurrent bronchitis or pneumonia 109.0 22.0-293.0 136.0 5 3.4 7 1.9 .04
Abnormal chest x-ray 89.0 8.0-2,011.0 162.5 92 62.2 280 76.7 < .001
Abnormal chest CT 27.0 8.0-1,126.0 49.0 42 28.4 317 86.8 < .001
Abnormal abdomen CT 10.0 8.0-67.0 59.0 3 2.0 18 4.9 .0026
Hoarseness lasting > 2 weeks 109 109-109 0 1 0.7 10 2.7 .23
Unexplained effusion 51.0 12.0-56.0 44.0 3 2.0 10 2.7 .02
Worsening persistent cough/bronchitis or new description of chronic cough lasting > 8 weeks 51.0 8.0-177.0 138.0 11 7.4 64 17.5 < .001
Unexplained weight loss in addition to respiratory symptoms 49.0 12.0-556.0 455.0 7 4.7 74 2.3 < .001
Chest pain or rib pain 77.0 63.0-117.0 54.0 3 2.0 40 11.0 .01
New onset/worsening pain in non-chest location 35.5 12.0-222.0 37.5 8 5.4 28 7.7 < .001

NOTE. The following clues were not seen in any patients: clubbing, new onset Cushing's disease, or superior vena cava obstruction. The following clues were identified only in the no missed opportunities group: abnormal sputum examination/sputum cytology (3), new onset of hypercalcemia symptoms/syndrome (5), and syndrome of inappropriate [secretion of] antidiuretic hormone (2). Adjustment variable, chronic obstructive pulmonary disease, was significant (odds ratio, 1.83; 95% CI, 1.17 to 2.86).

Abbreviations: IQR, interquartile range; CT, computed tomography.

*All clues were recognized in ≤ 7 days.

Table 5 compares the proportions of requested actions (procedures, consultations, or follow-up actions on clues) in patients with type II missed opportunities and no missed opportunities. For missed opportunities, median times to action completion are also listed. Patient factors were strongly associated with type II missed opportunities: completion of needle biopsies (15 [62.5%] of 24), completion of bronchoscopies (15 [100%] of 15), follow-up of abnormal chest x-rays (28 [38.9%] of 72), pulmonary consults (17 [65.4%] of 26), and follow-up of abnormal chest computed tomography scans (11 [61.1%] of 18). Follow-up of abnormal chest x-ray, completion of first needle biopsy, and follow-up of recurrent bronchitis were significant predictors of type II missed opportunities in the unadjusted logistic regression model. In the adjusted model, which controlled for the presence of COPD, only follow-up action on abnormal chest x-ray (OR, 2.07; 95% CI, 1.04 to 4.13; referent, no abnormal chest x-ray) and completion of first needle biopsy (OR, 3.02; 95% CI, 1.76 to 5.18; referent, no needle biopsy) were associated with type II missed opportunities. Appendix Table A1 summarizes logistic regression results for missed opportunities.


Table 5. Requested Actions and Associated Median Time to Completion in Patients With and Without Missed Opportunities

Table 5. Requested Actions and Associated Median Time to Completion in Patients With and Without Missed Opportunities

Requested Follow-Up Actions, Procedures, or Consultations Patients With Type II Missed Opportunities (n = 121)
Patients Without Missed Opportunities* (n = 365)
Time to Completion of Procedure or Consultation or Follow-Up Action Based on the Clue in Type II Patients (days)
No. %
Median Range IQR No. %
Follow-up on blood in sputum 63.0 46-279 233.0 3 2.5 33 9.0 .01
Follow-up on recurrent bronchitis or pneumonia 1075 1,075.0-1,075.0 0 1 0.8 7 1.9 .30
Follow-up on hoarseness lasting > 2 weeks 58.0 48-68 20.0 2 1.6 10 2.7 .06
Follow-up on abnormal chest x-ray 48.0 32-548 23.0 72 59.5 279 76.4 < .001
Follow-up on abnormal chest CT 42.5 31.0-366.0 59.0 18 14.9 314 86.0 < .001
Follow-up on abnormal abdomen CT 39.0 39.0-39.0 0 1 0.8 18 4.9 .10
Follow-up on worsening bronchitis/cough 117.5 31.0-204.0 173.0 2 1.6 64 17.5 .03
Follow-up on unexplained weight loss 98.5 36.0-1,029.0 504.0 4 3.3 74 20.3 < .001
Follow-up on pain in non-chest location 45.0 40.0-72.0 32.0 3 2.5 28 7.7 .005
Pulmonary consult 50.0 33.0-588.0 26.0 26 21.5 283 77.5 < .001
First bronchoscopic biopsy 85.0 31-387 171.0 13 10.7 240 65.8 < .001
Second bronchoscopic biopsy 53 49.0-57.0 8.0 2 1.6 21 5.8 .12
First needle biopsy 50.0 32-253.0 52.0 24 19.8 72 19.7 < .001
Thoracic surgery consult 55.0 34-386.0 49.0 5 4.1 36 9.9 .02
Open lung biopsy 37.5 33-63.0 16.5 4 3.3 9 2.5 .08

NOTE. The following were not seen in any patients: clubbing, new onset Cushing's disease, or superior vena cava obstruction. The following clues/procedure/consultation were identified only in patients with no missed opportunities: chest pain (40), abnormal sputum examination (3), unexplained effusion (10), new onset of hypercalcemia symptoms/syndrome (5), new onset of syndrome of inappropriate [secretion of] antidiuretic hormone (2), third bronchoscopic biopsy (2), second needle biopsy (2), mediastinoscopy (8), thoracentesis (26), and positron emission tomography scan (15).

Abbreviations: IQR, interquartile range; CT, computed tomograhy.

*All actions were completed in ≤ 30 days.

We used an advanced, integrated EHR to discover missed opportunities for an earlier lung cancer diagnosis and found evidence of missed opportunities in more than one-third (n = 222) of 587 patients diagnosed at two institutions. Missed opportunities led to significant delays in diagnosis. More than half of missed opportunities arose from failures to recognize diagnostic clues (in most patients, abnormal imaging results) already present in the EHR. Other missed opportunities resulted from failures to complete key diagnostic procedures or investigations in a timely manner; patient factors often contributed in these cases.

Previous studies of delayed lung cancer diagnosis have relied on interview and questionnaire data2022,27,45 or reviews of paper-based medical records.12,21,27,46 However, recall bias and the large potential for missing information in paper-based records limit the utility of these methods for studying opportunities to improve diagnostic care. Furthermore, previous work has involved a significant amount of subjective judgment11,47,48 and yielded little insight about the frequency and origins of delayed cancer diagnoses.10 Our study overcomes many of these limitations and, to the best of our knowledge, is the largest of its kind.

In the VA system, abnormal imaging results are transmitted to ordering providers through an automated notification system in the EHR.49 Radiologists transmit the abnormal reports to an inbox where the clinician can access and act on the reports. Reports are always accessible to providers and are marked as abnormal to heighten awareness. However, for several reasons, clinicians may not always act on abnormal imaging results in a timely manner.50 It is unlikely that this problem is unique to the VA.13,45,51 Outcomes of imaging notification may actually be better within the VA system because of the integrated nature of its EHR, the presence of a state-of-the-art notification system, and clear policies and procedures for follow-up of diagnostic information.49,52

The origins of missed opportunities are multifactorial, and multidisciplinary strategies are needed to improve the timeliness of the diagnostic process. EHR-based strategies to reduce missed opportunities should target communication, recognition of abnormal imaging results, and monitoring of follow-up actions.49,50 For example, programs in the EHR could identify high-risk patients with abnormal imaging and no evidence of follow-up; the program could then generate a trigger (ie, a signal to alert providers to review the medical record53) to a responsible clinician.54 Second, strategies could be designed to improve recognition of clues that might otherwise stay buried in the wealth of the information available in the EHR. For instance, the documentation of “hemoptysis” or “blood in sputum” in a 60-year-old previous smoker could result in a trigger to initiate or continue the work-up of lung cancer through decision support and text-recognition rules.55

Third, patient transition among different settings of care10 (eg, scheduling and completing procedures) is a high-risk area for preventable breakdowns in communication and coordination. The association of missed opportunities with oncology and pulmonary subspecialties highlights this issue. While trainees were less likely to be associated with missed opportunities, this might be because critical information (such as a test result) transmitted to trainees is also transmitted to their supervising physicians. Nevertheless, current EHR systems may not be able to support the sophisticated degree of tracking providers need to ensure fail-safe follow-up of high-risk patients. While such systems are being designed, the use of lung nodule clinical pathways56 or other programs for patient navigation57 appears promising. Additionally, the VA has recently initiated a national lung cancer collaborative program to improve the timeliness of lung cancer care. These approaches could be particularly beneficial for patients who miss appointments or procedures and are at risk of being lost to follow-up.

Our study findings may not be generalizable outside the VA setting. Moreover, our results may not generalize to other, similar investigations, because our model building was not exclusively based on theory and experience but included predictors on the basis of their chance covariation with the outcome. Studies of diagnostic breakdown traditionally suffer from methodologic limitation of low reliability,58 which we addressed by using two independent reviewers followed by consensus agreements. We may have also missed clues or follow-up actions that were completed but either were not documented in the chart or were documented where the information was hard to find among hundreds of other notes. However, in our previous work, we found that fewer than 2% of providers failed to document follow-up actions related to abnormal imaging results, so it is unlikely that we significantly overestimated missed opportunities because of lack of documentation.50 Another limitation is the lack of comparison information from comparable health systems or from systems that use paper-based records. Hindsight bias40 is of particular concern in studies such as ours, and we tried to minimize it by omitting data collection on outcomes, such as stage at diagnosis and patient harm. Finally, it is not clear whether reducing these delays would improve outcomes.16 Nevertheless, timeliness is considered one of six aims for improving quality of health care.59 Specific strengths of our study included a reliable data collection methodology and a rigorous definition of missed opportunities. Most important, an integrated EHR of a closed health system facilitated collection of data relevant to the entire diagnostic process.

In summary, delays in lung cancer diagnosis are not infrequent. Reducing delays will require strategies to address multiple contributing factors. Potential solutions include using the EHR to improve clinician recognition of abnormal imaging results and instituting programs to track patients with suspicious findings.

© 2010 by American Society of Clinical Oncology

Supported by K23 Career Development Award No. K23CA125585 from the National Institutes of Health (H.S.) and in part by the Houston Veterans Administration Health Services Research and Development Center of Excellence (HFP90-020).

Presented as an abstract at the 32nd Annual Meeting of the Society of General Internal Medicine, May 13-16, 2009, Miami, FL.

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

Authors’ disclosures of potential conflicts of interest and author contributions are found at the end of this article.

The author(s) indicated no potential conflicts of interest.

Conception and design: Hardeep Singh, Kamal Hirani, Himabindu Kadiyala, Olga Rudomiotov, Myrna M. Khan, Terry L. Wahls

Financial support: Hardeep Singh

Administrative support: Traber Davis

Provision of study materials or patients: Kamal Hirani, Himabindu Kadiyala, Olga Rudomiotov, Traber Davis, Terry L. Wahls

Collection and assembly of data: Hardeep Singh, Kamal Hirani, Himabindu Kadiyala, Olga Rudomiotov, Traber Davis, Terry L. Wahls

Data analysis and interpretation: Hardeep Singh, Olga Rudomiotov, Traber Davis, Myrna M. Khan, Terry L. Wahls

Manuscript writing: Hardeep Singh, Myrna M. Khan, Terry L. Wahls

Final approval of manuscript: Hardeep Singh, Kamal Hirani, Himabindu Kadiyala, Olga Rudomiotov, Traber Davis, Myrna M. Khan, Terry L. Wahls

1. RL Phillips Jr, LA Bartholomew, SM Dovey , etal : Learning from malpractice claims about negligent, adverse events in primary care in the United States Qual Saf Health Care 13: 121126,2004 Crossref, MedlineGoogle Scholar
2. WH Goodson III, DH Moore : Overall clinical breast examination as a factor in delayed diagnosis of breast cancer Arch Surg 137: 11521156,2002 Crossref, MedlineGoogle Scholar
3. WH Goodson III, DH Moore : Causes of physician delay in the diagnosis of breast cancer Arch Intern Med 162: 13431348,2002 Crossref, MedlineGoogle Scholar
4. KA Kern : Medicolegal analysis of the delayed diagnosis of cancer in 338 cases in the United States Arch Surg 129: 397403,1994 Crossref, MedlineGoogle Scholar
5. KA Kern : The delayed diagnosis of breast cancer: Medicolegal implications and risk prevention for surgeons Breast Dis 12: 145158,2001 Crossref, MedlineGoogle Scholar
6. SS Raab, DM Grzybicki, JE Janosky , etal : Clinical impact and frequency of anatomic pathology errors in cancer diagnoses Cancer 104: 22052213,2005 Crossref, MedlineGoogle Scholar
7. PM Turkington, N Kennan, MA Greenstone : Misinterpretation of the chest x-ray as a factor in the delayed diagnosis of lung cancer Postgrad Med J 78: 158160,2002 Crossref, MedlineGoogle Scholar
8. CJ Young, JL Sweeney, A Hunter : Implications of delayed diagnosis in colorectal cancer Aust N Z J Surg 70: 635638,2000 Crossref, MedlineGoogle Scholar
9. JS Billing, FC Wells : Delays in the diagnosis and surgical treatment of lung cancer Thorax 51: 903906,1996 Crossref, MedlineGoogle Scholar
10. H Singh, S Sethi, M Raber , etal : Errors in cancer diagnosis: Current understanding and future directions J Clin Oncol 25: 50095018,2007 LinkGoogle Scholar
11. TK Gandhi, A Kachalia, EJ Thomas , etal : Missed and delayed diagnoses in the ambulatory setting: A study of closed malpractice claims Ann Intern Med 145: 488496,2006 Crossref, MedlineGoogle Scholar
12. ER Salomaa, S Sallinen, H Hiekkanen , etal : Delays in the diagnosis and treatment of lung cancer Chest 128: 22822288,2005 Crossref, MedlineGoogle Scholar
13. A Yoshimoto, H Tsuji, E Takazakura , etal : Reasons for the delays in the definitive diagnosis of lung cancer for more than one year from the recognition of abnormal chest shadows Intern Med 41: 95102,2002 Crossref, MedlineGoogle Scholar
14. National Cancer Institute, ClinicalTrials.gov: National Lung Screening Trial (NLST), Identifier NCT00047385, 2002 http://clinicaltrials.gov/ct/show/NCT00047385.%202010 Google Scholar
15. The NHS Cancer Plan. London, United Kingdom, Department of Health, 2000 http://www.uicc-community.org/templates/ccc/images/NCP%20UK.pdf Google Scholar
16. MK Gould, SJ Ghaus, JK Olsson , etal : Timeliness of care in veterans with non-small cell lung cancer Chest 133: 11671173,2008 Crossref, MedlineGoogle Scholar
17. M Kanashiki, H Satoh, H Ishikawa , etal : Time from finding abnormality on mass-screening to final diagnosis of lung cancer Oncol Rep 10: 649652,2003 MedlineGoogle Scholar
18. N O'Rourke, R Edwards : Lung cancer treatment waiting times and tumour growth Clin Oncol (R Coll Radiol) 12: 141144,2000 Crossref, MedlineGoogle Scholar
19. J Corner, J Hopkinson, D Fitzsimmons , etal : Is late diagnosis of lung cancer inevitable? Interview study of patients’ recollections of symptoms before diagnosis Thorax 60: 314319,2005 Crossref, MedlineGoogle Scholar
20. J Corner, J Hopkinson, L Roffe : Experience of health changes and reasons for delay in seeking care: A UK study of the months prior to the diagnosis of lung cancer Soc Sci Med 62: 13811391,2006 Crossref, MedlineGoogle Scholar
21. K Ichinohe, M Takahashi, N Tooyama : Delay by patients and doctors in treatment of Pancoast tumor Wien Klin Wochenschr 118: 405410,2006 Crossref, MedlineGoogle Scholar
22. H Koyi, G Hillerdal, E Branden : Patient's and doctors’ delays in the diagnosis of chest tumors Lung Cancer 35: 5357,2002 Crossref, MedlineGoogle Scholar
23. AM Tod, J Craven, P Allmark : Diagnostic delay in lung cancer: A qualitative study J Adv Nurs 61: 336343,2008 Crossref, MedlineGoogle Scholar
24. AR Jensen, J Mainz, J Overgaard : Impact of delay on diagnosis and treatment of primary lung cancer Acta Oncol 41: 147152,2002 Crossref, MedlineGoogle Scholar
25. MP Devbhandari, MN Bittar, P Quennell , etal : Are we achieving the current waiting time targets in lung cancer treatment? Result of a prospective study from a large United Kingdom teaching hospital J Thorac Oncol 2: 590592,2007 Crossref, MedlineGoogle Scholar
26. AA Powell, EM Schultz, DL Ordin , etal : Timeliness across the continuum of care in veterans with lung cancer J Thorac Oncol 3: 951957,2008 Crossref, MedlineGoogle Scholar
27. A Yilmaz, E Damadoglu, C Salturk , etal : Delays in the diagnosis and treatment of primary lung cancer: Are longer delays associated with advanced pathological stage? Ups J Med Sci 113: 287296,2008 Crossref, MedlineGoogle Scholar
28. SN Weingart, MG Saadeh, B Simchowitz , etal : Process of care failures in breast cancer diagnosis J Gen Intern Med 24: 702709,2009 Crossref, MedlineGoogle Scholar
29. DS Lo, RA Zeldin, R Skrastins , etal : Time to treat: A system redesign focusing on decreasing the time from suspicion of lung cancer to diagnosis J Thorac Oncol 2: 10011006,2007 Crossref, MedlineGoogle Scholar
30. A Moody, M Muers, D Forman : Delays in managing lung cancer Thorax 59: 13,2004 MedlineGoogle Scholar
31. SK Vinod, DL O'Connell, L Simonella , etal : Gaps in optimal care for lung cancer J Thorac Oncol 3: 871879,2008 Crossref, MedlineGoogle Scholar
32. J Månsson, C Bengtsson : Pulmonary cancer from the general practitioner's point of view: Experience from the health centre area of Kungsbacka, Sweden Scand J Prim Health Care 12: 3943,1994 Crossref, MedlineGoogle Scholar
33. M Porta, M Gallen, N Malats , etal : Influence of “diagnostic delay” upon cancer survival: An analysis of five tumour sites J Epidemiol Community Health 45: 225230,1991 Crossref, MedlineGoogle Scholar
34. H Singh, A Naik, R Rao , etal : Reducing diagnostic errors through effective communication: Harnessing the power of information technology J Gen Intern Med 23: 489494,2008 Crossref, MedlineGoogle Scholar
35. H Singh, K Daci, LA Petersen , etal : Missed opportunities to initiate endoscopic evaluation for colorectal cancer diagnosis Am J Gastroenterol 104: 25432554,2009 Crossref, MedlineGoogle Scholar
36. G Buccheri, D Ferrigno : Lung cancer: Clinical presentation and specialist referral time Eur Respir J 24: 898904,2004 Crossref, MedlineGoogle Scholar
37. W Hamilton, D Sharp : Diagnosis of lung cancer in primary care: A structured review Fam Pract 21: 605611,2004 Crossref, MedlineGoogle Scholar
38. DK Lee : Suspected lung cancer: Its initial management and staging Prim Care Respir J 16: 106111,2007 Crossref, MedlineGoogle Scholar
39. SG Spiro, MK Gould, GL Colice : Initial evaluation of the patient with lung cancer: Symptoms, signs, laboratory tests, and paraneoplastic syndromes—ACCP evidenced-based clinical practice guidelines (2nd edition) Chest 132: 149S160S,2007 Crossref, MedlineGoogle Scholar
40. B Fischhoff : Hindsight does not equal foresight: The effect of outcome knowledge on judgment under uncertainty—1975 Qual Saf Health Care 12: 304311,2003 Crossref, MedlineGoogle Scholar
41. BTS recommendations to respiratory physicians for organising the care of patients with lung cancer—The Lung Cancer Working Party of the British Thoracic Society Standards of Care Committee Thorax 53: S1S8,1998 suppl 1 Crossref, MedlineGoogle Scholar
42. D Hanna, P Griswold, LL Leape , etal : Communicating critical test results: Safe practice recommendations Jt Comm J Qual Patient Saf 31: 6880,2005 Crossref, MedlineGoogle Scholar
43. J Cohen : A coefficient of agreement for nominal scales Educ Psychol Meas 20: 3746,1960 CrossrefGoogle Scholar
44. B Rosner : Fundamentals of Biostatistics (ed 5) 2000 Pacific Grove, CA Duxbury Google Scholar
45. M Bjerager, T Palshof, R Dahl , etal : Delay in diagnosis of lung cancer in general practice Br J Gen Pract 56: 863868,2006 MedlineGoogle Scholar
46. G Myrdal, M Lambe, G Hillerdal , etal : Effect of delays on prognosis in patients with non-small cell lung cancer Thorax 59: 4549,2004 MedlineGoogle Scholar
47. ML Graber, N Franklin, R Gordon : Diagnostic error in internal medicine Arch Intern Med 165: 14931499,2005 Crossref, MedlineGoogle Scholar
48. GD Schiff, S Kim, R Abrams , etal : Diagnosing diagnosis errors: Lessons from a multi-institutional collaborative project Advances in Patient Safety 2005 Vol. 2: Google Scholar
49. H Singh, H Arora, MS Vij , etal : Communication outcomes of critical imaging results in a computerized notification system J Am Med Inform Assoc 14: 459466,2007 Crossref, MedlineGoogle Scholar
50. H Singh, EJ Thomas, S Mani , etal : Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: Are electronic medical records achieving their potential? Arch Intern Med 169: 15781586,2009 Crossref, MedlineGoogle Scholar
51. LP Casalino, D Dunham, MH Chin , etal : Frequency of failure to inform patients of clinically significant outpatient test results Arch Intern Med 169: 11231129,2009 Crossref, MedlineGoogle Scholar
52. VR Choksi, CS Marn, Y Bell , etal : Efficiency of a semiautomated coding and review process for notification of critical findings in diagnostic imaging AJR Am J Roentgenol 186: 933936,2006 Crossref, MedlineGoogle Scholar
53. Patient Safety Network 2008 Agency for Healthcare Research and Quality Google Scholar
54. H Singh, EJ Thomas : Diagnostic event triggers: Current state of science and future directions AHRQ Special Report 2009 Google Scholar
55. DW Bates, AA Gawande : Improving safety with information technology N Engl J Med 348: 25262534,2003 Crossref, MedlineGoogle Scholar
56. WE Holden, DM Lewinsohn, ML Osborne , etal : Use of a clinical pathway to manage unsuspected radiographic findings Chest 125: 17531760,2004 Crossref, MedlineGoogle Scholar
57. TA Battaglia, K Roloff, MA Posner , etal : Improving follow-up to abnormal breast cancer screening in an urban population: A patient navigation intervention Cancer 109: 359367,2007 Crossref, MedlineGoogle Scholar
58. EJ Thomas, SR Lipsitz, DM Studdert , etal : The reliability of medical record review for estimating adverse event rates Ann Intern Med 136: 812816,2002 Crossref, MedlineGoogle Scholar
59. Crossing the Quality Chasm: A New Health System for the 21st Century 2000 Committee on Quality of Health Care in America and Institute of Medicine Washington, DC National Academies Press Google Scholar


We thank Annie Bradford, PhD, for assistance with technical writing.


Table A1. Logistic Regression Results for Missed Opportunities for Lung Cancer Diagnosis

Table A1. Logistic Regression Results for Missed Opportunities for Lung Cancer Diagnosis

Provider Characteristics Odds Ratio 95% CI P
Type of provider
    Trainee 0.41 0.27 to 0.62 < .001
    Nurse practitioner 0.99 0.46 to 2.14 .99
    Physician assistant 1.44 0.81 to 2.55 .21
    Referent: staff physician
    Emergency medicine 0.52 0.28 to 0.96 .04
    Oncology 18.72 2.30 to 152.46 .006
    Pulmonary 2.35 1.36 to 4.08 .002
    Other subspecialties 0.92 0.35 to 2.42 .87
    Surgery 1.86 0.89 to 3.87 .10
    Referent: primary care
Diagnostic clues (type I)
    Recurrent bronchitis 3.31 1.20 to 9.10 .02
Requested follow-up actions, procedures, or consultations (type II)
    Abnormal chest x-ray 2.07 1.04 to 4.13 .04
    First needle biopsy 3.02 1.76 to 5.18 < .001

*No patients with missed opportunities.


No companion articles


DOI: 10.1200/JCO.2009.25.6636 Journal of Clinical Oncology 28, no. 20 (July 10, 2010) 3307-3315.

Published online June 07, 2010.

PMID: 20530272

ASCO Career Center