Article Tools
Focus on Quality
Original Contributions
Article Tools
Pioneering Quality Assessment in European Cancer Centers: A Data Analysis of the Organization for European Cancer Institutes Accreditation and Designation Program
Abstract
The OECI A&D program allows comparisons between centers with regard to management, research, care, education, and designate CCs or CCCs. Through a peer review system, recommendations for improvements are given. Assessing the value of the program and outcome of research and patients' management is the next step.
In order to improve the quality of care in Cancer Centers (CC) and designate Comprehensive Cancer Centers (CCCs), the Organization for European Cancer Institutes (OECI) launched an Accreditation and Designation (A&D) program. The program facilitates the collection of defined data and the assessment of cancer center quality. This study analyzes the results of the first 10 European centers that entered the program.
The assessment included 927 items divided across qualitative and quantitative questionnaires. Data collected during self-assessment and peer-review from the 10 first participating centers were combined in a database for comparative analysis using simple statistics. Quantitative and qualitative results were validated by auditors during the peer review visits.
Volumes of various functions and activities dedicated to care, research, and education varied widely among centers. There were no significant differences in resources for radiology, radiotherapy, pathologic diagnostic, and surgery. Differences were observed in the use of clinical pathways but not for the practices of holding multidisciplinary team meetings and conforming to guidelines. Regarding human resources, main differences were in the composition and number of supportive care and research staff. All 10 centers applied as CCCs; five obtained the label, and five were designated as CCs.
The OECI A&D program allows comparisons between centers with regard to management, research, care, education, and designation as CCs or CCCs. Through the peer review system, recommendations for improvements are given. Assessing the added value of the program, as well as research and patient treatment outcomes, is the next step.
Cancer survival has improved throughout Europe in the past decade, but large disparities in survival among individual countries remain, as reported in the latest EUROCARE 5 study.1 Cancer control is an increasing challenge in Europe.
The Organization of European Cancer Institutes (OECI) stimulates collaboration between cancer institutions in order to enable mutual learning and improve the quality of cancer care research and education. It promotes a vision of oncology built on an integrated model of the cancer research-to-care process.2,3 The OECI Accreditation and Designation Program (A&D) was launched in 2008 with three objectives: (1) to provide a comprehensive accreditation for quality oncology care, taking into account prevention, care, research, education, and networking; (2) to develop a database of cancer centers in Europe, with information on their resources and activities; and (3) to designate the various types of cancer structures, including Comprehensive Cancer Centers (CCCs).4 The program addresses one of the key issues in current cancer care: rapid translation of research results into clinical and daily practice.4 This bottom-up approach differs from that of the United States, where accreditation is provided for by the Joint Commission, and the recognition of extraordinary translational cancer research programs as a dedicated aspect of CCCs by the National Cancer Institute.
Publications on the impact of accreditation on hospitals5 and its evaluation as a quality management tool6 are limited; none have focused on cancer. However, using the Donabedian structure (processes influence outcome) quality health outcome models were developed. Structural elements (such as the existence of a business plan, including aims to achieve, budget needed, evaluation of the plan), and process elements (such as use of guidelines, implementation and evaluation of clinical pathways) were known to be of influence on organizational performance for other illnesses.7–12 The OECI used this existing knowledge and translated it into comprehensive cancer care. Combining the experience of the multidisciplinary comprehensive cancer system in the Netherlands (a bottom-up system: by professionals for professionals to improve cancer performances in hospitals), the US designation of cancer centers (CC) for the health care insurance companies (top down, by the joint Commission), and the already existing quantitative questionnaire within the OECI, the A&D program was developed by the OECI.
We report here on a first analysis of the quantitative and qualitative data and comparative outcome of the A&D Program for the first series of 10 European Cancer Centers that have participated. The objective of this study is to describe the current landscape of cancer centers that have received accreditation and their compliance with the standards of the OECI program.
The accreditation process starts with a preliminary designation (Figure 1) followed by a 6-month self-assessment and a peer review visit by four auditors from different specialties. Auditors assess whether the organization meets the quality standards and to what degree the quality system has been implemented. Once the center has been reviewed, the OECI delivers a report identifying quality concerns and recommendations for an improvement plan. If the center is approved, the OECI gives the accreditation and final designation as a cancer unit, clinical CC; cancer research center, or CCC (see Designation definitions in Appendix 1, Data Supplement).
Figure 1. Preliminary designation decision tree.
Participation in the A&D program is voluntary. All OECI member centers (N = 77) were invited to participate in the program, and 27 accepted initially. Each application is assessed by the OECI A&D board regarding criteria that should be fulfilled before entering the program: the center commits to the completion of the program within the allocated time frame, the management structure is stable, no major changes (such as mergers) are expected, and cancer care is performed on an identifiable unit with a separate budget, management, and organizational structure. All participing center gave informed consent for the disclosure of their data on an anonymous basis; for this study, centers have been coded as center A, center B, and so on.
The assessment consists of 927 items divided across qualitative (n = 264) and quantitative (n = 663) questionnaires, covering management, prevention, care, infrastructures, human resources, research, and patient centeredness. Items are classified in organizational performance areas referred to as chapters. Qualitative chapters include 1: General standards, strategic plan and general management; 2: Screening, primary prevention, and health education; 3: Care; 4: Research, innovation, and development; 5: Teaching and continuous education; and 6: Patient related. Quantitative chapters include 1: Infrastructure for cancer care, 2: human resources, 3: research, and 4: education.
Qualitative questions are scored from 1 to 4, depending on the extent of implementation of the standard. The implementation level is based on Donabedian's PDCA (plan, do, check, act) A fully implemented standard received a score of 4 points; mostly implanted, 3 points; partially implemented, 2 points; and not implemented, 1 point. In order to calculate the global score for each chapter, a calculation is made using the number of times a specific answer (eg, mostly) is given, multiplied by the points allocated to each answer. That number is divided by the maximum possible score (total number of answers multiplied by 4).
Qualitative data are represented in two superimposed spider diagrams gathering the overall information for each center, as assessed during self-assessment and by the auditors (Appendix Figure A1, online only). Information includes the size of the gaps among the six organizational performance areas (chapters); the important categories of performance, as well as concentrations of strengths and weaknesses; and how the CC staff has evaluated the organizational performance areas compared with the auditor's scorings. Data are collected during self-assessment and peer review on the A&D Web tool (http://oeci.selfassessment.nu/cms) and aggregated in a single database to allow for comparisons among centers.
Ten cancer centers participated in the program between 2008 and 2012, and all initially applied as CCCs. Two identified as academic institutions, seven as public/nonprofit, and one as private.
There is wide difference in the reported number of new patients with cancer per year: Centers C indicated 2,481 new patients; this figure increases to 11,594 for center J (median 6,890), as shown in Table 1.
|
| Category | Center A | Center B | Center C | Center D | Center E | Center F | Center G | Center H | Center I | Center J |
|---|---|---|---|---|---|---|---|---|---|---|
| Activities | ||||||||||
| No. of new patients | 10,174 | 2,983 | 2,481 | 10,363 | 7,317 | 5,319 | 6,462 | 2,392 | 7,344 | 11,594 |
| No. of inpatient visits for overnight stays | 11,470 | 40,275 | 6,207 | 7,556 | 40,424 | 4,239 | 11,715 | 5,968 | 23,625 | 17,248 |
| No. of outpatient visits in consultation | 143,586 | 135,871 | 121,783 | 66,441 | 94,116 | 43,494 | 105,818 | 75,526 | 127,050 | 128,643 |
| No. of radiotherapy sessions | 2,573 | 1,210 | 1,371 | 7,319 | 2,278 | 3,506 | 2,711 | 1,425 | 3,880 | 2,456 |
| No. of bone marrow and/or stem cell transplantations per year | 97 | 7 | 0 | 100 | 29 | 79 | 0 | — | 156 | 195 |
| No. of samplings for tumor diagnosis | 18,752 | 77,314 | 98,399 | 5,855 | 20,663 | 24,216 | 20,009 | 51,328 | 56,839 | 17,652 |
| Facilities | ||||||||||
| No. of ambulatory day care beds or chairs | 78 | 24 | 31 | 78 | 33 | 43 | 32 | 13 | 79 | 89 |
| No. of inpatient beds for overnight stays | 319 | 160 | 186 | 173 | 180 | 288 | 370 | 154 | 241 | 339 |
| No. of cameras for nuclear medicine | 4 | 2 | 1 | 4 | 3 | 3 | 1 | 3 | 8 | 2 |
| No. of operating rooms | 7 | 45 | 11 | 4 | 6 | 5 | 7 | — | 10 | 10 |
| No. of intensive care beds | 8 | 13 | 0 | 6 | 6 | 5 | 12 | — | 5 | 28 |
| Total No. of facilities for radiology (mammography, MRI spectroscopy, facilities for MRI and CT scanners | 9 | 10 | 5 | 5 | 5.5 | 6 | 5 | — | 34 | 4 |
Abbreviations: CT, computed tomography; MRI, magnetic resonance imaging.
However, numbers for use of inpatient beds do not follow the same distribution, ranging from 15 new patients per bed to 59 new patients per bed. CCs with a number of new patients above the median have more than 30 new patients per bed, whereas centers with a number of new patients below the median treat 13 to 18 new patients per bed.
Regarding ambulatory care, we again see a wide distribution. Centers that treat fewer than 10,000 new patients show a varied ratio of new patients per bed, ranging from 80 to 221, whereas the three centers with more than 10,000 new patients have a ratio of 130 new patients per bed.
There are also differences in radiotherapy and radiology resources: all centers perform brachytherapy, and most centers (7 of 10) perform intensity-modulated radiotherapy. However, only two centers perform intraoperative radiotherapy, and two have access to proton therapy resources. Five centers perform stereotactic radiotherapy.
Except center I (7344 new patients), which hosts 34 different radiology machines, all centers are equipped with five to 10 different facilities. Of note, however, center I is a CC within a general university hospital, and the radiology facilities available are not dedicated to the CC only.
All centers use laparoscopy and sentinel node techniques. Common techniques such as laser therapy, radio frequency ablation, are in place in seven of the centers. Only half of the centers use intraoperative chemotherapy, hyperthermia, and isolated limb perfusion.
Only three centers have a bone marrow bank on site. While two centers do not undertake any bone marrow/stem-cell transplants (BMT), one center performs 195 BMTs in a year.
There are large differences in the use of clinical pathways (CPs) but few in the practice of multidisciplinary teams (MDTs). Only two centers (A and B) have CPs in place for all pathologies; center G and center D have CPs in place for all pathologies but one.
Excluding pediatric malignancies CPs (three of 10), and hematology CPs (five of 10) which exist only in centers providing such care, CPs have been developed for the most common cancers (breast, lung, gastrointestinal, gynecological, head and neck) in more than 90% of centers but are lacking in 30% of centers for bone, soft tissue, urologic, and skin cancers.
Most centers (except C, F, and G) work with guidelines and MDTs for all cancers. Centers C, F, and G have guidelines for all cancers except neuro-oncologic, bone, and soft tissue cancers, and MDTs for all cancers except neuro-oncological cancers. In 80% of centers, the MDTs are composed of at least a medical oncologist, surgical oncologist, radiotherapist, radiologist, pathologist, and nurse.
The number of employees dedicated to cancer care in the centers, expressed in full-time equivalents (FTE), varies from 460 to 11,447. The number of FTEs per patient varies from 2.6 to 21, with a mean of 8.5 (Figure 2A). There was no correlation between the patient volume of the CC and the number of FTEs.
Figure 2. Graphical representation of selected quantitative results. (A) Number of employees (in full-time equivalent [FTE]) dedicated to patients with cancer reported to total number of new patients per center (colors represent centers A to J), (B) number of supportive care staff for each cancer center (colors represent different types of supportive care staff), (C) number of research staff (in FTE) in Comprehensive Cancer Centers compared with clinical cancer centers (colors represent centers A to J), (D) number of research staff for each cancer center (colors represent different types of research area).
The number of FTE nurses per new inpatient or day care patient is not correlated to the number of FTE physicians per patient, indicating different types of human resources organization at the care level. In terms of human resources dedicated to care, a notable difference is found in the size and composition of the supportive care team (SCT; Figure 2B). Numbers of SCT FTEs are not correlated to the number of new patients.
The number of research-related FTEs (including researchers and technicians) varies from 17 to 132 in clinical CCs and from 134 to 479 in CCCs, with the highest numbers in center E (479) and J (370; Figure 2C).
The repartition of staff in different research areas (Figure 2D) seems to be related to the expertise of each center. In the two centers with the highest number of researchers, most research staff work in (onco)genomics and cell biology (center E) and in clinical trials and immunology (center J). The number of ongoing active studies varies from 6 to 252 in clinical CCs and from 91 to 508 in CCCs, with an overall median number of 91.
Center J (second highest in terms of FTE researchers) had the highest number of publications with an impact factor > 10 (n = 112), the highest number of international publications (661) and patents (50), and a high accrual rate of patients in trials (19%). Despite having 100 more researchers than center J, center E did not score as well. In terms of patient's accrual in clinical trials, centers could be grouped into two categories: five centers enrolled > 10% of their patients (range 10% to 23.5%), whereas the other five enrolled < 5% (0.07% to 3%).
There are no notable differences in the education resources and activities such as information centers, medical libraries, educational courses, and continuing medical education. There is a difference between CCs and CCCS in the existence of courses with international audiences.
The centers receive from 0 to 509 medical students per year and from 24 to 400 student nurses per year. In accordance with the number of researchers and research production, centers I, E, and J welcome the highest number of PhD students and produce the highest number of PhD theses per year.
Qualitative assessment covers the six organizational performance areas described previously. A global score for each chapter was calculated (see Methods for the detail of the calculation). A graphic representation of this calculation for all centers shows the results according to both the self-assessment and the peer review visit (Appendix Figure 1C). For all centers and almost all areas, the scoring done by the center through self-assessment is higher than the scoring of the peer review visit; centers A and C provided the most discordant assessments.
Although patient care scores best among the performance areas for all centers, important deficiencies can be seen in research (centers A, B, C, F, G, H, and I), education (centers C, F, and I), and even general oncological management (centers A, C, F, and H). Centers D, E, and J appear to adequately meet quality requirements in all areas.
Results of the peer-review with a selection of comments from auditors are presented in the Data Supplement. Recommendations cover mainly leadership issues, deliver of care, clinical decision support, and integration with research. Despite the existence of MDTs in all centers for most tumor sites, recommendations to improve MDT structure were provided for six of the 10 centers. Drug prescription system appeared as an area in need of improvement for five centers.
The consistency between quantitative data and qualitative findings was assessed. The strength or weakness of departments such as supportive care, pathology, or clinical research, as assessed by the auditors, is related to the number of staff in those departments, as can be seen for center B (weakness in SCT). Centers such as E and J with high levels of research resources and production were also identified as models for organization of research integration into care.
Five cancer centers applied for designation as a CCC and achieved it. Five cancer centers applied for designation as a CCC but did not achieve it (four of them were designated clinical CCs and one of them is awaiting designation pending major changes). For three of the latter, failure to obtain designation was related to research shortcomings; for the fourth center, it was research and care, and for the last it was care alone. These criteria were either quantitative (mainly the lack of publications in journals with high impact factor and lack of clinical trials) or qualitative (lack of integration of research into care or between laboratories).
The criteria related to care that caused the centers to fail to achieve designation as a CCC were purely qualitative, and mainly concerned lack of harmonization between patients and quality policy and guidelines. The comprehensiveness and clear dedication of an identifiable integrated structure for cancer management was a key issue in obtaining CCC designation, especially for cancer structures in large university hospitals.
The objective of this article was to report the global results of the first series of OECI centers participating in the A&D program and to investigate the potential use of the questionnaire data. The data collected allow comparisons between the centers, especially with regard to patient numbers or tumor type portfolio.
In terms of care, resources and infrastructures are not homogeneous among the centers, especially when results are reported as the number of patients or activities. Data linked to SCT show that supportive care remains an area of wide discrepancies. Differences are even wider in the research area and clearly allow for a distinction between CCs and CCCs.
Data about the number of researchers can help define the critical mass of researchers and the research specialization of a center; with regard to expenditure, such data can also provide information on investment priorities and relative overhead per FTE.
There are some limitations to the study. First, it is likely that at least some data may have been collected in different ways in the centers. This could be due to different understanding of the question or different reporting methods. Furthermore, we believe that the data should be analyzed in the context of the different national health systems. For example, the program is currently ongoing for all cancer centers in Italy, and this will allow us to assess the program for centers located in the same cultural, organizational, and regulatory framework.
Several accreditation programs have been developed worldwide. They consist mainly of general accreditation programs for hospitals.13 Further accreditations of specific structures, such as hematopoietic stem-cell transplantation units,14 palliative care,15 and research tissue banks,16 have been proposed. Some countries have established tumor-based accreditation systems.17,18 Finally, many countries have research evaluation programs, such as the National Cancer Institute in the United States. However, the OECI offers the only accreditation program dedicated to care, research, and education together, assessing the integration of those three aspects for cancer care specifically in the institutional setting.
The data collected so far are mainly structure and process related, whereas outcome data are preferred if performance is to be presented or compared. In oncology, this is an issue because medical records content should be connected to cancer registries in order to obtain sufficsient long-term follow-up and survival data. In the present update of the A&D program, an effort to assemble outcome data will be made. In the literature so far, there is little evidence of accreditation actually improving patient outcomes, as a result of either methodological issues in comparing organizational performance or the “black box” character of the mechanism through which A&D exerts its effect. Nevertheless, a recent article19 showed improved survival and faster adoption of some innovative procedures after peer review of cancer services in general hospitals.
Because its final goal is to improve quality of care by integration of high-quality cancer research, the OECI is currently developing a benchmarking project based on the A&D program. The general objective of this project is to benchmark comprehensive cancer care and yield best practice examples in a way that contributes to improving the quality of interdisciplinary patient treatment. Indicators are defined and pilot tested in the centers to measure the outcome and impact of the translational research carried out in those centers. The A&D program is in the process of reviewing the standards. A new questionnaire will be developed in which a division will be made between mandatory standards and optional standards.
In conclusion, the A&D program has been set up as a response to the need to promote integration of research, education, and clinical services. Setting up minimum standards of care and research and designating CCCs is only the first step of this process. Measuring the translational research and patients' outcome performed through these centers is the next step.
Acknowledgment
We wish to thank the directors of the cancer centers for providing the data and supporting this analysis.
Supported by the Wallenberg foundation, the Organization for European Cancer Institutes, and Framework Programs 6 and 7 of the European Commission.
Although all authors completed the disclosure declaration, the following author(s) and/or an author's immediate family member(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.
Employment or Leadership Position: Dominique de Valeriola, Organization for European Cancer Institutes (U), Institut Jules Bordet (C) Consultant or Advisory Role: None Stock Ownership: None Honoraria: None Research Funding: None Expert Testimony: None Patents, Royalties, and Licenses: None Other Remuneration: None
Conception and design: Mahasti Saghatchian, Henk Hummel, Christopher Harrison, Dominique de Valeriola, Renee Otter, José Laranja Pontes, Claudio Lombardo, Ulrik Ringborg, Thomas Tursz, Wim van Harten
Administrative support: Bert Koot
Collection and assembly of data: Mahasti Saghatchian, Frédérique Thonon, Femke Boomsma, Bert Koot, Eoin McGrath, Wim van Harten
Data analysis and interpretation: Mahasti Saghatchian, Frédérique Thonon, Christopher Harrison, Abinaya Rajan, Renee Otter, José Laranja Pontes, Wim van Harten
Manuscript writing: All authors
Final approval of manuscript: All authors
| 1. | R De Angelis , etal: Cancer survival in Europe 1999-2007 by country and age: Results of EUROCARE-5—a population-based study Lancet Oncol 15: 2– 3,2014 23 34 35 47 Crossref, Medline, Google Scholar |
| 2. | U Ringborg : The Stockholm declaration Mol Oncol 2: 10– 11, Crossref, Medline, Google Scholar |
| 3. | U Ringborg, M Pierotti, G Storme , etal: Managing cancer in the EU: The Organisation of European Cancer Institutes (OECI) Eur J Cancer 44: 772– 773,1990 Crossref, Google Scholar |
| 4. | M Saghatchian, H Hummel, R Otter , etal: Towards quality, comprehensiveness and excellence. The accreditation project of the Organisation of European Cancer Institutes (OECI) Tumori 94: 164– 171,2008 Crossref, Medline, Google Scholar |
| 5. | A Alkhenizan, C Shaw : Impact of accreditation on the quality of healthcare services: A systematic review of the literature Ann Saudi Med 31: 407– 416,2011 Crossref, Medline, Google Scholar |
| 6. | MP Pomey, L Lemieux-Charles, F Champagne , etal: Does accreditation stimulate change? A study of the impact of the accreditation process on Canadian healthcare organizations Implement Sci 5: 31,2010 Crossref, Medline, Google Scholar |
| 7. | K Vanheacht, M Panella, R Van Zelm , etal: An overview on the history and concepts of care pathways as complex interventions Int J Care Pathw 14: 117– 123,2010 Crossref, Google Scholar |
| 8. | R Simons, S Kasic, A Kirkpatrick , etal: Relative importance of designation and accreditation of trauma centers during evolution of a regional trauma system J Trauma 52: 827– 833,2002 Medline, Google Scholar |
| 9. | MP Pomey, AP Contandriopoulos, P François , etal: Accreditation: A tool for organizational change in hospitals? Int J Health Care Qual Assur Inc Leadersh Health Serv 17: 113– 124,2004 Crossref, Medline, Google Scholar |
| 10. | J Braithwaite, D Greenfield, J Westbrook , etal: Health service accreditation as a predictor of clinical and organisational performance: A blinded, random, stratified study Qual Saf Health Care 19: 14– 21,2010 Crossref, Medline, Google Scholar |
| 11. | R Wells, CH Lemak, JA Alexander , etal: Do licensing and accreditation matter in outpatient substance abuse treatment programs? J Subst Abuse Treat 33: 43– 50,2007 Crossref, Medline, Google Scholar |
| 12. | D Greenfield, J Braithwaite : Health sector accreditation research: A systematic review Int J Qual Health Care 20: 172– 183,2008 Crossref, Medline, Google Scholar |
| 13. | MT Fortes, RA Mattos, TW Baptista : Accreditation or accreditations? A comparative study about accreditation in France, United Kingdom and Cataluña Rev Assoc Med Bras 57: 239– 246,2011 Crossref, Medline, Google Scholar |
| 14. | D Samson, I Slaper-Cortenbach, D Pamphilon , etal: Current status of JACIE accreditation in Europe: A special report from the Joint Accreditation Committee of the ISCT and the EBMT (JACIE) Bone Marrow Transplant 39: 133– 141,2007 Crossref, Medline, Google Scholar |
| 15. | N Cherny, R Catane, D Schrijvers , etal: European Society for Medical Oncology (ESMO) Program for the integration of oncology and palliative care: A 5-year review of the Designated Centers' incentive program Ann Oncol 21: 362– 369,2010 Crossref, Medline, Google Scholar |
| 16. | E Herpel, C Röcken, H Manke , etal: Quality management and accreditation of research tissue banks: Experience of the National Center for Tumor Diseases (NCT) Heidelberg Virchows Arch 457: 741– 747,2010 Crossref, Medline, Google Scholar |
| 17. | SY Brucker, M Bamberg, W Jonat , etal: Certification of breast centres in Germany: Proof of concept for a prototypical example of quality assurance in multidisciplinary cancer care BMC Cancer 9: 228,2009 Crossref, Medline, Google Scholar |
| 18. | P Mroczkowski, S Hać, M Mik , etal: Preliminary results of the first quality assurance project in rectal cancer in Poland Pol Przegl Chir 83: 144– 149,2011 Crossref, Medline, Google Scholar |
| 19. | MJ Kilsdonk, BA van Dijk, R Otter , etal: The impact of organisational external peer review on colorectal cancer treatment and survival in the Netherlands Br J Cancer 110: 850– 858,2014 Crossref, Medline, Google Scholar |
Figure A1. Superimposed spider diagrams gathering the overall information for each center, as assessed during self-assessment and by the auditors. General Management refers to chapter 1: General standards, strategic plan, and general management; Prevention refers to chapter 2: Screening and primary prevention and health education; Care refers to chapter 3: Care; Research refers to chapter 4: Research; Education refers to chapter 5: Education and teaching; Patient-related refers to chapter 6: Patient related.
