Volume 38, Issue 7 p. 1253-1261
Research Article
Open Access

Validation of the Rainbow Model of Integrated Care Measurement Tool in Parkinson's Disease

Pim P. Valentijn PhD

Corresponding Author

Pim P. Valentijn PhD

Department of Health Services Research, School for Public Health and Primary Care, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands

Essenburgh Research & Consultancy, Essenburgh Group, Harderwijk, the Netherlands

Correspondence to: Dr. Pim P. Valentijn, Department of Health Services Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Duboisdomein 30, 6229 GT Maastricht, the Netherlands; E-mail: [email protected]

Search for more papers by this author
Carsten Eggers MD

Carsten Eggers MD

Department of Neurology, University Hospital of Marburg, Center for Mind, Brain and Behavior, Universities Marburg and Giessen, Marburg, Germany

Department of Neurology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany

Search for more papers by this author
Bastiaan R. Bloem MD, PhD

Bastiaan R. Bloem MD, PhD

Scientific Center for Quality of Healthcare, Radboud Institute for Health Sciences, Radboud University Medical Centre, Nijmegen, the Netherlands

Department of Neurology, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, the Netherlands

Search for more papers by this author
David Grimes MD

David Grimes MD

Neurology, Department of Medicine, The Ottawa Hospital, Ottawa, Ontario, Canada

Search for more papers by this author
Jennifer Goldman MD, MS

Jennifer Goldman MD, MS

Parkinson's and Movement Disorders Shirley Ryan Ability Lab and Departments of Physical Medicine and Rehabilitation and Neurology, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA

Departments of Physical Medicine and Rehabilitation and Neurology, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA

Search for more papers by this author
Jennifer McGinley PhD

Jennifer McGinley PhD

Physiotherapy, The University of Melbourne, Parkville, Victoria, Australia

Search for more papers by this author
Joan Gardner RN, BSN

Joan Gardner RN, BSN

Struthers Parkinson's Center, HealthPartners, Minneapolis, Minnesota, USA

Search for more papers by this author
Maria Elisa Pimentel Piemonte PT, PhD

Maria Elisa Pimentel Piemonte PT, PhD

Physical Therapy, Speech Therapy, and Occupational Therapy Department, University of São Paulo, São Paulo, Brazil

Search for more papers by this author
Nabila Dahodwala MD

Nabila Dahodwala MD

Parkinson's Disease and Movement Disorders Center, Department of Neurology, University of Pennsylvania, Philadelphia, Pennsylvania, USA

Search for more papers by this author
Laura Brennan PhD

Laura Brennan PhD

Thomas Jefferson University, Philadelphia, Pennsylvania, USA

Search for more papers by this author
Robert Iansek PhD

Robert Iansek PhD

Clinical Research Centre for Movement Disorders and Gait, Comprehensive Parkinson Care Program, Parkinson Foundation Centre of Excellence, Kington Centre Monash Health Cheltenham, Cheltenham, Australia

Search for more papers by this author
Norbert Kovacs MD, PhD

Norbert Kovacs MD, PhD

Department of Neurology, Medical School, University of Pecs, Pecs, Hungary

ELKH-PTE Clinical Neuroscience MR Research Group, Pecs, Hungary

Search for more papers by this author
Sotirios Parashos MD, PhD

Sotirios Parashos MD, PhD

Struthers Parkinson's Center, HealthPartners, Minneapolis, Minnesota, USA

Search for more papers by this author
Ariela Hilel RN, MA

Ariela Hilel RN, MA

Movement Disorders Center Unit, Neurology Division, Tel Aviv Medical Center, Tel Aviv, Israel

Search for more papers by this author
Roopa Rajan MD, DM

Roopa Rajan MD, DM

Department of Neurology, All India Institute of Medical Sciences, New Delhi, India

Search for more papers by this author
on behalf of the MDS Study Group on Management of Movement Disorders: Interdisciplinary and Integrated Care

the MDS Study Group on Management of Movement Disorders: Interdisciplinary and Integrated Care

Search for more papers by this author
First published: 06 May 2023

Relevant conflicts of interest/financial disclosures: Nothing to report.

Full financial disclosures and author roles may be found in the online version of this article.

Abstract

Background

Integrated care is essential for improving the management and health outcomes for people with Parkinson's disease (PD); reliable and objective measures of care integration are few.

Objective

The aim of this study was to test the psychometric properties of the Rainbow Model of Integrated Care Measurement Tool (RMIC-MT, provider version) for healthcare professionals involved in PD care.

Methods

A cross-sectional survey was administered online to an international network representing 95 neurology centers across 41 countries and 588 healthcare providers. Exploratory factor analysis with principal axis extraction method was used to assess construct validity. Confirmatory factor analysis was used to evaluate model fit of the RMIC-MT provider version. Cronbach's alpha was used to assess the internal consistency reliability.

Results

Overall, 371 care providers (62% response rate) participated in this study. No item had psychometric sensitivity problems. Nine factors (professional coordination, cultural competence, triple aims outcome, system coordination, clinical coordination, technical competence, community-centeredness, person-centeredness, and organizational coordination) with 42 items were determined by exploratory factor analysis. Cronbach's alpha ranged from 0.76 (clinical coordination) to 0.94 (system coordination) and showed significant correlation among all items in the scale (>0.4), indicating good internal consistency reliability. The confirmatory factor analysis model passed most goodness-of-fit tests, thereby confirming the factor structure of nine categories with a total of 40 items.

Conclusions

The results provide evidence for the construct validity and other psychometric properties of the provider version of the RMIC-MT to measure integrated care in PD. © 2023 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

Introduction

Integrated care is considered essential for improving the management and health outcomes for people with Parkinson's disease (PD).1 Integrated care can be defined as a coordinated way of working across multiple professionals, organizations, and sectors to improve the health, quality of care, and economic outcomes for a targeted population.2 It is a care approach intended to strengthen people-centered health systems through the promotion of coordinated care delivery by a multidisciplinary or interdisciplinary team that works across healthcare settings and levels of care.3 The Rainbow Model of Integrated Care (RMIC) identifies four core domains (person-centeredness, service coordination, professional coordination, and organizational coordination) and four ancillary domains (community-centeredness, technical competence, cultural competence, and system context) that are central to many integrated care models4 (Fig. 1). Both coordinated care and integrated care are used interchangeably and referred to as “integrated care” throughout this article.5 Comprehensive care has been strongly recommended by the World Health Organization for people living with chronic diseases, including PD. The World Health Organization advocates that integrated health services are crucial for the implementation of this kind of care.

Details are in the caption following the image
Rainbow Model of Integrated Care (RMIC). The RMIC describes three categories of integrated care: the scope, type, and enablers of integration, including eight domains. Figure is a graphical representation of the RMIC. The scope entails the person-centeredness and community-centeredness perspective of professionals, eg, focusing on patient needs and abilities instead of the disease (person-centeredness) and meeting a target group's specific healthcare requirements (community-centeredness). The type of integration consists of coordination activities at the micro (individual), meso (population), and macro (system) level and refers to four domains: (1) delivered and coordinated services to patients (clinical coordination), (2) collaboration between healthcare professionals (professional coordination), (3) collaboration between healthcare organizations (organizational coordination), and (4) implementation of new policies and regulations (system coordination). Functional (technical competence) and normative (cultural competence) enablers are needed to establish connectivity between the micro, meso, and macro levels. Technical competence refers to communication tools that can be used by all professionals and organizations in a network, whereas cultural competence refers to the development and maintenance of a common goal or plans for improvement. (The Rainbow Model for Integrated Care by P.P. Valentijn, 2015. Copyright 2017 by Essenburgh Group, Harderwijk, the Netherlands. Reproduced with permission.19) [Color figure can be viewed at wileyonlinelibrary.com]

Measuring integrated service delivery is considered key to evaluating and improving the quality of care. Yet, the majority of measurement tools have substantial limitations regarding their comprehensiveness, psychometric quality, and length of use in daily practice.6 Consequently, a standardized and validated measure is needed to facilitate the integrated service delivery for chronic neurodegenerative diseases, such as PD.

The RMIC measurement tools for patients and healthcare providers have been tested and validated for several disease areas, such as diabetes care,7, 8 primary care,2, 9 elderly care,10 and renal care.5 These studies showed that both versions of the measurement tool were easy-to-use instruments with good psychometric properties. However, further work is needed to validate the psychometric properties of the RMIC-MT for PD. The aim of this study was to test the psychometric properties of the Rainbow Model of Integrated Care Measurement Tool (RMIC-MT, provider version) for healthcare providers within the context of PD care.

Materials and Methods

A cross-sectional study design including a convenience sample of 588 care providers within an international network of 95 neurology centers that provide PD care in 41 countries (Argentina, Armenia, Australia, Austria, Belgium, Brazil, Canada, Chile, China, Finland, France, Germany, Greece, Guatemala, Hungary, India, Indonesia, Italy, Japan, Kazakhstan, Kenya, Kuwait, Latvia, Lebanon, Malaysia, Mexico, Morocco, the Netherlands, Norway, Poland, Romania, Russia, Senegal, Slovakia, Spain, Sweden, Taiwan, Turkey, Ukraine, United Kingdom, and United States) was used for validating the psychometric properties of the RMIC-MT provider version. The RMIC-MT provider survey was distributed online to healthcare providers from multiple professional disciplines (ie, neurologists, PD nurses, physical therapists) in each of the participating sites between 14 January 2021 and 24 March 2021.

Study Population

Target participants were healthcare professionals registered as members of the International Parkinson and Movement Disorder Society. First, the members received an invitation to participate in the study, collecting basic information about their centers' care coordinator and team members. Second, the identified care coordinators and their team members received an invitation to complete the RMIC-MT provider tool. In the participating centers, healthcare providers were considered eligible to participate in the study if they met the following criteria: (1) aged 18 years or older, and (2) were actively involved in the clinical and/or administrative process of the center.

Study Variables

The RMIC-MT provider version assessed how PD care professionals perceived the centers’ ability to deliver integrated care on a five-point Likert scale (ie, never, rarely, sometimes, often, always) on 48 items: person-centeredness (eg, needs assessment), community-centeredness (eg, population screening), service coordination (eg, personal care plan), professional coordination (eg, multidisciplinary team), organizational coordination (eg, interorganizational partnerships), system coordination (eg, policy and financing), technical competence (eg, interoperable medical records), and cultural competence (eg, collaboration culture). It is available for use under a specific license and can be obtained from the Essenburgh Group (https://www.essenburgh.com/the-rainbow-model-measurements-tools-for-integrated-care/). In addition, we collected data on geographical location, type of profession, and center characteristics, such as number of patients seen, team members included, and center's standard operating processes.

Sample Size Calculation

The estimated minimal sample was based on the requirement of 10 subjects per item within the survey for an explorative factor analysis.11 Given that the RMIC-MT provider version had 48 items, the required sample size was 480.

Statistical Analysis

Data were entered, cleaned, and checked before the analysis. Continuous variables were expressed as mean and standard deviations. Frequencies and percentages were used for categorical variables. Distribution properties of responses to the RMIC-MT items were used to study the psychometric sensitivity. Exploratory factor analysis based on a correlation matrix with principal axis factoring extraction method and (oblique) rotation was used to assess the underlying structure of the PC survey. Bartlett's test of sphericity and Kaiser-Meyer-Olkin measurement of sampling adequacy were used to determine whether the requirements for a factor analysis were met. The number of factors to consider were determined by considering the eigenvalues (>1), scree plot, and interpretability of the factor. Items that crossed loaded on more than one factor were placed with the factor that was most closely related conceptually. Items with poor factor loadings (<0.6) were considered problematic. In addition, a structural equation model with maximum likelihood was used to evaluate the exploratory factor analysis model fit by using the standard fit indices: root-mean-square error of approximation (≤0.06),12 standardized root-mean-square residual (≤0.08),12 comparative fit index (≥0.90–0.95),12 Normed Fit Index (NFI) (≥0.90–0.95),12 Tucker-Lewis index (TLI) (≥0.90–0.95),12 and the χ2/df ratio less than 3 (P < 0.05).13, 14 The internal consistency was assessed using item-total correlations and Cronbach's alpha. Item-total correlations assess the overall correlation between items within a scale and should be ≥0.4. A Cronbach's alpha of ≥0.70 was considered acceptable for a scale to be sufficiently reliable. Pearson correlation coefficients (r) were calculated to assess whether each item was in the right subscale by correlating items with the subscale means. Items that correlated more highly on subscales other than the one to which it was assigned were eliminated. The P values <0.05 were considered statistically significant. All statistical analyses were done using SPSS version 25.0 (IBM SPSS Statistics, 2015) and AMOS statistical package version 21.

Ethics

Research ethics approval was not needed because the study was considered noninterventional according to the Dutch Medical Research and Human Subjects Act (WMO).

Results

Participant Characteristics

A total of 371 healthcare providers (62% response rate) completed the online survey questionnaire. The majority of the participants came from the United States (28.4%, n = 12), Canada (6.9%, n = 29), France (6.9%, n = 29), and Germany (6.9%, n = 29) (see also Fig. 2). In addition, the majority of the participating healthcare providers were represented by neurologists (38.8%, n = 164), followed by nurses (9.5%, n = 40) and physical therapists (8.7%, n = 37). The job characteristics of the participants are listed in Table 1.

Details are in the caption following the image
Geographical representation of participating study respondents. [Color figure can be viewed at wileyonlinelibrary.com]
TABLE 1. Job characteristics of study participants
Job position Frequency (n) Percent (%)
Medical disciplines
Neurologist 164 38.8
Neurosurgeon 13 3.1
Gerontologist/geriatrician 6 1.4
Allied health disciplines
Nurse/Parkinson's disease nurse 40 9.5
Nurse practitioner/physician assistant 10 2.4
Physical therapist 37 8.7
Occupational therapist 14 3.3
Speech therapist 20 4.7
Social worker 10 2.3
Clinical psychologist/Neuropsychologist 14 3.3
Rehabilitation specialist 6 1.4
Unknown disciplines
Other 33 7.8

Exploratory Factor Analysis

The Kaiser-Meyer-Olkin test of sample adequacy (0.845) and Bartlett's test of sphericity (P < 0.0001) indicated that a factor analysis for the RMIC-MT provider version was appropriate. A nine-factor solution was obtained with a total of 48 items, which explained 55.3% of the variance. The number of factors was decided based on eigenvalues, percentage of variance, and theoretical consideration (see also Supporting Information Fig. S1). The factors are professional coordination, cultural competence, triple aims outcome, system coordination, clinical coordination, technical competence, community-centeredness, person-centeredness, and organizational coordination. Some items (eg, 48. Collaboration procedures; 42. Staffing; 16. Medical continuity; 40. Outcome transparency; and 1. Interpersonal trust) were discarded from further analysis because of low factor loading and cross-loading. Besides, some items (eg, 12. Shared decision-making; 46. Safety; and 36. Electonic Health Records) were discarded from further analysis because they did not load on their respective factors (see Table 2).

TABLE 2. Factor analysis RMIC-MT provider version
Item PRC CUC TA SC CLC TC CC PC OC
19. Interdisciplinary coordination 0.824
20. Information exchange 0.788
21. Interdisciplinary collaboration 0.782
22. Interdisciplinary follow-up 0.767
18. Interdisciplinary fragmentation 0.761
23. Interdisciplinary teamwork 0.748
17. Interdisciplinary communication 0.551
44. Respect 0.849
41. Fellowship 0.830
43. Teamwork 0.807
45. Support 0.803
47. Learning 0.599
48. Collaboration procedures 0.418 0.359
42. Staffing 0.353
31. Needs assessment 0.825
32. Experience assessment 0.769
35. Outcome assessment 0.684
34. Monitoring & follow-up 0.681
33. Quality objectives 0.681
12. Shared decision-making 0.345 0.325
30. Care coordination incentives 0.935
28. Interorganizational incentives 0.879
29. Interdisciplinary incentives 0.870
10. Case management 0.852
15. Multidisciplinary team 0.680
11. Follow-up of care 0.582
14. Quality procedures 0.531
13. Shared care plans 0.513
16. Medical continuity 0.447
46. Safety 0.444
37. Interoperable IT tools 0.873
39. Data integration 0.761
38. Interoperable EHRs 0.679
40. Outcome transparency 0.450
36. EHRs 0.383
8. Community collaboration 0.829
7. Health promotion 0.761
6. Community partnership 0.722
9. Population needs 0.693
4. Preference integration 0.713
5. Questioning 0.659
2. Listening 0.635
3. Social circumstances 0.602
1. Interpersonal trust 0.380
25. Interorganizational coordination 0.780
26. Interorganizational resources 0.709
27. Interorganizational staff 0.585
24. Interorganizational objectives 0.552
Eigenvalues 11.083 3.757 2.891 2.38 1.887 1.421 1.123 1.039 0.943
% of variance 23.089 7.828 6.023 4.958 3.931 2.961 2.34 2.165 1.965
  • Note: Extraction method: principal axis factoring. Rotation method: Promax with Kaiser normalization.
  • Abbreviations: RMIC-MT, Rainbow Model of Integrated Care Measurement Tool; PRC, professional coordination; CUC, cultural competence; TA, triple aim outcomes; SC, system coordination; CLC, clinical coordination; TC, technical competence; CC, community-centeredness; PC, person-centeredness; OC, organizational coordination; EHR, electronic health record.

Confirmatory Factor Analysis

Results showed that all the items were loaded on their respective factors. Acceptable model fit (nine factors, 40 items) measures were obtained, including χ2/df = 1.810, comparative fit index = 0.910, TLI = 0.896, NFI = 0.821, root-mean-square error of approximation = 0.044, and standardized root-mean-square residual = 0.066. Given that four of six indices met the required thresholds, despite the limited sample size, the exploratory factor model with nine factors was deemed an acceptable fit (Fig. 3 and see Supporting Information Table S5 and Fig. S3).

Details are in the caption following the image
Confirmatory factor analysis Rainbow Model of Integrated Care Measurement Tool (RMIC-MT). [Color figure can be viewed at wileyonlinelibrary.com]

Internal Consistency

Results of internal consistency indicated that the RMIC-MT provider version's reliability assumptions were met for all nine scales. All item-total correlations were greater than 0.4, as shown in Supporting Information Table S2. Cronbach's alpha values ranged from 0.936 for system coordination to 0.763 for clinical coordination and were thus acceptable (>0.70).

Construct Validity

Construct validity results showed that all the subscales of the RMIC-MT provider version were significantly and positively associated with each other at P < 0.01 and P < 0.05 level except for the relationship between person-centeredness and system coordination. The results suggest that construct validity of the subscale of the RMIC-MT provider version has been established (see Supporting Information Table S3).

Item Score Distribution

The mean, median, mode, standard deviation, floor and ceiling impact, skewness, and kurtosis were all used as descriptive statistics for the study scales. The RMIC-MT provider versions had no extreme floor, ceiling, skewness, or kurtosis values except for person-centeredness, indicating that the items' psychometric sensitivity was sufficient. The overview measurements scales in the RMIC-MT provider version are shown in Table S4.

Discussion

The purpose of this study was to validate the RMIC-MT provider version tool as a measure of care integration in PD care settings. Prior research has identified multiple care models that deliver integrated care for PD across outpatient, inpatient, and community settings.1 Several of these models improved the quality of life of people with PD or other key outcomes when tested in controlled studies.15-17 A recent meta-analysis suggested that outpatient-based integrated care resulted in modest yet significant improvements in quality of life in people with PD.3 However, the lack of an objective measure to quantify care integration limited the ability to compare and combine data from heterogeneous interventions that were reported under the umbrella term “integrated care.” The RMIC has a priori defined key components of integrated care in different disease settings but had not been studied in PD.

Comparison With Existing Evidence

The RMIC-MT provider version measurement tool identified 40 items (nine subscales) that were relevant in PD care centers as based on our survey results. This result corresponds to the nine dimensions of the original RMIC provider version: person-centeredness, community-centeredness, clinical coordination, professional coordination, organizational coordination, systems coordination, technical competence, cultural competence, and triple aims outcomes. Some original scale items were excluded from additional analysis in this study because they did not load adequately to the corresponding subscale in our sample. Important among these were shared decision-making, safety, and electronic health records. This may partly be because of the diversity of healthcare systems that we surveyed where decision-making processes may not be uniform and electronic health records may not be universally available. Nevertheless, given the broad coverage of our survey across diverse health systems, the results were generally consistent with those obtained for validation of the original RMIC provider version tool.2, 5, 8 Most of the variance in the RMIC provider version was explained by professional coordination subscale. Consistent with prior studies, this shows that most of the care integration revolves around multidisciplinary professional teams delivering PD care,3 whereas other aspects of care integration, such as organizational and systems integration, remain relatively unaddressed by the centers.5, 8-10 Person-centeredness, which is a core concept at the center of integrated care, also contributed little to the variance. The modified tool showed good internal consistency. All the subscales showed good construct validity, except the relationship between person-centeredness and laws and regulation, and overall construct validity was within acceptable limits.

Strengths and Limitations

The robust methodology used is a strength of this study, resulting in a reliable and valid PD-specific RMIC tool for healthcare providers. The sampling strategy that included centers with varying levels of movement disorder expertise distributed worldwide increases the external validity and applicability of our results. Moreover, this study builds on the existing RMIC tool used in other chronic conditions. Finally, the RMIC-MT provider version was easy to use for healthcare professionals involved in PD care. There are also a number of limitations for this study. One limitation of this study is that the target sample size could not be achieved (responder rate 62%), which limited the power of our analysis, and hence further comparisons between subscales and domains were not attempted. A second limitation of this study is that the Confirmatory Factor Analysis model did not fully meet the criteria for an acceptable fit based on two of the six fit indices used. Specifically, the NFI and TLI indices did not meet the desired levels of fit, despite the fact that the thresholds for the other indices were met. One possible explanation for these findings is the small sample size. A smaller sample size can lead to less precise estimates of the model's parameters and can increase the likelihood of obtaining a poor fit.12, 18 Another limitation is that the results found in this study could be possibly influenced by people who were more positively biased toward integrated care. Moreover, this study could be subject to selection bias. The cross-section study design, nonrandom sampling method, and limited sample size limit the generalizability of the results found in this study. This should be further investigated in further research. Moreover, we addressed only the provider version of the RMIC tool, but to better understand the care integration practices from a patient's perspective, the patient version should obviously also be used concomitantly. However, addressing this patient perspective was beyond the scope of this study. Despite these limitations, our results suggest that the RMIC-MT provider version can be reliably used in the PD setting to measure integrated care practices.

Conclusion

This study provides evidence for the construct validity and other psychometric properties of the PD-specific RMIC-MT provider version to measure integrated care in PD. Future research is warranted to understand whether greater levels of integration translate to better patient-relevant outcomes, for instance, patient satisfaction with care, quality-of-life, or care quality indices. This study is a first step in this direction. The tool may also be used to compare the levels of integration of different care models, as a whole and in specific subdomains, to identify strengths and weaknesses of the system. The tool may also be incorporated in future studies assessing new care coordination interventions in PD, to assess their effects on integration of care objectively.

Acknowledgments

This work was supported by a research grant from International Parkinson Disease and Movement Disorder Society. We appreciate Allie Oakley and Liza Tymchenko for their invaluable support with the data collection and analysis. Their expertise and attention to detail played a crucial role in the successful completion of this study.

    Financial Disclosures

    The authors declare that they have no relevant financial interests.

    Author Roles

    Study concept and design: P.P.V., C.E., and R.R.; acquisition, analysis, or interpretation of the data: P.P.V., C.E., R.R., B.R.B., D.G., J.G, J.M., J.G., M.E.P.P., N.D., L.B., R.I., N.K., S.P., and A.H.; drafting of the manuscript: P.P.V.; critical revision of the manuscript for important intellectual content: P.P.V., C.E., B.R.B., D.G., J.G., J.M., J.G., M.E.P.P., N.D., L.B., R.I., N.K., S.P., A.H., and R.R.; statistical analysis: P.P.V.; study supervision: P.P.V., C.E., and R.R. All authors read and approved the final manuscript. P.P.V. had full access to all data in the study and took responsibility for the integrity of the data and the accuracy of the data analysis.

    Data Availability Statement

    The data that support the findings of this study are available from the corresponding author upon reasonable request.