Patient-reported outcomes in coronary artery disease: the relationship between the standard, disease-specific set by the International Consortium for Health Outcomes Measurement (ICHOM) and the generic health-related quality of life instrument 15D

Abstract

Background

Patient-reported outcome (PRO) instruments measure health gains, including changes in health-related quality of life (HRQoL). Previous studies have assessed the reliability and relationship of multiple HRQoL instruments in search of the optimal instrument for feasible measurement of PROs. Although the 15D instrument was shown to have the best sensitivity and construct validity among cardiac patients, it is unknown how well it captures relevant disease-specific information scores compared to instruments included in the International Consortium for Health Outcomes Measurement (ICHOM) standard set. The aim of this study was to investigate whether the disease-specific PRO instruments and a generic HRQoL instrument capture disease related symptoms in coronary artery disease (CAD) patients.

Methods

Health status and HRQoL were assessed with the instruments included in the ICHOM standard set: Seattle Angina Questionnaire short-form (SAQ-7), Rose Dyspnea Scale (RDS), two-item Patient Health Questionnaire (PHQ-2), and with the 15D HRQoL instrument at baseline and 1 year from the treatment in a university hospital setting. Spearman correlation and explanatory factor analysis were used to assess the relationship of baseline scores and 1-year change in scores of 297 patients.

Results

At baseline, the overall 15D score and SAQ-physical limitation (SAQ-PL), 15D “breathing” and SAQ-PL, as well as “breathing” and RDS showed moderately strong correlations. The factor interpreted to reflect “Breathing-related physical activity”, based on high loadings of “breathing”, RDS, SAQ-PL, “mobility”, “vitality”, and “usual activities”, explained 19.2% of the total variance. Correlations between 1-year changes in scores were fair. The factor of “Breathing-related physical activity”, with significant loading of RDS, SAQ-PL, “breathing, “usual activities”, “vitality”, “sexual activity”, “mobility”, and disease-specific quality of life explained 20.5% of the total variance in 1-year change in scores. The correlation of angina frequency measured by SAQ-7 and the 15D instrument was poor.

Conclusions

The 15D detects dyspnea and depression similarly to RDS and PHQ-2 but not angina similarly to the SAQ-7. This may call for supplementing the 15D instrument with a disease-specific instrument when studying CAD patients.

Background

Validated patient-reported outcome (PRO) instruments measure health gains, including changes in health-related quality of life (HRQoL), perceived by patients after treatment [1]. Responding to health status assessment questionnaires should be as simple as possible for the respondents. Thus, long, or numerous questionnaires tend to impair response rates [1]. From a healthcare provider´s point of view, a PRO instrument should capture the health status change diversely with minimal resources needed for collection and analysis of the PROs.

In search of the optimal instruments for feasible measurement of PROs in routine care of cardiac patients, previous studies combined multiple HRQoL instruments and assessed their reliability and relationship [2,3,4,5,6]. Several studies have included preference-based generic instruments such as the 15D [7], Assessment quality of life instrument (AQoL), five-dimensional EuroQol instrument (EQ-5D), Health utilities index mark three instrument (HUI3), and short-form six dimension instrument (SF-6D) [4, 8,9,10,11]. Of these generic HRQoL instruments, the 15D has demonstrated to have the best sensitivity and construct validity among cardiac patients [4, 10, 11]. However, as 15D is a generic HRQoL instrument, it is unknown whether it can capture the variation in the disease-specific measures in patients with coronary artery disease (CAD).

The International Consortium for Health Outcomes Measurement (ICHOM) CAD working group recommends measurement with a standard set for CAD [12]. The set includes the following instruments: Seattle Angina Questionnaire short-form (SAQ-7) [13] for assessing functional status, angina and disease-specific HRQoL, Rose Dyspnea Scale (RDS) [14] for assessing dyspnea, and the two-item Patient Health Questionnaire (PHQ-2) [15, 16] for evaluating depressive symptoms. The measurement is recommended to be performed at baseline, and one month and 1 year from the treatment. However, only a few studies utilizing the ICHOM standard set of instruments have so far been published [17, 18]. Furthermore, although the standard set includes assessment of disease-specific HRQoL with the SAQ-7, it does not include measurement of generic HRQoL and thus excludes the possibility to compare HRQoL outcomes obtained in CAD to those obtained in other diseases or the general population, and the calculation of quality-adjusted life-years (QALYs).

We investigated how well the generic HRQoL, 15D, captures disease-specific quality of life and symptoms measured by the instruments included in the ICHOM standard set in a routine setting. The analyses were performed using baseline scores and the changes in scores during 1-year follow-up.

Methods

The patient recruitment

A total of 397 (Additional file 1: Table S1) CAD patients scheduled for elective index angiography or elective coronary artery bypass grafting (CABG) at the Heart Centre of the Kuopio University Hospital (KUH) between July 2017 and May 2018 self-assessed their health status and HRQoL with the ICHOM standard set of instruments (SAQ-7, RDS, PHQ-2), and the 15D HRQoL instrument before the treatment and at 1 year after treatment. All questionnaires were administered in paper form and the CAD diagnosis was based on a previous CAD diagnosis and findings in the baseline coronary angiography. This analysis was restricted to those 279 (70.3%) patients for whom 1-year change with all four instruments could be calculated. Based on intention to treat, 100 (35.8%) patients received optimal medical therapy (OMT), 155 (55.6%) underwent percutaneous coronary intervention (PCI), and 24 (8.6%) CABG. A total of 118 patients were excluded from the analysis, 69 patients (n = 39 at baseline, n = 30 at 1-year) due to ≥ 1 missing instrument scores, and 49 patients due to non-response at 1-year follow-up (8 of them had died).

Assessment of health status and health-related quality of life

The seven-item SAQ-7 measures physical limitation (SAQ-PL), angina frequency (SAQ-AF), and disease-specific HRQoL (SAQ-QL) and has a four-week recall period. The SAQ-7 generates a summary score (scale 0–100, 100 = full health, 0 = worst health). The SAQ-AF corresponds to two questions and categorizes angina frequency as following: daily angina (score = 0‒30), weekly angina (score = 31–60), monthly angina (score = 61‒99), and no angina (score = 100). The SAQ-PL corresponds to three questions and SAQ-QL corresponds to two questions. If all three domain scores are missing, the SAQ-7 summary score is not calculated. According to prior work, a change of 5‒8 points in the summary score is considered clinically important [13].

The four-item symptom-specific RDS measures dyspnea level during activity (scale 0‒4, 0 = no dyspnea, 4 = severe limitation of physical activity due to dyspnea) and has a four-week recall period. A one-point change in the RDS score is considered clinically important [14].

The two-item PHQ-2 screens for depressive symptoms during a 2-week recall period and generates a summary score (scale 0‒6, 0 = no depressive symptoms, 6 = severe depressive symptoms). A PHQ-2 score of two or more points indicates depressive symptoms in CAD patients [19].

The generic HRQoL instrument 15D measures fifteen dimensions of health: “mobility”, “vision”, “hearing”,”breathing”, “sleeping”, “eating”, “speech”, “excretion”, “usual activities”, “mental function”, “discomfort and symptoms”, “depression”, “distress”, “vitality”, and “sexual activity” [7]. Each dimension question has five response options describing the present health of the patient. The single index score (15D score), representing the overall HRQoL on a 0–1 scale (1 = full health, 0 = being dead) and the dimension level values, reflecting the goodness of the levels relative to no problems on the dimension (= 1), and to being dead (= 0), are calculated from the health state descriptive system (questionnaire) by using a set of population-based preference or utility weights. Based on age, gender, and other patients’ responses, one to three missing 15D answers can be imputed using regression analysis [20]. A positive change of > 0.015 in the overall 15D score indicates a clinically important improvement [21].

Statistical analysis

Statistical analysis was carried out by using the IBM SPSS statistical software (IBM SPSS, Inc., Chicago, IL, USA, version 25). The results are given as mean (standard deviation, SD), mean (95% confidence interval, CI), or percentages. The distribution of scores at the floor (worst possible scores) and the ceiling (best possible scores) of the instrument scales was explored. According to a previous work, floor, or ceiling effects of < 15% are considered acceptable in health status questionnaires [22]. High proportions of floor and ceiling scores prior to treatment may complicate the assessment of health benefit. Changes in instruments domain and total scores between the baseline and the 1-year follow-up measurement were examined with linear mixed model adjusting for the baseline value. To investigate whether the effect of a baseline score on the change in score differed between the treatment groups, a model with baseline scores and treatment group interaction was fitted. Statistically significant interactions are reported in the results. P-values < 0.05 were considered statistically significant.

To evaluate whether the 15D provides information included in the ICHOM standard set, we explored nonparametric Spearman correlations, suitable for ordinal and nonnormal data, between disease-specific instrument scores and SAQ-7 domain scores and the 15D at baseline, as well as the correlation of change in scores during 1-year follow-up. The opposite scales of RDS and PHQ-2 were reverse coded (by multiplying the scores by − 1) for the correlation analysis. Correlation coefficient values r < 0.3 were considered poor, values 0.3 ≤ r < 0.6 fair, values 0.6 ≤ r < 0.8 moderately strong, and values r ≥ 0.8 very strong [23]. Spearman correlation assumes a monotonic relationship between the investigated variables. Investigation of pairwise scatterplots did not implicate nonmonotonic relationship.

The baseline 15D dimension values, SAQ-PL, SAQ-AF, SAQ-QL, RDS, and PHQ-2 scores were included in an explanatory factor analysis to explore, to what extent there is common variability among these variables, and whether the interrelationships between the variables can be presented in a condensed way as fewer interpretable, underlying, or latent variables, i.e., factors. Similarly, the 1-year change in these observed variables were included in an explanatory factor analysis. First, principal components with an eigenvalue > 1 were extracted and then Varimax–rotated. In the interpretation of factors, attention was paid to loadings > 0.5, and especially to the highest loadings.

Ethical considerations

The study was approved by the Research Ethics Committee of the Northern Savo Hospital District and registered with trial number 5101114. All study participants gave written consent, and decision to participate in this study did not affect their treatment.

Results

The baseline characteristics of the 279 included study participants are shown in Additional file 1: Table S1. The proportion of PCI treated respondents was significantly higher in the group of respondents included in the study compared to those excluded.

SAQ-7, RDS, PHQ-2 and the generic 15D at baseline and at 1-year follow-up

The mean baseline instrument scores, mean 1-year changes, and the proportions of floor and ceiling scores at baseline and at 1-year follow-up for each instrument are presented in Table 1.

Table 1 Mean scores, mean change, and estimated 1-year follow-up changes, and floor and ceiling scores

During the 1-year follow-up, the mean change in SAQ-7 was 16.4 (CI 14.0–18.9), and the estimated changes in the treatment groups were 12.5 (CI 9.1–15.9) for OMT, 16.9 (14.2–19.6) for PCI, and 28.0 (CI 21.1–35.0) for CABG. The mean changes in the domain scores were 5.1 (CI 2.6–7.6) for SAQ-PL, 17.8 (CI 14.9–20.7) for SAQ-AF, and 25.7 (CI 21.8–29.6) for SAQ-QL. The estimated mean changes in the domain scores within treatment groups are presented in Table 1. The proportion of ceiling scores of SAQ-7 increased from 6.5% at baseline to 22.6% at 1 year. The proportion of ceiling domain scores increased significantly from 25.8% (n = 66) at baseline to 37.1% (n = 92) at 1 year for SAQ-PL, 13.7% (n = 38) at baseline to 44.8% (n = 125) at 1 year for SAQ-AF, and from 9.1% (n = 25) at baseline to 38.5% (n = 107) for SAQ-QL.

The 1-year mean change in RDS score was 0.5 (CI 0.4–0.6) and estimated changes in the treatment groups were 0.2 (CI 0.1–0.4) for OMT, 0.6 (CI 0.4–0.7) for PCI and 1.0 (0.7–1.4) for CABG. The proportion of patients without dyspnea (ceiling scores) nearly doubled from 15.1% (n = 42) at baseline to 29.7% (n = 83) at 1 year.

The mean change in PHQ-2 score was 0.1 (CI 0.0–0.3) during the 1-year follow-up. The estimated mean change at 1-year was 0.1 (CI − 0.2 to 0.2) for OMT, 0.1 (CI 0.0–0.3) for PCI, and 0.5 (CI 0.0–0.9) for CABG. The proportion of ceiling PHQ-2 scores showed slight, but not significant, change from 51.3% (n = 143) at baseline to 59.9% (n = 167) at 1 year.

The 1-year change in mean overall 15D score was 0.024 (CI 0.016–0.032) and the estimated mean changes were 0.005 (CI − 0.009 to 0.017) for OMT, 0.028 (CI 0.019–0.039) for PCI, and 0.074 (CI 0.051–0.104) for CABG. The association between baseline scores and 1-year change in the 15D dimensions “eating”, “speech”, “discomfort and symptoms” and “sexual activity” were different in the treatment groups (p for baseline dimension and treatment group interaction < 0.05). The estimated differences within the groups are shown at Table 1.

The proportion of overall 15D scores at the ceiling did not differ between baseline (1.1%) and 1-year follow-up (1.1%). However, the proportion of 15D dimension values at the ceiling changed as follows from baseline to 1-year: “mobility” from 42.7 to 53.8%, “breathing” from 21.5 to 39.4%, “usual activities” from 34.8 to 41.6%, “discomfort and symptoms” from 17.6 to 27.6%, “distress” from 57.0 to 64.5%, “vitality” from 15.1 to 27.2%, and”sexual activity” from 33.0 to 36.6%.

Correlation between the instrument scores of SAQ-7, RDS, PHQ-2 and the 15D score and 15D dimension values at baseline and at 1-year follow-up

Correlation coefficients of the 15D instrument scores and the ICHOM standard set instruments at baseline are presented in Fig. 1 and Additional file 1: Table S2 and at 1-year follow-up in Additional file 1: Table S2. At baseline, the overall 15D score and SAQ-PL (r = 0.69), the 15D dimension value of “breathing” and SAQ-PL (r = 0.61), the 15D dimension value of “usual activities” and SAQ-PL (r = 0.60), as well as the 15D dimension value of “breathing” and RDS (r = 0.66) showed moderately strong correlation (0.6 ≤ r < 0.8). The other correlations were fair (0.3 ≤ r < 0.6) or poor (r < 0.3). At 1-year follow-up, correlations were fairly similar to those observed at baseline (Additional file 1: Table S2).

Fig. 1

Spearman correlations between the overall 15D score and dimension values and the Seattle Angina Questionnaire short-form (SAQ-7), SAQ Physical Limitation (SAQ-PL), SAQ Angina Frequency (SAQ-AF), SAQ Quality of Life (SAQ-QL), Rose Dyspnea Scale (RDS) and two-item Patient Health Questionnaire (PHQ-2) at baseline

The factor analysis of baseline scores identified five factors (Table 2) that explained 61.0% of the total variance. The 15D dimension of “breathing” followed by RDS score, SAQ-PL, and the dimensions of “mobility”, “vitality”, and “usual activities” had high loadings on Factor 1. Based on the variables loading highly onto Factor 1, it could be interpreted to reflect “Breathing-related physical activity”, explaining 19.2% of the total variance.

Table 2 Rotated component matrix of the 15D dimension values and instrument scores

The 15D dimension of “distress” followed by “depression”, PHQ-2 score, and the 15D dimension of “mental functions” had high loadings on Factor 2. For this, Factor 2 could be interpreted to reflect “Mental health”, explaining 15.0% of the total variance.

The variables SAQ-AF and SAQ-QL had high loadings on Factor 3. Based on these variables, Factor 3 could be interpreted to reflect disease-specific, i.e., “Angina-related quality of life”, explaining 9.3% of the total variance. Factors 4 and 5 were 15D-specific factors, reflecting health problems CAD patients may have, be they CAD-related or not.

Correlations between the 1-year changes in the scores of the 15D variables and those of the ICHOM standard set were only fair at best (Table 3). The highest correlations (r = 0.40‒0.42) were observed between the overall 15D score and the RDS, PHQ-2, SAQ-7, SAQ-PL, and SAQ-QL.

Table 3 Spearman correlations between 1-year changes in the 15D variables and instrument scores

The factor analysis based on these change variables identified 7 factors (Table 4) that explained 58.2% of the total variance. The interpretation of Factors 1, explaining 20.5% of the total variance, and Factor 2, explaining 7.5% of the total variance, is similar to Factors 1 (“Breathing-related physical activity”) and 2 (“Mental health”) based on baseline scores.

Table 4 Rotated component matrix of the 1-year changes in 15D dimension values and instrument scores

The rest of the factors were mainly 15D-specific, but quite difficult to interpret, although again they seem to reflect health problems CAD patient may have, be they CAD-related or not.

Discussion

Our study is the first to explore the correlation of scores and dimension values generated by the generic HRQoL instrument 15D with the scores of instruments included in the ICHOM standard set for treatment outcome measurement in CAD. It demonstrated improvement in all four instrument scores during 1-year follow-up.

The 15D dimension value of “breathing” and the RDS score showed moderately strong correlation at baseline. Consistently, in the factor analysis of baseline scores, the 15D dimension of “breathing” and the RDS score had strongest loadings to factor 1 reflecting “Breathing-related physical activity”. Consistent with our results, Mazur et al. previously demonstrated strong correlation between the 15D and dyspnea assessed with the disease-specific Airways questionnaire 20 in COPD patients [24].

Previously, the 15D dimension of “depression” demonstrated strong to very strong correlation with the Beck Depression Inventory in patients with depressive disorders, both at baseline and at 5-year follow-up [25]. Correspondingly, in our study the factor analysis of baseline scores revealed significant loadings of the 15D dimension of “depression”, and the PHQ-2 score on Factor 2 named “Mental health”. Furthermore, factor analysis of 1-year changes in scores demonstrated the importance of the mental health factor in CAD, as the 1-year change in “distress”, “depression” and the PHQ-2 score were significantly loaded on Factor 2.

Anxiety is recognized as a comorbidity in CAD [26, 27], and was recently, in a large study, found to predict cardiac readmission [28]. Unlike any of the instruments included in the ICHOM standard set, the 15D instrument measures anxiety (dimension of “distress”) in addition to “depression”. We found fair correlation between the baseline 15D dimension value of “distress” and the PHQ-2 score. Moreover, the importance of “distress” was supported by significant loadings on the factor of “Mental health” both at baseline and on factor based on 1-year change in scores.

Our study found fair correlation between the overall 15D score and the SAQ-PL at baseline, and furthermore, the factor analysis of baseline values showed that the 15D dimensions reflecting physical health together with “breathing” and the SAQ-PL score loaded highly on the same factor named “Breathing-related physical activity”.

The factor analysis based on 1-year changes in scores revealed that the change in the disease-specific quality of life measured with SAQ-QL had moved from the baseline factor of “Angina-related quality of life” to the change factor of “Breathing-related physical activity”. Even though conclusions should not be made based solely on the explanatory factor analysis of a rather small data, this may indicate that a variation in the change of generic captures the variation in the changes in breathing and physical symptoms more strongly than it captures the variation in change of anginal symptoms.

However, electively treated CAD patients may have adapted their physical activity level prior to treatment to avoid anginal symptoms, and thus scored better in SAQ-AF at baseline. Consequently, they may have perceived treatment benefit mainly as improved physical activity during the follow-up. The substantially larger proportion of ceiling SAQ-AF and RDS scores observed at 1-year follow-up may also reflect health gain. It may also be explained by the fact that only respondents with instrument scores at baseline and at 1-year follow-up were included in the study and thus, healthier respondents may be represented.

Previous work has demonstrated that the four-week recall period of SAQ-7 is reliable compared with daily self-reporting of angina [29]. Although 15D is a validated instrument in patients with chronic pain [8, 30], it did not correlate, or correlated only poorly, with the disease-specific angina pain frequency measured with the SAQ-AF. Moreover, the factor analysis based on baseline scores and 1-year change scores confirmed that the SAQ-AF and the 15D dimension of “discomfort and symptoms” did not load on the same factor.

This lack of correlation might be explained by the fact that the 15D records present symptoms. Thus, the absence of anginal symptoms at the time of responding to the 15D questionnaire, may explain the modest correlation between the “discomfort and symptoms” dimension and the SAQ-AF score. Additionally, the 15D dimension “discomfort and symptoms” is not limited solely to pain, as it includes other types of physical discomfort and symptoms such as itching and nausea. Consequently, the dimension is not directly comparable with the SAQ-AF that measures anginal frequency. However, considering the importance of capturing anginal symptoms in CAD, this may call for combining this disease-specific variable to a generic HRQoL instrument, like the 15D, in CAD patients.

To achieve better reliability in the correlation analysis, the study was limited to those who responded to all four questionnaires at both baseline and at 1-year follow-up. It is possible that those with worse health may not have responded to all four questionnaires at 1 year which is a limitation of the study [31].

To the best of our knowledge, this is the first study to compare the performance of the 15D with the instruments included in the ICHOM standard set in CAD patients who have received OMT or undergone PCI, or CABG in a routine care setting. Another strength of our study is the utilization of the 15D instrument to measure generic HRQoL as it has been found to have higher discriminatory power and better validity in the disease area of heart disease than some other generic instruments [10, 11, 32, 33].

Conclusions

The 15D instrument partially captured dyspnea, physical limitation, and depression measured by the instruments included in the ICHOM standard set for CAD. Still, as implied by the modest to moderately strong correlations, the SAQ-7, RDS and PHQ-2 capture slightly different information than the 15D. However, the 15D dimension of “discomfort and symptoms” showed only modest correlation with angina frequency measured by the SAQ-AF, which indicates that to detect angina, the 15D instrument should be supplemented with a disease-specific instrument.