Psychometric evaluation of a Canadian version of the Seattle Angina Questionnaire (SAQ-CAN)

Abstract

Background

The Seattle Angina Questionnaire (SAQ) is a widely-used patient-reported outcomes measure in patients with heart disease. This study assesses the validity and reliability of the SAQ in a Canadian cohort of individuals with stable angina.

Methods and results

Data are from the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease (APPROACH) registry, a population-based registry of patients who received cardiac catheterization in Alberta, Canada. The cohort consists of 4052 patients undergoing cardiac catheterization for stable angina and completed the SAQ within 2 weeks. Exploratory factor analysis and confirmatory factor analysis (CFA) were used to assess the factorial structure of the SAQ. Internal and test–retest reliabilities of a new measure (i.e., SAQ-CAN) was measured using Cronbach α and intraclass correlation coefficient, respectively. CFA model fit was assessed using the root mean square error of approximation (RMSEA) and comparative fit index (CFI). Construct validity of the SAQ-CAN was assessed in relation to Hospital Anxiety and Depression Scales (HADS), Euro Quality of life 5 dimension (EQ5D), and original SAQ. Of the 4052 patients included in this analysis, 3281 (80.97%) were younger than 75 years old, while 3239 (79.94%) were male. Both exploratory and confirmatory factor analyses revealed a four-factorial structure consisting of 16 items that provided a better fit to the data (RMSEA = 0.049 [90% CI = (0.047, 0.052)]; CFI = 0.975). The 16-item SAQ demonstrated good to excellent internal reliability (Cronbach’s α range from 0.77 to 0.90), moderate to strong correlation with the Original SAQ and EQ5D but negligible correlations with HADS.

Conclusion

The SAQ-CAN has acceptable psychometric properties that are comparable to the original SAQ. We recommend its use for assessing coronary health outcomes in Canadian patients with Coronary Artery Disease.

Introduction

Heart disease, the second-leading cause of death in Canada, affects up to 8.5% [1] of adult Canadians and accounts for an annual estimated cost of $21.2 billion [2]. In addition to conferring increased risks of premature mortality and major non-fatal morbidity, chronic heart disease leads to significant ongoing symptoms and associated impairment in functional status and health-related quality of life (HRQOL) [3]. Professional societies, such as the American Heart Association, have advocated integrating patients’ perspectives of their health status as a key cardiovascular health outcome that should be used in clinical trials of new interventions, observational studies, and routine clinical practice [4]. Consequently, the assessment of the HRQOL as a primary or secondary outcome in clinical trials and observations studies of patients with coronary artery disease (CAD) has continued to grow [4,5,6].

Several patient-reported outcome measures have been specifically developed for measuring symptoms burden, functional status, and quality of life in people with CAD. The Seattle Angina Questionnaire (SAQ), a widely used disease-specific measure of quality of life in patients with heart disease, is a 19-item self-administered questionnaire that measures 5 dimensions of HRQOL [7]. Originally developed in a population of US veterans, SAQ has been translated into more than 52 languages [8] and adapted for use in several countries. While a number of studies have investigated the construct validity and reliability of the SAQ, only a few studies have examined its factorial validity. In those studies, the original factorial structure of the SAQ was not tenable. For example, Kimble et al.’s validation of the SAQ in a population of predominantly female sample of stable angina patients showed the emergence of new subscales (e.g., division of the physical limitation subscale into two separate factors) and misfit of one of the SAQ items [9]. Similarly, the translation and validation of the Farsi version of the SAQ yielded a five-factor solution with subscales that were not identical to the original SAQ subscales [10]. Garrath et al. examined the psychometric properties of the SAQ in a United Kingdom sample of stable angina patients and found that the original factorial structure of the SAQ resulted in the removal of 4 items, resulting in the emergence of the 15-item United Kingdom version of the SAQ with 3 subscales [11].

Despite its wide use, SAQ has not been previously validated in a Canadian sample of individuals with heart disease. The study investigated the measurement properties of the SAQ in Canadian patients by assessing the validity and reliability of the SAQ in a Canadian cohort of stable angina patients.

Methods

Data source

De-identified secondary data were obtained from the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease (APPROACH) registry [12, 13], a population-based registry of all patients, who had cardiac catheterization in the province of Alberta, Canada. The APPROACH registry contains detailed demographics and clinical information. Individuals in the registry are followed longitudinally after catheterization for assessment of subsequent procedures and patient-reported health status from those who consent to follow up. The study cohort consisted of adults (≥ 18 years) patients with coronary artery disease who underwent cardiac catheterization for stable angina from January 1, 2003, and December 31, 2016, and completed the SAQ 2 weeks after cardiac catheterization. Data collected at catheterization included demographic characteristics (sex, age, address), clinical comorbidities, disease severity measures, and coronary angiography results. Participants also completed several patient-reported outcome measures (PROMs), including the Seattle Angina Questionnaire (SAQ), EuroQol-5-Dimension, Hospital Anxiety and Depression scale (HADS).

Measures

Seattle Angina Questionnaire

The SAQ is a 19-item self-administered questionnaire that measures 5 dimensions of HRQOL for CAD assessed over the past 4 weeks. These include physical limitation (9 Items), angina stability (1 item), angina frequency (2 items) treatment satisfaction (4 items), and disease perception (3 items). The items are scored on a 5-or 6-point Likert scales, and the sum of item scores in each domain is then transformed to scores ranging from 0 (no functioning) to 100 (highest level of functioning) by subtracting the lowest possible score, dividing by the range of the scale and multiplying by 100 [7].

EuroQoL-5 dimension

The euro quality of life 5 dimension (EQ5D) is a 5-item generic measure of HRQOL. It has a five descriptive system, including mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Each item is based on a 3-response Likert scale with response options ranging from “no problem” to “severe problems” [14]. The 5-item scale is also accompanied by a visual analogue scale (VAS) where respondents evaluate the state of their health by indicating a position on a vertical, calibrated line starting at 0 (the worst health state imaginable) to 100 (the best health state imaginable). The EQ5D has been validated in cardiac patients and in several populations (including the Canadian population) and is known to demonstrate good psychometric properties [15,16,17].

Hospital anxiety and depression scale

The hospital anxiety and depression scale (HADS) is a self-reported measure of anxiety and depression. Of the 14 items, 7 items are related to anxiety symptoms, while the other 7 items are related to depressive symptoms. Each is rated on a 4-point Likert scale [18]. The total HADS score ranges between 0 and 42, with 0 to 14 considered as low, 15 to 28 considered as moderate and 29 to 42 considered as high. For each of the anxiety and depression subscales, the scores range between 0 and 21, where 0–7 is considered as low, 8–14 moderate and 15–21 as high. HADS has undergone extensive reliability and validity testing in cohorts with different chronic medical conditions and populations (including Canadian populations) and has been widely validated in patients with heart disease [19, 20]. For this analysis, the baseline information of HADS with a total for each of the subscales was used.

Statistical analysis

Means and standard deviation (SD) were used to summarize continuous outcomes, while frequencies and percentages were used to summarize categorical variables. The 19 items of the SAQ were assessed for floor and ceiling effects [21]. Exploratory factor analysis (EFA) using the varimax rotation and maximum likelihood extraction method was used to identify the underlying dimensions of the 19 items data [22]. The number of factors was evaluated using the Eigen value criteria (number of Eigen values > 1) and scree plot. Items with component loadings ≥ 0.40 [22] on the dimensions were retained. Confirmatory factor analysis (CFA) was used to test the hypothesis about the optimal factorial structure for the data. Model fit was assessed using the root mean square error of approximation (RMSEA; RMSEA < 0.08) [23] and comparative fit index (CFI; CFI ≥ 0.95) [24]. The item-total correlation as an indicator of item specificity was calculated for the individual items and the sum of the scores on the remaining items in that scale and was considered for both the original SAQ and SAQ-CAN. The internal consistency reliability of the dimensions was assessed using Cronbach’s alpha (α), with 0.7 ≤ α < 0.8 considered acceptable [25]. The reproducibility of the SAQ-CAN and original SAQ using the intra-class correlation coefficient was assessed. Higher intra-class correlation coefficients (range 0–1) indicate greater reproducibility. The responsiveness of the SAQ-CAN was measured using the standardized response mean (SRM) [26] over 1 year. The construct validity of the identified SAQ-CAN dimensions was assessed in relation to validated scales using correlation analysis. Specifically, the association between the identified dimensions and original SAQ-19, HADS_A, HADS_D, and EQ5D using Pearson correlation. The polyserial correlation was used to assess the association between the identified dimensions and the ordinal items of the EQ5D. All analyses were conducted in SPSS and AMOS v25.

Results

Descriptive analyses

Table 1 describes the demographic and clinical characteristics of this cohort. Of the 4052 patients included, 3239(79.94%) were male, 771(19.03%) were at least 75 years of age, while 2645(65.28%) had a left ventricular ejection fraction of greater than or equal to 50%. Although this cohort’s demographic characteristics are comparable to the Canadian population of individuals with heart disease [27], this cohort had fewer female patients than the general population of Canadian older adults and seniors [28]. Of the 19 SAQ items, only four items had floor effects above 15%, while most of the items had substantial ceiling effects (see Table 2).

Table 1 Baseline characteristics of study participants (N = 4052)

Table 2 Floor, ceiling item ceiling effect and item-total correlation for SAQ-CAN items

Psychometric analyses

Table 3 describes the results of the EFA, which revealed 4 main dimensions with Eigen values > 1, explaining 57.62% of the variation in the items. A repeat of the EFA without three items; one with smaller factor loadings (“bothersome with taking pills as prescribed”) and two with cross-loadings on two different factors (“climbing a hill/stairs”, “gardening/vacuuming”) resulted in the same 4 dimensions with 60.39% of the variance explained by the factors (See Fig. 1).

Table 3 Exploratory factor analysis of SAQ before removal of failed Items

Fig. 1

Exploratory factor analysis scree plot for 19-item Seattle Angina Questionnaire

The themes for these four dimensions were indoor physical functioning (3 items), outdoor physical functioning (4 items), treatment experience (3 items), and angina symptoms burden (6 items). Table 4 and Fig. 2 describe the results of the confirmatory factor analysis showing that a four-factorial structure for 16 items provided the best fit for the data (RMSEA = 0.049 (90% CI = [0.047–0.052]) and CFI = 0.975). The factor analytic output displayed in Fig. 2 shows the correlations among measured variables, latent factors of the constructs, and error terms for the variables.

Table 4 Exploratory factor analysis of the SAQ for stable angina patients after removal of failed items (SAQ-CAN)

Fig. 2

Factorial structure of the Seattle Angina Questionnaire for stable angina. NB: OPF outdoor physical functioning, IPF indoor physical functioning, ASB angina symptoms burden, TRE treatment experience

Table 5 describes the internal consistency and reliability of the 16-item SAQ-CAN in comparison with the original 19-item SAQ. SAQ-CAN items had a good level of item total correlation with the remainder of their scale (0.46–0.86) and exceeded the accepted standard of the midrange of 0.4–0.8 [29]. The average levels of item-total correlation exceeded those for the original SAQ (0.19–0.80). The Cronbach’s alpha values for the four dimensions of the 16-item SAQ ranged between 0.77 and 0.90, exceeding the Cronbach’s alpha values for the 5 subscales of the 19-item. Similarly, the 16-item SAQ had a higher intra-class correlation coefficient compared to the original SAQ, showing that it had greater reproducibility.

Table 5 Internal consistency and reliability of the SAQ-CAN and original SAQ

Table 6 describes the measure of change using a paired t-test, effect size and SRM. The paired t-test showed that the longitudinal sample tended to have better functioning (p < 0.001) for all domains. Only treatment experience has a smaller effect size, while angina symptoms burden had a moderate effect size, and indoor physical functioning has a large effect size [30].

Table 6 Mean (95% confidence limits) baseline and 12-month SAQ-CAN and SAQ, effect size (ES), standardized response means (SRM)

There was a substantial improvement in functioning from baseline to 1 year. Response mean was smallest for treatment experience and outdoor physical functioning, but indoor physical functioning had the highest response mean.

Table 7 describes the association between the subscales of the SAQ-CAN, SAQ, HADS, and EQ5D VAS. The indoor and outdoor physical functioning subscales showed a significantly strong correlation with the physical limitation subscale of the SAQ. Angina symptoms burden of the SAQ-CAN was strongly correlated with angina symptoms and angina frequency subscales of the SAQ, while the treatment experience subscale of the SAQ-CAN was strongly correlated with the treatment satisfaction subscale of the SAQ. The SAQ-CAN subscales exhibited a moderate correlation with the EQ5D VAS, but weak correlations with depression and anxiety subscales of the HADs. The polyserial correlation of the SAQ-CAN subscale with the EQ5D subscale showed a moderate negative correlation (see Table 8).

Table 7 Correlation between SAQ-CAN and other measures

Table 8 Polyserial correlation with the SAQ-CAN subscale, and EQ-5D subscales

Discussion

This study evaluated the psychometric properties of the SAQ in a Canadian sample of patients with stable angina. The analysis revealed that the original factorial structure of the SAQ was not valid in our sample and resulted in the removal of three redundant items with a negligible contribution to the clinically meaningful dimensions. The resulting measure is the SAQ-CAN, which comprised of 16-items that aggregates into four subscales with excellent validity, reliability, and responsiveness. Unlike the 19-item SAQ comprising 5 subscales, the SAQ-CAN items aggregate into four subscales: namely indoor physical functioning, outdoor physical functioning, angina symptoms/burden, and treatment experience subscales. These findings are consistent with previous studies on the validation of SAQ in other populations where different number and types of subscales emerged [9, 11]. For example, the validation of the SAQ in a UK population of patients with stable angina resulted in a similar 16-item measure (SAQ-UK) with three subscales [9, 11]. Furthermore, another unique feature of the SAQ-CAN is its delineation of the physical functioning subscale into two separate subscales (indoor physical functioning and outdoor physical functioning subscales) in the SAQ-CAN. In contrast, the items of the “angina stability” and “angina frequency” subscales of the original SAQ constitute the angina symptoms/burden subscale of the SAQ-CAN. These differences are consistent with findings from other validation studies of the SAQ. For example, Kimble et al. [9] also reported the division of the physical limitation subscale into two separate factors including ‘limitation in activities with middle to high exertional requirements’ and ‘limitation in activities with low exertional requirements’ in women with chronic stable angina [9]. Similarly, the translation and validation of the Farsi version of the SAQ yielded a five-factor solution with subscales that were not identical to the original SAQ subscales [10]. This highlights the need for preliminary validation and adaptation of the measure in each population before its deployment in clinical care.

A major strength of this study is its investigation of both construct and factorial validity, reliability, and responsiveness of the SAQ-CAN. Although SAQ is a widely-used measure, its factorial validity has not been replicated in any other study. A possible explanation for this limitation may be attributed to the factorial structure and subscale composition of the SAQ, which included a subscale with a single item (angina frequency). The findings of this study will further facilitate the interpretation of the SAQ-CAN’s scores and changes in those scores over time. This study is not without its limitations. First, floor and ceiling effects in the items could result in difficulty discriminating between the functioning of individuals within the lower or upper range of the scale. There is floor effect in four items; “running or jogging”, “lifting or moving heavy objects”, “participate in strenuous sports” and “feelings about symptoms persistent”. Second, our assessment of the test–retest reliability of the SAQ was based on data collected between 1 year-interval. Test–retest reliability is usually assessed over much shorter periods than in this study, usually producing reliability estimates that are much closer to those derived from internal consistency tests. Our future research will seek to validate these findings in a Canadian prospective cohort study where the SAQ-CAN can be administered within a shorter interval to confirm its responsiveness. Third, the validation of the SAQ-CAN, which is a subset of the original 19-item SAQ using a sample of patients who completed the original 19-item SAQ, suggests that comparisons of strength of correlation between each measure and other measures are not entirely independent. This might lead to biased estimates of correlations and consequently influence conclusions about the validity of the SAQ-CAN. Fourth, the validation of the SAQ-CAN in this study was based on secondary analyses of the population based on data of patients with chronic stable angina who completed the SAQ along with other important measures in the APPROACH registry. Future research will seek to replicate these findings in a prospective longitudinal study of individuals with stable angina. Fifth, we did not investigate a split-sample approach for conducting EFA and CFA in our sample despite having a fairly large sample. Future research will seek to replicate this factorial structure of the SAQ-CAN in an independent sample of stable angina patients. Finally, our validation of the SAQ-CAN relies on classical test theory approaches, which are known for their shortcomings [29]. Future research will examine the use of modern classical test theory approaches (i.e., item response theory) to further study the psychometric properties of the SAQ-CAN.

In conclusion, this study provides evidence for the measurement properties of a Canadian version of the SAQ. The SAQ-CAN is recommended as a patient-reported outcome measure for use in clinical trials and observational studies to assess health outcomes and the effectiveness of interventions in Canadians with coronary artery disease.