If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Patient and Family Partnership Committee, Canadian Critical Care Trials Group, Markham, Ontario, CanadaDivision of Critical Care, University of Calgary, Calgary, Alberta, Canada
Interdepartmental Division of Critical Care, Department of Medicine, University of Toronto, Toronto, Ontario, CanadaLi Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, Ontario, CanadaDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
Engaging families in care leads to improved patient- and family-centreed outcomes and is recommended by cardiovascular societies. However, no validated tools are currently available to measure family engagement in acute cardiac care. We previously described the development of the FAMily Engagement (FAME) instrument. The purpose of this study is to validate the FAME instrument in acute cardiac care.
Methods
The FAME questionnaire was administered to family members of patients in a cardiovascular intensive care unit and ward at an academic tertiary care hospital in Montreal, Canada. After hospital discharge, we assessed family satisfaction in the intensive care unit (FS-ICU) and mental health (using the Hospital Anxiety and Depression Scale [HADS]). Higher FAME scores indicate increased care engagement. Reliability was assessed using internal consistency testing. Predictive validity was evaluated by assessing the relationship between the FAME score and the FS-ICU score and whether the FAME score was correlated with the HADS score. Convergent validity was assessed by comparing the FAME score with engagement elements of the FS-ICU score.
Results
A total of 160 family participants were included (age 54.8 ± 14.8 years; 66% women; 36% non-White). The most common relationships to the patient were spouse/partner and adult child (both n = 62; 39%). The mean FAME score was 70.8 ± 16.0. The FAME instrument had high internal consistency (Cronbach’s a = 0.86). The FAME score was associated with family satisfaction in the multivariable analysis (P < 0.001). No correlation occurred between FAME and HADS anxiety or depression scores.
Conclusions
The FAME tool demonstrated reliability and convergent and predictive validity in the acute care cardiac population. Further research is needed to explore whether selected engagement interventions can impact the FAME score favourably.
Résumé
Contexte
La participation des familles aux soins entraîne une amélioration des résultats centrés sur le patient et sur la famille et est recommandée par les sociétés cardiovasculaires. Cependant, il n’existe actuellement aucun outil validé pour mesurer la participation des familles aux soins cardiaques aigus. Par le passé, nous avons décrit l’élaboration de l’instrument FAMily Engagement (FAME). L’étude dont il est question ici vise à valider l’instrument FAME en contexte de soins cardiaques aigus.
Méthodologie
Nous avons soumis le questionnaire FAME aux proches des patients admis aux soins intensifs cardiovasculaires et à l’unité de soins cardiovasculaires dans un hôpital universitaire de soins tertiaires à Montréal, au Canada. Lorsque le patient a reçu son congé de l’hôpital, nous avons évalué la satisfaction de la famille à l’égard de l’unité des soins intensifs (FS-ICU, de l'anglais : family satisfaction in the intensif care unit) de même que l’état de santé mentale (à l’aide de l’échelle d’anxiété et de dépression en milieu hospitalier [EHAD]). Un score élevé au questionnaire FAME indiquait une participation plus active aux soins. La fiabilité a été évaluée selon la méthode de cohérence interne. Pour mesurer la validité prédictive, nous avons étudié le lien entre le score au questionnaire FAME et le score FS-ICU, et déterminé si le score au questionnaire FAME était corrélé au score à l’EHAD. La validité convergente a été évaluée en comparant le score du questionnaire FAME avec les composantes de la participation du score FS-ICU.
Résultats
Au total, 160 participants ont été inclus (âge : 54,8 ± 14,8 ans; 66 % de femmes; 36 % de personnes non blanches). En général, les participants étaient soit les conjoint(e)s/partenaires des patients, soit les enfants adultes des patients (n = 62 pour chacun des deux cas; 39 %). Le score FAME moyen était de 70,8 ± 16,0. L’instrument FAME présentait une forte cohérence interne (a de Cronbach = 0,86). Le score FAME était associé à la satisfaction familiale dans l’analyse multivariée (p < 0,001). Aucune corrélation n’a été notée entre le score FAME et les scores de dépression ou d’anxiété de l’EHAD.
Conclusions
L’outil FAME a présenté une fiabilité et une validité convergente et prédictive dans la population des patients recevant des soins cardiaques aigus. D’autres recherches sont nécessaires pour déterminer si certaines interventions relatives à la participation peuvent améliorer le score FAME.
Family engagement is an approach wherein family members are considered to be essential and active partners with the healthcare team.
Family members are a valuable resource for patient care. They may be involved in providing physical and emotional support, communicating with the healthcare team, participating as surrogate decision-makers, and providing direct patient care. Engaging families in care is the operational means to achieve patient- and family-centreed care, which is a goal of contemporary healthcare delivery.
A growing body of evidence supports the approach of engaging families in care to improve family experiences and achieve better outcomes for patients and family members.
Currently, no validated tools have been developed that are designed specifically to measure family engagement in acute care. A recent American Heart Association scientific statement highlighted the need for a validated instrument to measure family engagement in acute cardiac care.
Measuring family engagement is important to allow researchers and healthcare organizations to benchmark efforts between institutions and quantify the impact of engagement interventions. Quantification of family engagement in care is also needed to better understand the relationship between family engagement and outcomes.
Our interdisciplinary team, which included partnership with patient and family advocates, developed a novel family engagement measure for acute care.
The objective of this study was to validate this family engagement measure in an acute cardiac care setting. We hypothesized that the FAMily Engagement (FAME) tool is reliable and has predictive and convergent validity in this population.
Methods
Study design
We conducted a prospective observational study in the cardiovascular intensive care unit (CVICU) and cardiovascular ward of the Jewish General Hospital, an academic tertiary care centre in Montreal, Canada. This study was conducted as a quality improvement (QI) project and was approved by the institutional quality improvement department. Research ethics approval was waived for this study. The study was reported according to the COnsensus-Based Standards for the Selection of Health Status Measurement INstruments (COSMIN) reporting guidelines for patient-reported outcome measures.
Patient advisors were involved in study conception and design, and manuscript review.
Participants
On days of questionnaire administration, consecutive family members on the CVICU and cardiovascular ward were approached by quality improvement (QI) personnel. QI personnel approached family members during both the daytime and evening so as to include family members who were present at only one time of the day. Family members were eligible if they were older than 18 years and able to understand English or French. The dates of questionnaire administration were discontinuous; it was conducted over a 4-month period (May 2022 to August 2022) to capture different healthcare provider practice patterns and obtain the maximum number of responses on administration dates.
We used the following approach to identify family members: If patients could communicate, they were asked to identify the most responsible family member involved in their care. For patients who could not communicate, the family member present at the bedside was eligible to participate. People considered to be ”family” typically include anyone the patient would like to have involved in her/his care, regardless of their biological connection.
Only one family member per patient was allowed to complete the questionnaire. Family members were given the option of completing the questionnaire on paper or electronically.
Data collection procedures
Participant recruitment consisted of 2 phases. In phase 1, QI personnel obtained sociodemographic information and administered the FAME questionnaire to each family participant. After this phase, the QI team reviewed the data and decided that additional data were needed to capture the impact of engagement on family-centreed outcomes. In phase 2, QI personnel obtained sociodemographic information and administered the FAME questionnaire to each family participant at enrollment; they subsequently administered the Family Satisfaction in the Intensive Care Unit (FS-ICU) and the Hospital Anxiety and Depression Scale (HADS) within 2 weeks following the patient’s hospital discharge, either by phone or via an e-mail link. The QI personnel were medical students participating in the project and were not employed by the hospital or part of the care team. The healthcare team was blinded to which patients’ families were enrolled in the study.
Initial questionnaire administration was performed on 25 nonconsecutive days. Daytime data collection occurred on 23 days, and evening data collection took place on 7 days. Of the evening data collection dates, 5 dates had data collection both during the daytime and in the evening. Evening was defined as after 5 PM. Daytime weekend data collection occurred on only 1 occasion.
On the dates of questionnaire administration, QI personnel systematically walked through the cardiac units several times during the daytime and/or evening. If a family member was present in the room, QI personnel would approach the patient and the family member to describe the project. If no family members were present, QI personnel returned repeatedly to see if a family member was present.
The following sociodemographic information was collected from each participant: age, gender, relationship to the patient, race/ethnicity, living status (living with or not living with the patient), and highest level of education achieved. We recorded the following data using the FAME questionnaire: language (English or French); format (paper or electronic); cardiovascular unit (CVICU or ward); type of admission (medical or cardiac surgical); and time of day administered (daytime or evening).
Description of the FAME tool development and initial testing
The development and initial testing of the FAME measure have been previously described.
The FAME tool was developed in an iterative process by an interdisciplinary team with expertise in family engagement in care and questionnaire development methodology (Supplemental Fig. S1). Patient and family partners were involved throughout the development process. The FAME questionnaire comprises 12 items scored on a 5-point Likert scale ranging from 1 (strongly agree) to 5 (strongly disagree). Scale scores are subsequently transformed into a 0-100 scoring system (ie, a scale score of 1 is changed to 100; a score of 5 is changed to a 0). Higher scores indicate greater family engagement in care. The total FAME engagement score is calculated by adding up individual scores and dividing by the number of questions. Subscale scores assess the following engagement domains: perception of engagement, family presence, communication/education, decision-making, contributing to care, family-centreed care, and family needs. Both English and French versions were developed. Initial testing found that the FAME instrument has face validity and high content validity.
The tool is available in more than 20 languages, including French. The tool consists of 24 questions that cover the following domains: overall satisfaction (average of questions 1-24); satisfaction with care (average of questions 1-14); and satisfaction with decision-making (average of questions 15-24). Some items pertain to key elements of patient- and family-centreed care, such as information sharing, communication, decision-making, and family needs. Scores are reported with a range of 0-100, with higher scores indicating a higher level of care satisfaction.
The survey is composed of 14 questions—7 related to depression, and 7 related to anxiety. Each question is scored from 0 to 3, with a maximum of 21 points in each subscale (HADS-anxiety; HADS-depression). Anxiety or depression is a score of ≥ 8 on the respective scale, with higher scores indicating more anxiety or depression symptoms.
Current family engagement practice in study setting
The acute cardiovascular units at the Jewish General Hospital allow open visitation, which allows family presence at any time, without periods of restriction. Family visitors receive information on ways to assist with the care of their loved one (eg, with mobility, oral care, or nutrition). The following family engagement practices are permitted, but are not routinely practiced in the CVICU or acute cardiac ward: family presence during daily multidisciplinary team rounds, resuscitation, or invasive procedures; structured patient and family care conferences to establish goals of care in high-risk patients; and promotion of use of patient and/or family ICU diaries. No patient and family advisory group thatmeets regularly is available. Integrative therapies, such as those involving music or pets, are not routinely performed in either unit.
Statistical analyses
Continuous data are presented as mean ± standard deviation, and between-group differences were tested with the independent samples t-test or 1-way analysis of variance, as appropriate. Categorical data are presented as frequencies and percentages and were compared using the χ2 test or the Fisher exact test, as appropriate.
To assess for reliability of the FAME tool, we performed internal consistency testing using the Cronbach alpha, reflecting the ordinal nature of the questionnaire responses. The Cronbach alpha is interpreted as follows: excellent, 0.91-1.0; good, 0.81-0.90; good and acceptable, 0.71-0.80; acceptable, 0.61-0.70; and nonacceptable, 0.01-0.60.
To assess for convergent validity, we used Pearson's correlation to assess the convergent relationship between the FAME score and relevant domains with the engagement elements of the FS-ICU score.
To assess for predictive validity, we used a multivariable linear regression model adjusted for selected participant characteristics (age, relationship, gender, racial/ethnic status) to evaluate the relationship between family engagement (total FAME score) and family care satisfaction (FS-ICU). We used Pearson's correlation to evaluate the relationship between engagement (total FAME score) and the HADS anxiety and depression scores.
We followed the framework for missing data in observational studies to account for outcome follow-up data.
Framework for the treatment and reporting of missing data in observational studies: the treatment and reporting of missing data in observational studies framework.
A sensitivity analysis was performed to compare the FAME score when excluding participants with the following missing demographic variables: age, gender, race, living status. Between-group comparison of demographic data was performed to compare differences between those who completed and those who did not complete the follow-up questionnaires. All P values are 2-sided, with values ≤ 0.05 indicating statistical significance. Statistical tests were done using SPSS 26.0 statistical software (IBM, Armonk, NY).
Results
Cohort characteristics
A total of 160 family members participated, of 198 family members who were approached (81%). The mean age of participants was 54.8 ± 14.8 years; 66% were women, and 36% were non-White (Table 1). The most common relationships were spouse/partner and adult child (both n = 62; 39%) and sibling (n = 16; 10%).
Table 1Demographics and FAMily Engagement (FAME) questionnaire distribution characteristics
The overall mean FAME score was 70.1 ± 17.8. FAME scores ranged from 16.6 to 100 (Supplemental Fig. S2). The highest FAME subdomain scores were for family presence (85.2 ± 18.5) and overall perception of engagement (75.2 ± 16.4), and the lowest FAME subdomain scores were for family needs (57.8 ± 27.2) and decision-making (55.0 ± 34.6; Fig. 1).
Figure 1The mean FAME score (blue column) with standard deviation (bar) for each family engagement domain is presented.
The FAME tool had high internal consistency (Cronbach alpha = 0.86). When individual questions were removed from the questionnaire, the Cronbach alpha ranged from 0.83 to 0.86.
Convergent validity
The FAME communication domain was correlated with the FS-ICU communication question score (r = 0.36; P = 0.01). The FAME decision-making score was correlated with the FS-ICU decision-making subscale score (r = 0.36; P = 0.01). The FAME family needs score was not correlated with the FS-ICU family needs question score (r = 0.21; P = 0.18).
Predictive validity
In the univariable analysis, the total FAME score was associated with the total FS-ICU score (P = 0.02), but not with age, gender, race, relationship to the patient, living status, or level of education (all P > 0.05; Supplemental Table S1; Supplemental Fig. S3). In the multivariable analysis, the total FAME score (P = 0.02) and relationship to the patient (P = 0.02) were predictors of the total FS-ICU score (Table 2).
Table 2Multivariable linear regression analysis for predictors of family satisfaction in the intensive care unit score
The mean HADS-anxiety and HADS-depression scores were 5.2 ± 4.4 and 3.3 ± 3.9, respectively. A total of 13 of 58 family participants (22.4%) met the criteria for anxiety, and 9 of 66 (13.6%) met the criteria for depression. No correlation was found between the FAME score and the HADS-anxiety (r = 0.04; P = 0.78) or depression (r = -0.052; P = 0.69). No difference was seen in the FAME score in people with vs without anxiety and/or depression (P > 0.86 for both).
Post hoc analyses
The FAME communication domain was correlated with the FS-ICU decision-making subscale score (r = 0.55; P < 0.001). The FAME score differed based on the type of admission and patient location, but not on timing or language of questionnaire administration (Supplemental Table S2). The mean FAME score differed by relationship type, but not by age, gender, race, living status, or level of education (Supplemental Table S3). No difference was present in FAME scores when participants with missing demographic variables were excluded (P > 0.05 for all). No difference was present in demographic characteristics between participants who completed vs did not complete the follow-up questionnaires (P > 0.05 for all). A forest plot of FS-ICU predictors is displayed in Supplemental Figure S4. The mean FS-ICU score varied considerably based on relationship type (range: 62.2 ± 31.4 to 91.7 ± 21.4; Supplemental Table S4).
Discussion
The objective of this study was to validate a novel instrument to measure family engagement in an acute cardiac setting. We found that the FAME instrument was reliable and had predictive and convergent validity in this population. The total FAME score was an independent predictor of family satisfaction with care, but we did not identify a relationship between the FAME score and family depression or anxiety. To our knowledge, the FAME tool is the first validated instrument focused on measuring family engagement in acute care.
In general, gold standards for predictive validity testing for questionnaire-based measures are lacking.
in: APA handbook of research methods in psychology, Vol 1: Foundations, planning, measures, and psychometrics. American Psychological Association,
Washington, DC2012: 621-642
Predictive validity was determined in our study by comparing the FAME score with the FS-ICU score, which is a validated and widely reported measure of family satisfaction with care. We found a strong relationship between the total FAME and FS-ICU scores. However, no correlation was seen in our study between the FAME and HADS scores. Prior studies exploring family engagement interventions in the critical care setting have reported improvements in family mental health outcomes, such as anxiety and depression.
Key differences exist in the patient population in the cardiovascular ICU vs the general ICU setting, notably patient incapacitation, illness acuity, and unit length of stay,
and presumably, differential impacts on family member mental health also occur. Further studies are needed to explore this link between family engagement and mental health outcomes, particularly in other critical care contexts.
Although the bulk of the literature on family engagement in care is focused on the ICU setting, family members may engage in similar behaviours (ie, shared decision-making, communication, and direct care) in the acute care ward. Thus, we included family members of people hospitalized in the acute cardiac ward in our study. Interesting to note is that family engagement scores were higher for family members of people in the cardiovascular ICU than they were for family members of people in the acute care ward (a 13-point difference). The reason for this discrepancy in engagement score by location is unclear and should be explored further. Potential explanations are that family members have more opportunities to participate in care in an ICU, that ICU-level providers devote increased attention to involving families in care, and that the need for family members to participate in certain aspects of care in an ICU is higher. Nevertheless, our findings demonstrate that family members engage in care to a considerable degree in the acute care ward setting.
We found that the FAME score differed based on the timing of administration of the FAME tool during the patient’s hospitalization. We found that the engagement score was higher earlier on during the hospitalization. However, more patients in the early-administration group ( < 24 hours since admission) were in the ICU, which overall had higher FAME scores, compared to the later-administration ( > 24 hours since admission) group. Engagement is an ongoing process, and the measurement of engagement is likely a dynamic process that changes throughout the course of hospitalization. Including serial measurements using the FAME tool during hospitalization, to test the responsiveness of the tool to changes in engagement pattern, is warranted in future studies.
Family engagement is a potentially modifiable target for intervention. Prior family engagement interventions measured the outcome of the engagement intervention (eg, family presence during cardiopulmonary resuscitation, writing an ICU diary) but not the impact of the intervention on the engagement process itself.
Thus, whether the interventions studied actually improved family engagement—that is, whether they affected what they were purported to affect—is unknown. As a process measure, the FAME instrument can quantify engagement during hospitalization, and this may have an important utility in research and QI efforts. By measuring engagement during hospitalization, QI teams can explore in real time how interventions can improve engagement. For example, suppose a family member has a low overall FAME score during a patient’s hospitalization, or has a low score in a certain engagement domain. In that case, the care team could receive an alert, and the family member could be offered support, if desired, in the deficient area. Institutional quality teams can also track engagement behaviour over time internally and compare performance across institutions. Further research is needed to explore whether engagement interventions improve the FAME score and whether increased engagement results in improved outcomes.
The FAME instrument requires validation in other healthcare contexts, such as the general critical care and medical ward setting, and in other ecologic populations (eg, the lesbian, gay, bisexual, transgender, queer [LGBTQ] population). The responsiveness and interpretability (minimal clinically important difference) of the FAME score need to be determined. FAME can be used to explore the impact of various characteristics (ie, gender and relationship to the patient) on family engagement. Use of the FAME score is needed to understand the link between family engagement and patient-reported outcomes. Future studies should also explore whether increasing family engagement, particularly for transition of care planning, can reduce the rate of readmission. The FAME tool can be used in real time during hospitalization to evaluate the impact of family-centreed interventions on family engagement.
Strengths and limitations
Strengths of the study include the following: the FAME measure and its properties were validated according to established guidelines; consecutive family members were enrolled; the overall participant profile was racial/ethnically diverse; and patient and family partners were involved throughout the measurement development and validation process. This study has limitations. This was a single-centre study in an acute cardiac population, and the results may not be generalizable to other populations. This study was observational, so unmeasured confounders may have affected the findings, and no causal relationship can be assumed. To minimize this possibility, we controlled for factors potentially associated with family satisfaction in the multivariable analysis. Only family members who were physically present when QI personnel passed by were included. To increase family participation, QI personnel attempted to approach family both during the daytime and evening, but some family members may have been missed. In addition, reasons for refusal to participate were not recorded. Understanding the reasons for nonparticipation by family members could be useful to ensure that no participatory bias is present in the study findings. In the current study, patient-level data were not linked with the FAME results. Variables such as length of hospital stay, disease state (de novo presentation or acute exacerbation of a chronic condition), prior acute care use, and patient survival status may impact the level of family engagement, as well as outcomes such as family satisfaction. Future studies should explore the connection between family engagement and clinical data and outcomes.
Conclusion
The FAME measure is reliable and has predictive and convergent validity in an acute cardiac population. Further studies are needed to evaluate the FAME measure in other populations and to explore whether interventions can increase the FAME score and improve outcomes.
Funding Sources
This study received operating funds from a McGill Department of Medicine Scholarly Quality Award; M.G. is supported by a Clinical Research Award from the Fonds de recherche en santé du Québec.
Disclosures
M.G., S.D., N.F., and K.B. were involved in the development of the FAME measure. The other authors have no conflicts of interest to disclose.
Framework for the treatment and reporting of missing data in observational studies: the treatment and reporting of missing data in observational studies framework.
in: APA handbook of research methods in psychology, Vol 1: Foundations, planning, measures, and psychometrics. American Psychological Association,
Washington, DC2012: 621-642
Ethics Statement: This study was conducted as a quality improvement project and was approved by the institutional quality improvement department. Research ethics approval was waived for this study. The study was reported according to the Consensus-Based Standards for the Selection of Health Status Measurement Instruments (COSMIN) reporting guidelines for patient-reported outcome measures.
00285-2&id=gr1.jpg){kind=link}