If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Corresponding author: Dr Ricky D. Turgeon, Faculty of Pharmaceutical Sciences, University of British Columbia, Room 5523 – 2405 Wesbrook Mall, Vancouver, British Columbia V6T 1Z3, Canada. Tel.: +1-1-236-777-6961.
Patient educational resources on heart failure (HF) medications may improve patient understanding, which is critical for informed decision-making and patient self-efficacy. The purpose of our study was to evaluate the quality and readability of written medication educational resources available online.
Methods
Two investigators searched Google, Yahoo, and Bing for written patient educational resources that addressed at least one HF medication. We assessed educational quality using the Ensuring Quality Information for Patients (EQIP) tool (range 0 [worst] to 100 [best]), and we evaluated readability using the Flesch-Kincaid Grade Level.
Results
From 693 identified webpages, 39 HF medication educational resources met study eligibility. Among included resources, the median Ensuring Quality Information for Patients score was 61% (interquartile range 54%-68%), with 2 (5%) rated as high quality (score ≥ 75%). The median Flesch-Kincaid Grade Level was 8 (interquartile range 8-12), with 4 (10%) resources meeting the recommended 6th-grade reading level.
Conclusions
Most HF medication educational resources available on the Internet are of acceptable educational quality, but could readily be improved. Most resources were beyond the recommended reading grade level for educational resources, limiting their utility for patients with a low literacy level.
Résumé
Contexte
Les ressources éducatives destinées aux patients au sujet des médicaments contre l’insuffisance cardiaque (IC) pourraient améliorer la compréhension des patients, ce qui est essentiel pour la prise de décisions éclairées et pour le sentiment d’autoefficacité des patients. L’objectif de notre étude était d’évaluer la qualité et la lisibilité des ressources médicales éducatives écrites en ligne.
Méthodologie
Deux membres de l’équipe de recherche ont utilisé Google, Yahoo et Bing pour repérer les ressources éducatives écrites destinées aux patients et portant sur au moins un médicament contre l’IC. La qualité éducative des ressources a été évaluée avec l’outil « Ensuring Quality Information for Patients » (EQIP), qui fournit avec un score allant de 0 (pire) à 100 (meilleur), et la lisibilité a été évaluée avec le test « Flesch-Kincaid Grade Level ».
Résultats
Sur 693 pages Web repérées, 39 ressources éducatives sur les médicaments contre l’IC répondaient aux critères de notre étude. Pour les ressources évaluées, le score médian sur l’échelle EQIP était de 61 % (intervalle interquartile de 54 % à 68 %), et deux d’entre elles (5 %) ont obtenu un score indiquant une qualité élevée (score ≥ 75 %). Le résultat médian au test Flesch-Kincaid Grade Level était de 8 années de scolarité (intervalle interquartile de 8 à 12), et quatre ressources (10 %) respectaient le niveau de lecture recommandé, qui correspond à une 6e année de scolarité.
Conclusions
La plupart des ressources éducatives en ligne sur les médicaments contre l’IC sont d’une qualité éducative acceptable, mais des améliorations sont possibles. La plupart des ressources corres-pondent à un niveau de lecture plus élevé que ce qui est recommandé pour la rédaction de ressources éducatives, ce qui limite leur utilité pour les patients qui ont un faible niveau de littératie.
Despite the high morbidity and mortality rates associated with HF, it can be successfully managed with a combination of healthy behaviours, medical devices, and medications.
The variety and complexity of HF treatment options may result in patients and their caregivers feeling overwhelmed, which can negatively impact patient outcomes.
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
Various types of educational resources exist, including those designed to enhance patient knowledge (eg, informational pamphlets, patient handouts) and specifically to support shared decision-making (ie, decision aids). Regardless of the type, ideally, written educational resources regarding HF should facilitate discussions between patients and their healthcare providers, should be based on high-quality evidence, and should be developed with the patient’s informational needs and health literacy in mind.
Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
Assessment of the quality of existing patient educational tools focused on sudden cardiac arrest: a systematic evaluation by the Sudden Cardiac Arrest Thought Leadership Alliance.
For instance, a 2014 review assessed the quality of available patient educational resources on left ventricular assist devices, for patients with advanced HF, using the Flesch-Kincaid Grade Level score, the Fry algorithm, and a modified version of the International Patient Decision Aid Standards.
Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
Despite the fact that pharmacotherapy is the cornerstone of HF management, educational resources on HF medication have not yet undergone the same level of rigorous evaluation. Therefore, the purpose of our study was to collect, categorize, and evaluate the quality and readability of online written patient educational resources regarding HF medication.
Methods
Search strategy
From May 2021 to June 2021, 2 investigators (S.L. and F.Y.) independently conducted simultaneous, comprehensive searches of the top 3 Internet search engines—Google, Yahoo, and Bing—using Google Chrome (the most commonly used browser) to collect Internet-based educational resources about HF medications.
Search queries included the following: (i) “Heart failure medication patient information”; (ii) “Heart failure medications patient information”; (iii) “Heart failure medication patient handout”; (iv) “Heart failure medications patient handout”; (v) “Heart failure drugs patient information”; and (vi) “Heart failure drugs patient handout”. The search queries used were developed through discussion and collaboration between the researchers, with the goal of replicating search terms that patients looking for this information might use (ie, more general, broad terms, in lay language). The investigators used an empirical method to minimize the influence of prior search history, which involved clearing the cookies from their web browsers and entering “incognito mode” prior to each search. To emulate typical Internet searches of patients looking for health information, we considered the results from the first 2 pages of each search.
How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews.
The number of results per page was 10 for Google and Yahoo, and 8-10 for Bing, depending on ads. Additionally, we manually searched Web sites from key cardiovascular, HF, pharmacy, and patient advocacy organizations; HF medication manufacturers; and the Ottawa Decision Aid Inventory (Supplemental Table S1). Key Web sites were defined as being those that are well known Web sites of cardiovascular and HF societies, prominent Canadian cardiac centres, and health authorities. Evaluators assessed resources as a whole and multiple pages, if relevant, but hyperlinks to different URLs were not assessed. Further, S.L. and F.Y. individually removed any intra-assessor duplicate resources from their results prior to combining their lists. After the results were combined, between-assessor duplicates were removed. Lastly, although we were unable to determine the exact intention of the resources evaluated, the inferred intention was that they were developed to provide patient education. Therefore, we included resources irrespective of their stated (or unstated) intention if they met all the eligibility criteria below.
Eligibility criteria
We included educational resources that met the following criteria: (1) written resources directed at patients; (2) written in English; (3) included ≥ 10 sentences about HF medications (as per Iacovetto et al. 2014
Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
) to ensure inclusion of resources with substantive content on medications; (4) accessible without having to register and/or pay a fee; (5) specifically described ≥ 1 different HF medication treatment option (i.e. individual medication or drug class); and (6) provided medication information specific to their use in HF.
Primary outcome: EQUIP score
The primary outcome was educational resource quality, based on the Ensuring Quality Information for Patients (EQIP) score. EQIP is a validated 20-item questionnaire first described in 2004 that measures the quality of patient information on a scale from 0% (worst) to 100% (best) by assessing the domains of completeness, appearance, understandability, and usefulness (Supplemental Appendix S1).
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
Each question has 3 possible answers with an accompanying score of 1 (yes), 0.5 (partly), or 0 (no). The total EQIP score was determined by averaging individual items’ scores using the following formula: [((# of Yes∗1) + (# of Partly∗0.5) + (# of No∗0))/20 – (# of does not apply))] ∗100 = % score.
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
Domains that were not applicable to a given resource were excluded from the overall score calculation. Two reviewers (F.Y. and S.L.) evaluated each resource using the EQIP score, with a discrepancy between reviewers defined as a difference of ≥ 10% in EQIP score. Discrepant scores, which could occur due to the subjective nature of some of the questions, were reviewed by the 2 reviewers in order to reach a consensus, with disagreements resolved by a third author (R.T.). Following resolution of discrepant scores, the 2 EQIP scores calculated for each resource were averaged. This final averaged score was used in the analysis and to categorize each resource as one of the following: (i) well written, high-quality (score of 76%-100%); (ii) good quality with minor problems (score of 51%-75%); (iii) serious problems with quality (score of 26%-50%); and (iv) severe problems with quality (score of 0%-25%), consistent with categories recommended in the original EQIP development publication.
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
Readability was assessed primarily using the Flesch-Kincaid Grade Level score, and secondarily, via the Flesch Reading Ease score, by copying the text of each resource into a Microsoft Word 2016 (Microsoft, Redmond, WA) document and using the software’s built-in readability function. The Flesch-Kincaid Grade Level and the Flesch Reading Ease scores have previously been validated and used to measure the readability of patient educational resources.
Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
For this study, HF medication patient educational resources with a higher than 6th-grade reading level were considered to have low readability. We further used a Flesch Reading Ease score < 80 to indicate low readability.
Lastly, we collected data on each resource’s total word count, for descriptive purposes.
Statistical analysis
We performed descriptive analyses, and we report data using medians and interquartile ranges (IQRs) for continuous variables, and frequencies and percentages for categorical variables. All analyses were performed using Microsoft Excel 2016 (Microsoft, Redmond, WA).
Results
From 693 search results, we identified 39 HF medication educational resources that met our eligibility criteria (Fig. 1). Despite the methods employed and the fact that the searches conducted were identical, F.Y.’s and S.L.’s search results differed slightly, likely due to differences in geographic location (Alberta vs British Columbia). Key characteristics of HF medication educational resources are described individually in Table 1, and summary statistics are provided in Table 2.
The median EQIP score was 61% (IQR 54% to 68%; Table 2; Fig. 2A). Of the 39 included educational resources, 2 (5%) were of high quality, 30 (77%) were of good quality with minor problems, and 7 (18%) had serious problems with quality. Assessment of the specific components of the total EQIP score revealed that 36 resources (92%) used lay terms, 38 (97%) addressed the reader personally, and 39 (100%) had a respectful tone. However, only 2 resources (5%; including the highest-scoring resource) reported consulting with patients/family members during development. In terms of medication-specific information, 38 resources (97%) addressed the purpose of described HF medications, 28 (72%) qualitatively described benefits (eg, decrease mortality), 28 (72%) qualitatively described the medication risks and side-effects, and 21 (54%) addressed alternative medications (eg, angiotensin-receptor blocker if cough develops while taking an angiotensin-converting enzyme inhibitor). Further, 34 resources (87%) used generic names (instead of, or in addition to, brand names) and distinguished brand names as such, whereas the remaining 5 resources (13%) only discussed medication classes. Lastly, 24 resources (62%) did not address quality-of-life issues, with the remaining 15 (38%) only partly addressing this item. Only 5 resources (13%) included visual content to supplement the text.
Figure 2Boxplot of (A) Ensuring Quality Information for Patients (EQIP) score and (B) Flesch-Kincaid Grade Level for 39 heart failure medication educational resources.
The median Flesch-Kincaid Grade Level was an 8th-grade reading level (IQR 8th to 10th grade) (Fig. 2B). Only 4 resources (10%) achieved the recommended 6th-grade reading level, and no resource had a 5th-grade or lower reading level. The median Flesch Reading Ease score was 59 (IQR 52 to 65), with all resources demonstrating low readability based on this score.
Discussion
HF is a chronic, complex condition that can be treated using a variety of healthy behaviours, medical devices, and medications.
Provision of written resource for patients to review on their own time and at their own pace can significantly improve information retention, compared to verbal instruction.
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
In this study, we found that most online HF medication patient educational resources were of good quality with minor problems, based on the EQIP tool. However, most HF medication educational resources were written at a grade level beyond the reading level of most people in the general population.
We identified several key areas for improvement among HF medication patient educational resources. In general, resources had a respectful tone, used generic names of medications, presented information in a logical order, and qualitatively described the purpose, side-effects, benefits (though not quantitatively,) and alternatives of medications. However, few followed a summative flow (ie, outlining the document content and covering what is stated), involved patients and caregivers in their development, or addressed quality-of-life issues.
Having a clear definition outlining the document content can help patients, caregivers, and healthcare providers understand whether the resource is suitable for the patient’s specific informational needs.
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
Therefore, involving patients and caregivers in development of resources can help tailor educational materials to the needs of patients, making them more useful and relevant to their target audience. Further, describing the quality-of-life and practical impacts of medications ensures resource transparency, enables realistic patient expectations,
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
Such practical considerations align closely with quality-of-life issues incorporated into the EQIP questionnaire (eg, how a treatment impacts a patient’s daily routine or social life) and should be included in HF medication educational resources.
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
These details are also more likely to align with patients’ individual goals of therapy (eg, improve symptoms, daily functioning, capacity for social interaction, etc.) and are crucial to include in patient education to engage patients in their care and medication management.
Our search identified resources with a wide range of readability, with few resources achieving sufficient readability. The readability level of HF medication educational resources thereby poses a problem, as patients may not be able to fully comprehend the included information, regardless of its quality. An additional consideration for barriers to use of educational resources is the length of time required to read through them. The average adult reads at a pace of approximately 250 words/minute.
Of the top 5 highest-scoring resources based on the EQIP tool, the median word count was 4778, indicating approximately 19 minutes of reading time. For the average patient, who may be new to their diagnosis and receiving an overwhelming amount of information, resource length is likely a substantial barrier to absorbing and implementing the included information.
Limitations to this study warrant discussion. First, we restricted resource inclusion to written materials available online. We did not capture information presented within social media communities (eg, Reddit, Facebook, etc.), video resources (eg, YouTube), or local/clinic-specific resources that are not available online. We anticipate greater variability of quality of these resources, as they may be subject to less curation. Second, the EQIP instrument was open to interpretation, which resulted in initial variability in scoring between the 2 reviewers. However, we mitigated this variability by incorporating duplicate scoring and an a priori plan to identify and resolve discrepancies. Third, the EQIP questionnaire used for our primary outcome defines quality from the perspective of healthcare professionals, rather than from the perspective of the end-user (ie, the patient). Therefore, it may not encompass all domains of quality or treatment considerations that are important to patients, or capture the relative value that patients place on each domain.
Conclusions
Most HF medication educational resources available on the Internet are of acceptable educational quality, but could readily be improved. Most resources were beyond the recommended reading grade level for educational resources, limiting their utility for patients with a low literacy level.
Funding Sources
The authors have no funding sources to declare.
Disclosures
The authors have no conflicts of interest to disclose.
Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
Assessment of the quality of existing patient educational tools focused on sudden cardiac arrest: a systematic evaluation by the Sudden Cardiac Arrest Thought Leadership Alliance.
How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews.
Ethics Statement: The research being reported has adhered to the relevant ethical guidelines. This study did not require approval by an ethics review board as no patient data was used.
00130-5&id=fx1.jpg){kind=link}
00130-5&id=gr1.jpg){kind=link}
00130-5&id=gr2.jpg){kind=link}