Advertisement

Evaluation of Online Written Medication Educational Resources for People Living With Heart Failure

Open AccessPublished:July 11, 2022DOI:https://doi.org/10.1016/j.cjco.2022.07.004

      Abstract

      Background

      Patient educational resources on heart failure (HF) medications may improve patient understanding, which is critical for informed decision-making and patient self-efficacy. The purpose of our study was to evaluate the quality and readability of written medication educational resources available online.

      Methods

      Two investigators searched Google, Yahoo, and Bing for written patient educational resources that addressed at least one HF medication. We assessed educational quality using the Ensuring Quality Information for Patients (EQIP) tool (range 0 [worst] to 100 [best]), and we evaluated readability using the Flesch-Kincaid Grade Level.

      Results

      From 693 identified webpages, 39 HF medication educational resources met study eligibility. Among included resources, the median Ensuring Quality Information for Patients score was 61% (interquartile range 54%-68%), with 2 (5%) rated as high quality (score ≥ 75%). The median Flesch-Kincaid Grade Level was 8 (interquartile range 8-12), with 4 (10%) resources meeting the recommended 6th-grade reading level.

      Conclusions

      Most HF medication educational resources available on the Internet are of acceptable educational quality, but could readily be improved. Most resources were beyond the recommended reading grade level for educational resources, limiting their utility for patients with a low literacy level.

      Résumé

      Contexte

      Les ressources éducatives destinées aux patients au sujet des médicaments contre l’insuffisance cardiaque (IC) pourraient améliorer la compréhension des patients, ce qui est essentiel pour la prise de décisions éclairées et pour le sentiment d’autoefficacité des patients. L’objectif de notre étude était d’évaluer la qualité et la lisibilité des ressources médicales éducatives écrites en ligne.

      Méthodologie

      Deux membres de l’équipe de recherche ont utilisé Google, Yahoo et Bing pour repérer les ressources éducatives écrites destinées aux patients et portant sur au moins un médicament contre l’IC. La qualité éducative des ressources a été évaluée avec l’outil « Ensuring Quality Information for Patients » (EQIP), qui fournit avec un score allant de 0 (pire) à 100 (meilleur), et la lisibilité a été évaluée avec le test « Flesch-Kincaid Grade Level ».

      Résultats

      Sur 693 pages Web repérées, 39 ressources éducatives sur les médicaments contre l’IC répondaient aux critères de notre étude. Pour les ressources évaluées, le score médian sur l’échelle EQIP était de 61 % (intervalle interquartile de 54 % à 68 %), et deux d’entre elles (5 %) ont obtenu un score indiquant une qualité élevée (score ≥ 75 %). Le résultat médian au test Flesch-Kincaid Grade Level était de 8 années de scolarité (intervalle interquartile de 8 à 12), et quatre ressources (10 %) respectaient le niveau de lecture recommandé, qui correspond à une 6e année de scolarité.

      Conclusions

      La plupart des ressources éducatives en ligne sur les médicaments contre l’IC sont d’une qualité éducative acceptable, mais des améliorations sont possibles. La plupart des ressources corres-pondent à un niveau de lecture plus élevé que ce qui est recommandé pour la rédaction de ressources éducatives, ce qui limite leur utilité pour les patients qui ont un faible niveau de littératie.

      Graphical abstract

      Heart failure (HF) affects approximately 750,000 Canadians.
      • Government of Canada
      Canadian Chronic Disease Surveillance System (CCDSS). Available at:.
      Despite the high morbidity and mortality rates associated with HF, it can be successfully managed with a combination of healthy behaviours, medical devices, and medications.
      • McDonald M.
      • Virani S.
      • Chan M.
      • et al.
      CCS/CHFS heart failure guidelines update: defining a new pharmacologic standard of care for heart failure with reduced ejection fraction.
      The variety and complexity of HF treatment options may result in patients and their caregivers feeling overwhelmed, which can negatively impact patient outcomes.
      • McHorney C.A.
      • Mansukhani S.G.
      • Anatchkova M.
      • et al.
      The impact of heart failure on patients and caregivers: a qualitative study.
      ,
      • Boyde M.
      • Peters R.
      Education material for heart failure patients: what works and what does not?.
      One intervention to overcome this issue is patient education about HF, their disease condition, and available treatment options.
      • Boyde M.
      • Peters R.
      Education material for heart failure patients: what works and what does not?.
      ,
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
      Provision of written patient educational resources may help improve patient understanding of their condition and treatment options.
      • van Weel C.
      More self reliance in patients and fewer antibiotics: still room for improvement.
      Various types of educational resources exist, including those designed to enhance patient knowledge (eg, informational pamphlets, patient handouts) and specifically to support shared decision-making (ie, decision aids). Regardless of the type, ideally, written educational resources regarding HF should facilitate discussions between patients and their healthcare providers, should be based on high-quality evidence, and should be developed with the patient’s informational needs and health literacy in mind.
      • Boyde M.
      • Peters R.
      Education material for heart failure patients: what works and what does not?.
      Emerging evidence documents the suboptimal quality and readability of patient educational resources.
      • Nassiri M.
      • Mohamed O.
      • Berzins A.
      • et al.
      Surfing behind a boat: quality and reliability of online resources on scaphoid fractures.
      • Iacovetto M.C.
      • Matlock D.D.
      • McIlvennan C.K.
      • et al.
      Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
      • Hameed I.
      • Hameed N.U.F.
      • Oakley C.T.
      • et al.
      Systematic assessment of online health information for coronary revascularization.
      • Hazelton G.
      • Al-Khatib Sears S.
      • et al.
      Assessment of the quality of existing patient educational tools focused on sudden cardiac arrest: a systematic evaluation by the Sudden Cardiac Arrest Thought Leadership Alliance.
      • Engelmann J.
      • Fischer C.
      • Nkenke E.
      Quality assessment of patient information on orthognathic surgery on the Internet.
      • Dalziel K.
      • Leveridge M.J.
      • Steele S.S.
      • Izard J.P.
      An analysis of the readability of patient information materials for common urological conditions.
      • Kher A.
      • Johnson S.
      • Griffith R.
      Readability assessment of online patient education material on congestive heart failure.
      • Lee K.S.
      • Cho Y.M.
      • Oh S.H.
      • Jung M.S.
      • Yoon J.Y.
      Evaluation of the heart failure in Internet patient information: descriptive survey study.
      For instance, a 2014 review assessed the quality of available patient educational resources on left ventricular assist devices, for patients with advanced HF, using the Flesch-Kincaid Grade Level score, the Fry algorithm, and a modified version of the International Patient Decision Aid Standards.
      • Iacovetto M.C.
      • Matlock D.D.
      • McIlvennan C.K.
      • et al.
      Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
      This study found that most educational resources available online that address left ventricular assist devices are of suboptimal quality.
      • Iacovetto M.C.
      • Matlock D.D.
      • McIlvennan C.K.
      • et al.
      Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
      Despite the fact that pharmacotherapy is the cornerstone of HF management, educational resources on HF medication have not yet undergone the same level of rigorous evaluation. Therefore, the purpose of our study was to collect, categorize, and evaluate the quality and readability of online written patient educational resources regarding HF medication.

      Methods

      Search strategy

      From May 2021 to June 2021, 2 investigators (S.L. and F.Y.) independently conducted simultaneous, comprehensive searches of the top 3 Internet search engines—Google, Yahoo, and Bing—using Google Chrome (the most commonly used browser) to collect Internet-based educational resources about HF medications.
      • Nassiri M.
      • Mohamed O.
      • Berzins A.
      • et al.
      Surfing behind a boat: quality and reliability of online resources on scaphoid fractures.
      ,
      • Hameed I.
      • Hameed N.U.F.
      • Oakley C.T.
      • et al.
      Systematic assessment of online health information for coronary revascularization.
      ,
      • Engelmann J.
      • Fischer C.
      • Nkenke E.
      Quality assessment of patient information on orthognathic surgery on the Internet.
      ,
      • NetMarketShare
      Market share statistics for Internet technologies. Available at:.
      ,
      • statcounter
      Search engine market share worldwide. Available at:.
      Search queries included the following: (i) “Heart failure medication patient information”; (ii) “Heart failure medications patient information”; (iii) “Heart failure medication patient handout”; (iv) “Heart failure medications patient handout”; (v) “Heart failure drugs patient information”; and (vi) “Heart failure drugs patient handout”. The search queries used were developed through discussion and collaboration between the researchers, with the goal of replicating search terms that patients looking for this information might use (ie, more general, broad terms, in lay language). The investigators used an empirical method to minimize the influence of prior search history, which involved clearing the cookies from their web browsers and entering “incognito mode” prior to each search. To emulate typical Internet searches of patients looking for health information, we considered the results from the first 2 pages of each search.
      • Engelmann J.
      • Fischer C.
      • Nkenke E.
      Quality assessment of patient information on orthognathic surgery on the Internet.
      ,
      • Eysenbach G.
      • Kohler C.
      How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews.
      • Aldairy T.
      • Laverick S.
      • McIntyre G.T.
      Orthognathic surgery: Is patient information on the Internet valid?.
      • Karamitros G.A.
      • Kitsos N.A.
      Clefts of the lip and palate: is the Internet a trustworthy source of information for patients?.
      The number of results per page was 10 for Google and Yahoo, and 8-10 for Bing, depending on ads. Additionally, we manually searched Web sites from key cardiovascular, HF, pharmacy, and patient advocacy organizations; HF medication manufacturers; and the Ottawa Decision Aid Inventory (Supplemental Table S1). Key Web sites were defined as being those that are well known Web sites of cardiovascular and HF societies, prominent Canadian cardiac centres, and health authorities. Evaluators assessed resources as a whole and multiple pages, if relevant, but hyperlinks to different URLs were not assessed. Further, S.L. and F.Y. individually removed any intra-assessor duplicate resources from their results prior to combining their lists. After the results were combined, between-assessor duplicates were removed. Lastly, although we were unable to determine the exact intention of the resources evaluated, the inferred intention was that they were developed to provide patient education. Therefore, we included resources irrespective of their stated (or unstated) intention if they met all the eligibility criteria below.

      Eligibility criteria

      We included educational resources that met the following criteria: (1) written resources directed at patients; (2) written in English; (3) included ≥ 10 sentences about HF medications (as per Iacovetto et al. 2014
      • Iacovetto M.C.
      • Matlock D.D.
      • McIlvennan C.K.
      • et al.
      Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
      ) to ensure inclusion of resources with substantive content on medications; (4) accessible without having to register and/or pay a fee; (5) specifically described ≥ 1 different HF medication treatment option (i.e. individual medication or drug class); and (6) provided medication information specific to their use in HF.

      Primary outcome: EQUIP score

      The primary outcome was educational resource quality, based on the Ensuring Quality Information for Patients (EQIP) score. EQIP is a validated 20-item questionnaire first described in 2004 that measures the quality of patient information on a scale from 0% (worst) to 100% (best) by assessing the domains of completeness, appearance, understandability, and usefulness (Supplemental Appendix S1).
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
      Each question has 3 possible answers with an accompanying score of 1 (yes), 0.5 (partly), or 0 (no). The total EQIP score was determined by averaging individual items’ scores using the following formula: [((# of Yes∗1) + (# of Partly∗0.5) + (# of No∗0))/20 – (# of does not apply))] ∗100 = % score.
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
      Domains that were not applicable to a given resource were excluded from the overall score calculation. Two reviewers (F.Y. and S.L.) evaluated each resource using the EQIP score, with a discrepancy between reviewers defined as a difference of ≥ 10% in EQIP score. Discrepant scores, which could occur due to the subjective nature of some of the questions, were reviewed by the 2 reviewers in order to reach a consensus, with disagreements resolved by a third author (R.T.). Following resolution of discrepant scores, the 2 EQIP scores calculated for each resource were averaged. This final averaged score was used in the analysis and to categorize each resource as one of the following: (i) well written, high-quality (score of 76%-100%); (ii) good quality with minor problems (score of 51%-75%); (iii) serious problems with quality (score of 26%-50%); and (iv) severe problems with quality (score of 0%-25%), consistent with categories recommended in the original EQIP development publication.
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.

      Secondary outcome: readability

      Readability was assessed primarily using the Flesch-Kincaid Grade Level score, and secondarily, via the Flesch Reading Ease score, by copying the text of each resource into a Microsoft Word 2016 (Microsoft, Redmond, WA) document and using the software’s built-in readability function. The Flesch-Kincaid Grade Level and the Flesch Reading Ease scores have previously been validated and used to measure the readability of patient educational resources.
      • Iacovetto M.C.
      • Matlock D.D.
      • McIlvennan C.K.
      • et al.
      Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
      ,
      • Dalziel K.
      • Leveridge M.J.
      • Steele S.S.
      • Izard J.P.
      An analysis of the readability of patient information materials for common urological conditions.
      ,
      • Kher A.
      • Johnson S.
      • Griffith R.
      Readability assessment of online patient education material on congestive heart failure.
      In the US, an estimated 61% of adults read at a 6th-grade level,
      • Hameed I.
      • Hameed N.U.F.
      • Oakley C.T.
      • et al.
      Systematic assessment of online health information for coronary revascularization.
      and it is therefore recommended that patient information resources be written at a 6th- grade or lower reading level.
      • Dalziel K.
      • Leveridge M.J.
      • Steele S.S.
      • Izard J.P.
      An analysis of the readability of patient information materials for common urological conditions.
      ,
      • Kher A.
      • Johnson S.
      • Griffith R.
      Readability assessment of online patient education material on congestive heart failure.
      ,

      Weiss BD. Removing barriers to better, safer care. Health literacy and patient safety: Help patients understand. In: Manual for Clinicians, 2nd ed. 2007. Available at: http://www.partnershiphp.org/Providers/HealthServices/Documents/Health%20Education/CandLToolKit/2%20Manual%20for%20Clinicians.pdf. Accessed August 26, 2021.

      For this study, HF medication patient educational resources with a higher than 6th-grade reading level were considered to have low readability. We further used a Flesch Reading Ease score < 80 to indicate low readability.
      • Kher A.
      • Johnson S.
      • Griffith R.
      Readability assessment of online patient education material on congestive heart failure.
      ,
      • Edmunds M.R.
      • Barry R.J.
      • Denniston A.K.
      Readability assessment of online ophthalmic patient information.
      ,
      • Flesch R.
      A new readability yardstick.
      Lastly, we collected data on each resource’s total word count, for descriptive purposes.

      Statistical analysis

      We performed descriptive analyses, and we report data using medians and interquartile ranges (IQRs) for continuous variables, and frequencies and percentages for categorical variables. All analyses were performed using Microsoft Excel 2016 (Microsoft, Redmond, WA).

      Results

      From 693 search results, we identified 39 HF medication educational resources that met our eligibility criteria (Fig. 1). Despite the methods employed and the fact that the searches conducted were identical, F.Y.’s and S.L.’s search results differed slightly, likely due to differences in geographic location (Alberta vs British Columbia). Key characteristics of HF medication educational resources are described individually in Table 1, and summary statistics are provided in Table 2.
      Figure thumbnail gr1
      Figure 1Study flow diagram. HF, heart failure.
      Table 1Individual characteristics and quality assessment of 39 HF medication educational resources, in descending order of EQIP score
      Resource titleDeveloper/ funding sourceEQIP Score, %Flesch-Kincaid Grade LevelWord count
      A Marvellous Guide to Medicines for Heart Failure https://pumpingmarvellous.org/wp-content/uploads/2018/05/Heart-Failure-Medicine-Guide.pdfPumping Marvellous798.69039
      Patient Education: Heart Failure (Beyond the Basics)

      https://www.uptodate.com/contents/heart-failure-beyond-the-basics
      UpToDate7694778
      Heart Failure - A Guide for Patients and Families

      https://www.ottawaheart.ca/sites/default/files/uploads/heart-failure-patient-guide.pdf
      University of Ottawa Heart Institute7387571
      Heart Failure

      http://www.cardiacbc.ca/health-info/heart-conditions/heart-failure
      Cardiac Services British Columbia7281020
      Living With Heart Failure

      https://www.heartandstroke.ca/-/media/pdf-files/canada/health-information-catalogue/en-living-with-heart-failure.pdf
      Heart & Stroke Foundation71620,853
      Heart Failure Patient Education Handbook

      https://www.asante.org/app/files/public/2134/Heart-Failure-Handbook.pdf
      Asante717.54717
      Heart Failure Diagnosis & Treatment

      https://www.mayoclinic.org/diseases-conditions/heart-failure/diagnosis-treatment/drc-20373148
      Mayo Clinic7194518
      Digoxin: A Medicine for Heart Problems

      https://familydoctor.org/digoxin-a-medicine-for-heart-problems/?adfree=truetreat-heart-failure
      familydoctor.org706.5929
      Heart Failure Medications

      https://www.ottawaheart.ca/heart-failure-patient-guide/heart-failure-medications
      University of Ottawa Heart Institute69101638
      Medications to Manage Heart Failure

      https://www.health.qld.gov.au/__data/assets/pdf_file/0030/993009/Medications-booklet-web-version.pdf
      Queensland Government69103068
      A Decision Aid for Entresto

      https://www.cardiosmart.org/docs/default-source/assets/decision-aid/heart-failure-drug-options.pdf?sfvrsn=aaff9c98_1
      American College of Cardiology (CardioSmart)698.71024
      Introduction to Medications

      https://ourhearthub.ca/medications/
      Ted Rogers Institute for Heart Research678.1759
      Heart Failure

      https://myhealth.alberta.ca/Health/pages/conditions.aspx?hwid=hw44415#tp17546
      Alberta Health Services676.26062
      Congestive Heart Failure

      https://patient.info/heart-health/heart-failure-leaflet
      patient.Info658.72032
      What Medications Are Used to Treat Heart Failure?

      https://heart-failure.net/medications
      heartfailure.net6511.31077
      A Quick Guide to Living With Heart Failure

      https://www.novartis.com/sites/novartis_com/files/nvs-hf-patient-booklet.pdf
      Novartis / Pumping Marvellous6484070
      Heart Failure Medications

      https://my.clevelandclinic.org/departments/heart/patient-education/recovery-care/heart-failure/medications
      Cleveland Clinic646.42059
      Managing Heart Failure

      https://www.coeuretavc.ca/-/media/pdf-files/canada/health-information-catalogue/en-managing-heart-failure-v3.ashx
      Heart & Stroke Foundation627.37400
      What to Expect: Living With Heart Failure

      https://www.upmc.com/-/media/upmc/patients-visitors/education/documents/living-with-heart-failure-booklet.pdf
      University of Pittsburgh Medical Center6185458
      Commonly Prescribed Heart Failure Medications

      https://heartfailureoxford.org.uk/patient/heart-failure-medications/
      Heart Failure Oxfordshire619.51531
      Discharge Packet for Patients Diagnosed With Heart Failure

      https://www.heart.org/-/media/files/health-topics/heart-failure/hf-discharge-packet.pdf?la=en&hash=90463681A07EE6230276BC27A08F5D337D1D6D8C
      American Heart Association60611,630
      Medicines for Congestive Heart Failure

      https://healthy.kaiserpermanente.org/washington/health-wellness?item=/common/healthAndWellness/conditions/heartDisease/chfMedications.html
      Kaiser Permanente589.21045
      Everything You Need to Know About Heart Failure Medications

      https://www.healthline.com/health/heart-failure/heart-failure-medications
      Healthline58101983
      About Heart Failure

      https://www.cardiomyopathy.org/about-cardiomyopathy/heart-failure
      Cardiomyopathy.org5810.2890
      CHF Booklet

      https://www.meritushealth.com/documents/CHF-booklet.pdf
      Meritus Health5695230
      Heart Failure

      https://www.farxiga.com/heart-failure
      AstraZeneca56103081
      HF Patient Education Booklet

      http://www.gov.pe.ca/photos/original/hpei_cp_hf_book.pdf
      Prince Edward Island Government556.74078
      Heart Failure Handbook

      https://www.thechristhospital.com/Documents/Our%20Services/Heart%20Failure%20Handbook.pdf
      The Christ Hospital5572120
      Heart Failure—Treatment

      https://www.nhs.uk/conditions/heart-failure/treatment/
      United Kingdom National Health Services54111933
      10 Drugs Commonly Prescribed for Heart Failure

      https://www.healthgrades.com/right-care/heart-failure/10-drugs-commonly-prescribed-for-heart-failure
      HealthGrades5481428
      Self-Care Guide for the Heart Failure Patient
      • White M.F.
      • Kirschner J.
      • Hamilton M.A.
      Self-care guide for the heart failure patient.
      American Heart Association549.61020
      Heart Failure Medicines

      https://www.heartfailurematters.org/what-your-doctor-can-do/heart-failure-medicines/
      heartfailuremattters.org5310437
      Treatment of Heart Failure?

      https://www.aahfn.org/mpage/treatement_hf
      American Association of Heart Failure Nurses507.52089
      Understanding Heart Failure: Answers to Common Questions

      https://www.med.umich.edu/1libr/CCG/HeartFailure.pdf
      Michigan Medicine497.51328
      Congestive Heart Failure Medications

      https://www.medicinenet.com/congestive_heart_failure_medications/drug-class.htm
      MedicineNet49121328
      Heart Failure

      https://www.nhlbi.nih.gov/health/heart-failure
      National Heart Lung and Blood Institute4985269
      Heart Failure Medicines

      https://myhealth.alberta.ca/Alberta/AlbertaDocuments/Heart-Failure-Medicines-Oct-2019.pdf
      Alberta Health Services467.3886
      Medications

      https://www.cardiosmart.org/topics/heart-failure/treatment/medications
      American College of Cardiology (CardioSmart)388.4313
      Heart Failure Treatments

      https://www.ucsfhealth.org/conditions/heart-failure/treatment
      University of California San Francisco Health31111272
      EQIP scores range from 0% (worst) to 100% (best).
      CHF, congestive heart failure; EQUIP, Ensuring Quality Information for Patients; HF, heart failure.
      Table 2Summary characteristics and quality assessment of heart failure medication educational resources
      CharacteristicValueFirst and third quartile (range)
      Word count20321045 to 4778 (313 to 20,853)
      Format
      Webpage25 (64)
      PDF14 (36)
      Country of origin
      USA22 (56)
      United Kingdom7 (18)
      Canada9 (23)
      Australia1 (3)
      EQIP, median6154 to 69 (31 to 79)
      High quality2 (5)
      Good quality, minor problems30 (77)
      Serious problems7 (18)
      Severe problems0
      Readability
      Flesch-Kincaid Grade Level, median88 to 10 (6 to 12)
      ≤6th grade4 (10)
      Flesch Reading Ease, median5952 to 65 (36 to 72)
      ≥ 800
      Values are n (%), unless otherwise indicated.
      EQIP, Ensuring Quality Information for Patients score; PDF, portable document format.

      EQIP

      The median EQIP score was 61% (IQR 54% to 68%; Table 2; Fig. 2A). Of the 39 included educational resources, 2 (5%) were of high quality, 30 (77%) were of good quality with minor problems, and 7 (18%) had serious problems with quality. Assessment of the specific components of the total EQIP score revealed that 36 resources (92%) used lay terms, 38 (97%) addressed the reader personally, and 39 (100%) had a respectful tone. However, only 2 resources (5%; including the highest-scoring resource) reported consulting with patients/family members during development. In terms of medication-specific information, 38 resources (97%) addressed the purpose of described HF medications, 28 (72%) qualitatively described benefits (eg, decrease mortality), 28 (72%) qualitatively described the medication risks and side-effects, and 21 (54%) addressed alternative medications (eg, angiotensin-receptor blocker if cough develops while taking an angiotensin-converting enzyme inhibitor). Further, 34 resources (87%) used generic names (instead of, or in addition to, brand names) and distinguished brand names as such, whereas the remaining 5 resources (13%) only discussed medication classes. Lastly, 24 resources (62%) did not address quality-of-life issues, with the remaining 15 (38%) only partly addressing this item. Only 5 resources (13%) included visual content to supplement the text.
      Figure thumbnail gr2
      Figure 2Boxplot of (A) Ensuring Quality Information for Patients (EQIP) score and (B) Flesch-Kincaid Grade Level for 39 heart failure medication educational resources.

      Readability

      The median Flesch-Kincaid Grade Level was an 8th-grade reading level (IQR 8th to 10th grade) (Fig. 2B). Only 4 resources (10%) achieved the recommended 6th-grade reading level, and no resource had a 5th-grade or lower reading level. The median Flesch Reading Ease score was 59 (IQR 52 to 65), with all resources demonstrating low readability based on this score.

      Discussion

      HF is a chronic, complex condition that can be treated using a variety of healthy behaviours, medical devices, and medications.
      • McDonald M.
      • Virani S.
      • Chan M.
      • et al.
      CCS/CHFS heart failure guidelines update: defining a new pharmacologic standard of care for heart failure with reduced ejection fraction.
      Provision of written resource for patients to review on their own time and at their own pace can significantly improve information retention, compared to verbal instruction.
      • Boyde M.
      • Peters R.
      Education material for heart failure patients: what works and what does not?.
      ,
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
      In this study, we found that most online HF medication patient educational resources were of good quality with minor problems, based on the EQIP tool. However, most HF medication educational resources were written at a grade level beyond the reading level of most people in the general population.
      We identified several key areas for improvement among HF medication patient educational resources. In general, resources had a respectful tone, used generic names of medications, presented information in a logical order, and qualitatively described the purpose, side-effects, benefits (though not quantitatively,) and alternatives of medications. However, few followed a summative flow (ie, outlining the document content and covering what is stated), involved patients and caregivers in their development, or addressed quality-of-life issues.
      Having a clear definition outlining the document content can help patients, caregivers, and healthcare providers understand whether the resource is suitable for the patient’s specific informational needs.
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
      ,
      • Coulter A.
      • Entwistle V.A.
      • Gilbert D.
      Informing patients: an assessment of the quality of patient information materials.
      ,
      • Department of Health
      Toolkit for Producing Patient Information.
      Next, patients and healthcare providers may have different perceptions of what information is important to convey in educational materials.
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
      ,
      • Coulter A.
      • Entwistle V.A.
      • Gilbert D.
      Informing patients: an assessment of the quality of patient information materials.
      ,
      • Perkins L.
      Developing a tool for health professionals involved in producing and evaluating nutrition education leaflets.
      ,
      • Coulter A.
      Evidence based patient information.
      Therefore, involving patients and caregivers in development of resources can help tailor educational materials to the needs of patients, making them more useful and relevant to their target audience. Further, describing the quality-of-life and practical impacts of medications ensures resource transparency, enables realistic patient expectations,
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
      ,
      • Morris K.C.
      Psychological distress in carers of head injured individuals: the provision of written information.
      and allows patients to better understand and implement self-care and management strategies.
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
      ,
      • Department of Health
      Toolkit for Producing Patient Information.
      A recent study established a framework for patient-oriented practical issues to be included in evidence summaries and shared decision-making tools.
      • Heen A.F.
      • Vandvik P.O.
      • Brandt L.
      • et al.
      A framework for practical issues was developed to inform shared decision-making tools and clinical guidelines.
      Such practical considerations align closely with quality-of-life issues incorporated into the EQIP questionnaire (eg, how a treatment impacts a patient’s daily routine or social life) and should be included in HF medication educational resources.
      • Moult B.
      • Franck L.S.
      • Brady H.
      Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
      ,
      • Heen A.F.
      • Vandvik P.O.
      • Brandt L.
      • et al.
      A framework for practical issues was developed to inform shared decision-making tools and clinical guidelines.
      These details are also more likely to align with patients’ individual goals of therapy (eg, improve symptoms, daily functioning, capacity for social interaction, etc.) and are crucial to include in patient education to engage patients in their care and medication management.
      • McHorney C.A.
      • Mansukhani S.G.
      • Anatchkova M.
      • et al.
      The impact of heart failure on patients and caregivers: a qualitative study.
      Our search identified resources with a wide range of readability, with few resources achieving sufficient readability. The readability level of HF medication educational resources thereby poses a problem, as patients may not be able to fully comprehend the included information, regardless of its quality. An additional consideration for barriers to use of educational resources is the length of time required to read through them. The average adult reads at a pace of approximately 250 words/minute.
      • Brysbaert M.
      How many words do we read per minute? A review and meta-analysis of reading rate.
      Of the top 5 highest-scoring resources based on the EQIP tool, the median word count was 4778, indicating approximately 19 minutes of reading time. For the average patient, who may be new to their diagnosis and receiving an overwhelming amount of information, resource length is likely a substantial barrier to absorbing and implementing the included information.
      • McHorney C.A.
      • Mansukhani S.G.
      • Anatchkova M.
      • et al.
      The impact of heart failure on patients and caregivers: a qualitative study.

      Limitations

      Limitations to this study warrant discussion. First, we restricted resource inclusion to written materials available online. We did not capture information presented within social media communities (eg, Reddit, Facebook, etc.), video resources (eg, YouTube), or local/clinic-specific resources that are not available online. We anticipate greater variability of quality of these resources, as they may be subject to less curation. Second, the EQIP instrument was open to interpretation, which resulted in initial variability in scoring between the 2 reviewers. However, we mitigated this variability by incorporating duplicate scoring and an a priori plan to identify and resolve discrepancies. Third, the EQIP questionnaire used for our primary outcome defines quality from the perspective of healthcare professionals, rather than from the perspective of the end-user (ie, the patient). Therefore, it may not encompass all domains of quality or treatment considerations that are important to patients, or capture the relative value that patients place on each domain.

      Conclusions

      Most HF medication educational resources available on the Internet are of acceptable educational quality, but could readily be improved. Most resources were beyond the recommended reading grade level for educational resources, limiting their utility for patients with a low literacy level.

      Funding Sources

      The authors have no funding sources to declare.

      Disclosures

      The authors have no conflicts of interest to disclose.

      Supplementary Material

      References

        • Government of Canada
        Canadian Chronic Disease Surveillance System (CCDSS). Available at:.
        • McDonald M.
        • Virani S.
        • Chan M.
        • et al.
        CCS/CHFS heart failure guidelines update: defining a new pharmacologic standard of care for heart failure with reduced ejection fraction.
        Can J Cardiol. 2021; 37: 531-546
        • McHorney C.A.
        • Mansukhani S.G.
        • Anatchkova M.
        • et al.
        The impact of heart failure on patients and caregivers: a qualitative study.
        PloS One. 2021; 16 (e0248240)
        • Boyde M.
        • Peters R.
        Education material for heart failure patients: what works and what does not?.
        Curr Heart Fail Rep. 2014; 11: 314-320
        • Moult B.
        • Franck L.S.
        • Brady H.
        Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.
        Health Expect. 2004; 7: 165-175
        • van Weel C.
        More self reliance in patients and fewer antibiotics: still room for improvement.
        BMJ. 2002; 324: 94
        • Nassiri M.
        • Mohamed O.
        • Berzins A.
        • et al.
        Surfing behind a boat: quality and reliability of online resources on scaphoid fractures.
        J Hand Surg Asian Pac Vol. 2016; 21: 374-381
        • Iacovetto M.C.
        • Matlock D.D.
        • McIlvennan C.K.
        • et al.
        Educational resources for patients considering a left ventricular assist device: a cross-sectional review of Internet, print, and multimedia materials.
        Circ Cardiovasc Qual Outcomes. 2014; 7: 905
        • Hameed I.
        • Hameed N.U.F.
        • Oakley C.T.
        • et al.
        Systematic assessment of online health information for coronary revascularization.
        JAMA Intern Med. 2021; 181: 1003-1006
        • Hazelton G.
        • Al-Khatib Sears S.
        • et al.
        Assessment of the quality of existing patient educational tools focused on sudden cardiac arrest: a systematic evaluation by the Sudden Cardiac Arrest Thought Leadership Alliance.
        Patient Prefer Adherence. 2013; 7: 361-368
        • Engelmann J.
        • Fischer C.
        • Nkenke E.
        Quality assessment of patient information on orthognathic surgery on the Internet.
        J Craniomaxillofacial Surg. 2020; 48: 661-665
        • Dalziel K.
        • Leveridge M.J.
        • Steele S.S.
        • Izard J.P.
        An analysis of the readability of patient information materials for common urological conditions.
        Can Urol Assoc J. 2016; 10: 167-170
        • Kher A.
        • Johnson S.
        • Griffith R.
        Readability assessment of online patient education material on congestive heart failure.
        Adv Prev Med. 2017; 2017: 9780317
        • Lee K.S.
        • Cho Y.M.
        • Oh S.H.
        • Jung M.S.
        • Yoon J.Y.
        Evaluation of the heart failure in Internet patient information: descriptive survey study.
        Int J Environ Res Public Health. 2021; 18: 1047
        • NetMarketShare
        Market share statistics for Internet technologies. Available at:.
        www.netmarketshare.com
        Date accessed: September 11, 2021
        • statcounter
        Search engine market share worldwide. Available at:.
        • Eysenbach G.
        • Kohler C.
        How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews.
        BMJ. 2002; 324: 573-577
        • Aldairy T.
        • Laverick S.
        • McIntyre G.T.
        Orthognathic surgery: Is patient information on the Internet valid?.
        Eur J Orthod. 2012; 34: 466-469
        • Karamitros G.A.
        • Kitsos N.A.
        Clefts of the lip and palate: is the Internet a trustworthy source of information for patients?.
        Int J Oral Maxillofac Surg. 2018; 47: 1114-1120
      1. Weiss BD. Removing barriers to better, safer care. Health literacy and patient safety: Help patients understand. In: Manual for Clinicians, 2nd ed. 2007. Available at: http://www.partnershiphp.org/Providers/HealthServices/Documents/Health%20Education/CandLToolKit/2%20Manual%20for%20Clinicians.pdf. Accessed August 26, 2021.

        • Edmunds M.R.
        • Barry R.J.
        • Denniston A.K.
        Readability assessment of online ophthalmic patient information.
        JAMA Ophthalmol. 2013; 131: 1610-1616
        • Flesch R.
        A new readability yardstick.
        J Appl Psychol. 1948; 32: 221-233
        • Coulter A.
        • Entwistle V.A.
        • Gilbert D.
        Informing patients: an assessment of the quality of patient information materials.
        BMJ. 1999; 318: 1494
        • Department of Health
        Toolkit for Producing Patient Information.
        HMSO, London2002
        • Perkins L.
        Developing a tool for health professionals involved in producing and evaluating nutrition education leaflets.
        J Human Nutr Diet. 2001; 13: 41-49
        • Coulter A.
        Evidence based patient information.
        BMJ. 1998; 317: 225-226
        • Morris K.C.
        Psychological distress in carers of head injured individuals: the provision of written information.
        Brain Injury. 2001; 15: 239-254
        • Heen A.F.
        • Vandvik P.O.
        • Brandt L.
        • et al.
        A framework for practical issues was developed to inform shared decision-making tools and clinical guidelines.
        J Clin Epidemiol. 2021; 129: 104-113
        • Brysbaert M.
        How many words do we read per minute? A review and meta-analysis of reading rate.
        J Mem Lang. 2019; 109: 104047
        • White M.F.
        • Kirschner J.
        • Hamilton M.A.
        Self-care guide for the heart failure patient.
        Circulation. 2014; 129: e293-e294