Fikirte Wagaw, MPH1; Catherine A. Okoro, PhD2; Sunkyung Kim, PhDthree; Jessica Park, MPH1; Fred Rachman, MD1 (View writer affiliations)
Suggested quotation for this text: Wagaw F, Okoro CA, Kim S, Park J, Rachman F. Linking Data From Health Surveys and Electronic Health Records: A Demonstration Project in Two Chicago Health Center Clinics. Prev Chronic Dis 2018;15:170085. DOI: http://dx.doi.org/10.5888/pcd15.170085.
Monitoring and understanding inhabitants well being requires conducting well being-associated surveys and surveillance. The goal of our research was to evaluate whether or not knowledge from self-administered surveys could possibly be collected electronically from sufferers in city, main-care, security-internet clinics and subsequently linked and in contrast with the identical sufferers’ digital well being data (EHRs).
Data from self-administered surveys have been collected electronically from a comfort pattern of 527 sufferers at 2 Chicago well being facilities from September by November, 2014. Survey knowledge have been linked to EHRs.
A whole of 251 (47.6%) sufferers who accomplished the survey consented to having their responses linked to their EHRs. Consenting contributors have been older, extra prone to report honest or poor well being, and took longer to finish the survey than those that didn’t consent. For eight of 18 categorical variables, general share of settlement between survey knowledge and EHR knowledge exceeded 80% (intercourse, race/ethnicity, pneumococcal vaccination, self-reported physique mass index [BMI], diabetes, hypertension, medicine for hypertension, and hyperlipidemia), and of those, the extent of settlement was good or wonderful (κ ≥Zero.64) apart from pneumococcal vaccination (κ = Zero.40) and hyperlipidemia (κ = Zero.47). Of 7 steady variables, settlement was substantial for age and weight (concordance coefficients ≥Zero.95); nonetheless, aside from calculated survey BMI and EHR–BMI (concordance coefficient = Zero.88), all different steady variables had poor settlement.
Self-administered and internet-primarily based surveys could be accomplished in city, main-care, security-internet clinics and linked to EHRs. Linking survey and EHR knowledge can improve public well being surveillance by validating self-reported knowledge, finishing gaps in affected person knowledge, and lengthening pattern sizes obtained by present strategies. This strategy would require selling and sustaining affected person involvement.
Monitoring and understanding inhabitants well being requires conducting well being-associated surveys and surveillance. The Behavioral Risk Factor Surveillance System (BRFSS), for instance, is a state-primarily based system of phone surveys that accumulate knowledge on well being-threat behaviors, continual circumstances, use of preventive companies, and well being-associated high quality of life (HRQoL) of adults (1). BRFSS could be modified to evaluate rising and pressing well being points and offers knowledge on measures sometimes unrecorded in the medical setting (eg, train, HRQoL, well being attitudes, consciousness, well being data) (2,three). Searching for brand new knowledge sources is vital, nonetheless, as a result of inhabitants-primarily based surveys could be pricey and time-consuming and will produce biased outcomes which are laborious to generalize (1,four–12).
Expanded use of digital well being data (EHRs) — full with acceptable safety of affected person confidentiality — might help enhance the design and supply of public well being interventions and medical care; knowledge in EHRs can be utilized to assist discover new causes of infectious illness and to handle outbreaks by triggering public well being alerts, offering suggestions to clinicians, and enhancing communications between public well being practitioners and medical organizations (11–13). Additionally, EHRs might help establish sufferers needing medical care, illness administration, preventive well being companies, and behavioral counseling (2,three,14–17). EHRs can even assist management rising well being care prices by eliminating pointless assessments, procedures, and prescriptions (17).
EHRs could assist enhance affected person care and inhabitants well being when linked to survey knowledge and different details about well being-associated habits, HRQoL, and particulars about working and dwelling circumstances (2,three,18). For folks managing a continual sickness, for instance, the EHR can validate responses, as a result of survey solutions could be linked to recorded medical occasions. Likewise, behaviors (eg, train) recorded in a current survey may set off alerts and suggestions again by the EHR. Inclusion of affected person-reported measures in EHRs can improve affected person-centered care, affected person well being, and capability to conduct inhabitants-primarily based analysis (2,three).
The goal of this research was to discover the feasibility of electronically gathering self-administered affected person survey knowledge in city, main-care, security-internet clinics and subsequently linking and evaluating that knowledge with sufferers’ EHR knowledge.
Alliance of Chicago Community Health Services (Alliance; http://alliancechicago.org/) is a federal Health Center Controlled Network. Alliance approached four of its community well being facilities about undertaking participation, choosing them for his or her giant affected person quantity, various geographic places, distinct and various affected person populations, and historical past of participation in new initiatives. Although three well being facilities authorised the undertaking, just one was in a position to take part in the undertaking’s timeframe. We carried out our research in 2 of that well being middle’s clinics, and it was authorised by that clinic’s analysis overview committee.
We recruited clinic sufferers aged 18 years or older through the use of fliers and bulletins in ready areas and in check-in procedures. Survey directors used standardized scripts to summarize the survey’s objectives for sufferers. Participants reviewed an digital consent type and acquired a tough copy of the shape; they offered separate knowledgeable consent for survey participation and for subsequent survey–EHR linkage. Each survey participant acquired a modest incentive (no matter consent to EHR linkage). Survey directors have been accessible to help sufferers all through knowledge assortment.
From September 2014 by November 2014, a comfort pattern of 527 sufferers accomplished the self-administered, internet-primarily based survey on numerous manufacturers of digital tablets, desktop computer systems, and mobile telephones. Tablet knowledge plans have been bought to reduce affect on well being middle assets and to reduce knowledge connectivity points.
Questions from the Illinois BRFSS (http://app.idph.state.il.us/brfss/) have been used to gather info on sufferers’ sociodemographic traits, well being behaviors, continual circumstances, receipt of preventive care companies, and medical care. Questions associated to continual circumstances have been chosen on the idea of their potential to be matched to knowledge accessible in the EHR. Questions on medicine use, laboratory findings, and blood stress readings helped us examine knowledge on self-reported continual circumstances with EHR content material. The variety of questions every participant was requested was decided by intercourse (eg, intercourse-particular preventive care companies), age (eg, age-particular most cancers screenings), and survey responses that decided query branching. The survey took a median of 20 to 30 minutes to finish and was hosted through the use of the Survey Analytics Online Survey Platform (Survey Analytics LLC).
Of 527 survey contributors, 47% (n = 251) consented to have their survey responses linked to their EHR; 99% (n = 248) of those consenting sufferers had an EHR. At the top of the survey and EHR extraction, 2 de-recognized analytic knowledge units have been created: 1) a set that contained solely the survey knowledge of sufferers who didn’t consent to the EHR linkage and a pair of) a set that contained the survey and EHR knowledge of sufferers who consented to EHR linkage.
When doable, variations in categorical variable development between survey knowledge and EHR knowledge have been resolved by collapsing the unique classes to type a typical metric. Continuous variables apart from blood stress have been constructed equally in the survey instrument and the EHR. Patients reporting well being care skilled stated that they had hypertension (HBP) have been requested to enter their systolic and diastolic blood stress. For the EHR abstraction, the final three systolic and diastolic blood stress readings have been taken, and the imply systolic and diastolic pressures have been calculated. Self-reported weight and peak have been assessed utilizing 2 survey questions: “About how much do you weigh without shoes?” and “About how tall are you without shoes?” Patients have been categorised as underweight (physique mass index [BMI, kg/m2] <18.5), regular weight (BMI 18.5–<25), obese (BMI 25–<30), or overweight (BMI ≥30). Self-classified BMI was assessed with the survey query, “Would you classify your weight as low (underweight), normal weight, overweight, or obese?”
We calculated the distribution of the research inhabitants by survey period, sociodemographic traits, and self-rated well being standing, general and by consent to EHR linkage. For categorical variables, we used the χ2 take a look at to evaluate vital variations between those that agreed to survey–EHR linkage and those that didn’t. For steady variables, we used the t take a look at to evaluate vital imply variations between the two affected person teams. To assess concordance between survey knowledge and EHR knowledge, we examined 248 sufferers who consented to the EHR linkage and for whom an EHR file was discovered. For categorical variables, we utilized Cohen’s (19) κ coefficient with four predefined settlement ranges: wonderful settlement (κ ≥Zero.9), good settlement (κ ≥Zero.6 to κ <Zero.9), honest settlement (κ ≥Zero.three to κ <Zero.6), and poor settlement (κ <Zero.three). Because we noticed some instances which will belong to the κ paradox (20), we additionally calculated general settlement in share (= 100 × the variety of concordant counts/the full pattern measurement). For steady variables, we utilized Lin’s (21,22) concordance correlation coefficient (ρc) with four predefined settlement ranges: nearly excellent (ρc >Zero.99), substantial (ρc ≥Zero.95 to ρc ≤Zero.99), reasonable (ρc ≥Zero.90 to ρc<Zero.95), and poor (ρc <Zero.90) (23). For all analyses, P < .05 was thought of vital, and knowledge have been analyzed in SAS model 9.three (SAS Institute, Inc).
Participant ages ranged from 18 to 87 years (imply, 43.four y; customary deviation [SD], 14.7 y) (Table 1). The pattern was predominantly non-Hispanic black (90.four%), feminine (70.four%), by no means married or a member of an single couple (61.2%), spoke English as their main language (96.three%), and had Medicaid or Medicare as main medical health insurance protection (69.1%). More than 70% reported their well being standing as wonderful, excellent, or good, and 62.7% reported no incapacity. Most had annual family incomes lower than $20,00Zero, rented their main residence, and had no kids in the family.
Seven well being behaviors of the comfort pattern have been distributed as follows: at all times or practically at all times sporting a seat belt (90.Zero%), watching or lowering sodium consumption (60.2%), consuming a number of drinks of alcohol in the previous 30 days (59.6%), participating in leisure-time bodily exercise (58.three%), consuming 5 or extra servings day by day of vegatables and fruits (42.2%), presently smoking cigarettes (35.four%), and growing medicine use in the previous 30 days with out the recommendation of a well being care skilled (eight.9%) (Figure).
Self-reported well being behaviors of a comfort pattern, research linking self-reported survey knowledge with digital well being file knowledge, 2 Chicago well being clinics, 2014. [A tabular model of this determine can also be accessible.]
Compared with sufferers who didn’t conform to having their survey outcomes linked to their EHRs, those that agreed have been older (imply 45.four y vs 41.5 y, P = .Zero03), extra prone to report honest or poor well being (32% vs 24%, P = .03), and took longer to finish the digital survey (27.eight minutes vs 21.three minutes, P < .01) (Table 1).
Of the 18 categorical variables we examined, general settlement between the survey and EHR knowledge exceeded 80% for eight variables (intercourse, race/ethnicity, pneumococcal vaccination, self-reported BMI, diabetes, HBP, medicine for HBP, hyperlipidemia), and of those, the extent of settlement was good or wonderful (κ ≥ Zero.64) apart from pneumococcal vaccination (κ = Zero.40) and hyperlipidemia (κ = Zero.47) (Table 2). Race/ethnicity and diabetes had a share settlement above 91% however decrease κ statistics values of Zero.65 (95% confidence interval [CI], Zero.45–Zero.85) and Zero.76 (95% CI, Zero.66–Zero.87), respectively. Self-classified BMI confirmed the bottom degree of concordance (general settlement = 19.5%, κ = Zero.16).
Of the 7 steady variables we examined, the settlement degree between the survey and EHR knowledge was substantial for age (= Zero.95; 95% CI, Zero.94–Zero.96) and weight (= Zero.98; 95% CI, Zero.97–Zero.98) (Table three). With the exception of BMI (= Zero.88; 95% CI, Zero.84–Zero.91), all different steady variables had poor settlement; diastolic blood stress amongst sufferers who reported hypertension had the bottom settlement (= Zero.28; 95% CI, Zero.13–Zero.41).
This research in contrast outcomes from a self-administered internet-primarily based survey with de-recognized affected person knowledge from EHRs in an city main-care setting. We discovered a passable diploma of concordance between survey knowledge and EHR knowledge for nonmodifiable demographic traits and for some well being-associated measures: diabetes, HBP, HBP medicine, weight, and calculated categorical and steady BMI. We discovered decrease ranges of concordance for modifiable sociodemographic traits, pneumococcal vaccination, hyperlipidemia, self-categorised BMI, hemoglobin A1c amongst sufferers reporting diabetes, and blood stress amongst sufferers reporting hypertension ― particularly diastolic stress. EHR knowledge on self-reported well being-threat behaviors have been unavailable for comparability; knowledge on tobacco use screening have been accessible.
Fewer than half the surveyed sufferers gave EHR linkage consent; these consenting confirmed vital variations from those that didn’t. Similar to different researchers’ findings (24), these consenting have been older and extra prone to report honest or poor well being. Unlike different analysis findings (24), nonetheless, we didn’t discover vital variations by intercourse, employment standing, or sort of medical health insurance protection. Further investigation into what elements could improve consent or improve affected person engagement may assist undertaking sustainability and representativeness of the affected person inhabitants.
Generally, our concordance findings have been according to research which have used related strategies (7,25,26). Our degree of settlement was just like earlier analysis assessing knowledge high quality between ambulatory medical file knowledge and affected person survey knowledge for diabetes and BMI, however we had a better degree of concordance for HBP, HBP medicine, and hyperlipidemia and a decrease degree of concordance for lipid-reducing medicine (26). Additionally, we had substantial settlement for weight and, in distinction, poor settlement for peak. We additionally discovered good general settlement for BMI primarily based on self-reported peak and weight (86%) however poor general settlement for self-categorised BMI (20%). Studies present folks usually overestimate their peak and underestimate their weight and BMI (6). This reporting bias varies, nonetheless, by the demographic traits of the research inhabitants (eg, intercourse, race, age). For instance, males usually tend to exaggerate their peak than ladies are. Our comfort pattern was predominantly feminine, non-Hispanic black, and aged 45 to 64 years. Differences between self-report and direct measures may be because of the respective inhabitants’s sociocultural perceptions of physique weight and could also be biased by social desirability (6). Our outcomes reveal the necessity for direct measures that validate self-reported knowledge, as a result of sufferers have been extra prone to understand themselves as in a decrease BMI class than their calculated BMI class confirmed. Our outcomes may mirror a lack of understanding of their BMI and, consequently, larger threat of poor well being outcomes. Further analysis is required to totally perceive and tackle this discovering (eg, improved affected person–supplier communication, weight problems screening and intervention). Because our outcomes aren’t generalizable to the well being middle’s affected person inhabitants or to different affected person populations (comfort pattern/obvious choice bias), interpretation must be achieved with care.
Survey and EHR knowledge could have poor concordance for a lot of causes and will present the place every knowledge supply might help enhance the accuracy and completeness of affected person and inhabitants knowledge. When survey and EHR medical measures aren’t concordant, EHR knowledge are typically extra correct than survey knowledge as a result of biases related to self-report differ (5–7). For instance, accurately remembering the date of 1’s final tetanus shot or hemoglobin A1c take a look at result’s tough. For modifiable sociodemographic traits, nonetheless, self-reported knowledge are prone to be extra correct than EHR knowledge, as a result of busy well being facilities have few assets or incentives to replace nonclinical knowledge parts. Institutional incentives additionally could affect poor concordance, as when a sliding charge scale may encourage underneath-reporting of revenue or non-public well being care protection or when medical health insurance plans cost larger premiums to customers who smoke (27).
Our research has a number of limitations. First, we used a comfort pattern of sufferers from 2 Chicago well being middle clinics. This pattern choice bias limits our potential to make inferences to the well being middle’s affected person inhabitants throughout all its clinic websites and its comparability to different affected person populations in the world; the pattern was predominantly feminine, non-Hispanic black, single, and low revenue; sufferers had public insurance coverage protection and have been extra prone to have a mobile phone or an electronic mail tackle than a landline phone. As a end result, for public well being surveillance, a number of knowledge assortment modes and knowledge sources could also be wanted to successfully attain and make sure the representativeness of knowledge for inhabitants subgroups. Moreover, public well being professionals and coverage makers should concentrate on subpopulations which are unconnected to the well being care system and whose members have restricted well being data or lack them completely (four). Second, lower than 50% of the sufferers surveyed consented to EHR linkages. Further evaluation of the elements related to consent, and that are amenable to modification, is required to entry the wealth of knowledge accessible in EHRs. Third, evaluation of the linked knowledge discovered some variables with low prevalence that prevented additional evaluation of settlement. Fourth, some variables had good settlement however low κ scores, suggesting that settlement could have occurred by likelihood alone. Finally, neither knowledge supply could also be thought of a gold customary for all objects measured. For instance, survey knowledge could have inherent biases, and EHR knowledge and the info extraction course of could have complexities that aren’t absolutely recognized or accounted for. Nevertheless, these limitations could change over time with significant use of EHRs, developments in well being info applied sciences, and emphasis on high quality and affected person-centered care in addition to implementing new strategies that combine life-style measures into prescribed well being care (eg, prescribed bodily exercise) (2–four,28).
These limitations however, a symbiotic relationship exists between survey knowledge and medical knowledge. Self-reported knowledge are wanted to reinforce medical knowledge for medical companies (eg, immunizations, screenings, behavioral counseling), imaging and different diagnostics, and drugs obtained outdoors of the affected person’s well being middle (2,7). Self-reported measures, though topic to biases, are very important to offering a whole image of affected person well being, as a result of many well being-associated measures is probably not in the EHR (eg, behaviors, HRQoL, well being attitudes, consciousness, data) or up-to-date (eg, modifiable sociodemographic traits) (2,17,29,30). At the identical time, EHRs can be utilized to validate self-reported medical measures and facilitate the event of correction elements that may be utilized to self-reported knowledge in the absence of bodily measurement, which is usually pricey or not doable (6). In unison, the two knowledge sources have the potential to enhance illness administration, cut back prices, and improve two-means knowledge alternate between public well being and medical organizations.
As well being techniques and their info applied sciences proceed to evolve, researchers ought to proceed the seek for excessive-high quality affected person well being knowledge. Doing so might help well being practitioners, public well being professionals, and coverage makers efficiently consider and cut back current well being disparities. Furthermore, public well being coverage and apply could be guided by knowledge science strategies (together with predictive analytics) through the use of mixed knowledge sources. Population-based surveys, EHRs, and different knowledge sources all have a task in offering a extra full image of the well being of all Americans, whereas bettering their well being and entry to care. To this finish, this undertaking demonstrated the feasibility of laptop-assisted assortment of client survey knowledge and matching it to EHR knowledge. This strategy can improve well being info from distinctive, usually underrepresented populations with well being disparities, improve effectivity and breadth of surveillance actions, and enhance validity of goal measures. More analysis is required to advertise and maintain affected person involvement in their well being and well being data, which is significant to the success and sustainability of this strategy.
F.W. supervised the research. C.A.O. offered technical steering, contributed to knowledge evaluation, and drafted and revised the paper. S.Ok. offered statistical steering and contributed to knowledge evaluation. J.P. assisted with the research. F.R. conceived the research. All authors contributed to decoding outcomes and revisions of the manuscript. The authors thank Bruce Steiner, MS, Behavioral Risk Factor Surveillance System Coordinator, Illinois Department of Public Health (IDPH), for his ideas; George Khalil, DrPH, for programming of the net-primarily based survey instrument; and David Flegel, MS, technical author, for his companies. The authors additionally thank Mrs. Berneice Mills-Thomas and the workers at Near North Health Services Corporation for supporting the implementation of this undertaking. We are additionally grateful for the work of the survey staff lead by Diana Beasley, which included Elizabeth Adetoro, Anusha Balaji, Audrey Patterson, Jessica Park, Matthew Sakumoto, and Mary Kay Shaw. Funding for this undertaking was acquired from IDPH (contract no. 52400015C). The findings and conclusions in this report are these of the authors and don’t essentially signify the official place of the Centers for Disease Control and Prevention.
Corresponding Author: Fikirte Wagaw, MPH, Chicago Department of Public Health, 333 S. State St, 2nd Floor, Chicago, IL 60604. Telephone: 312-747-9891. Email: firstname.lastname@example.org.
Author Affiliations: 1Alliance of Chicago Community Health Services, Chicago, Illinois. 2Division of Population Health, Centers for Disease Control and Prevention, Atlanta, Georgia. threeNorthrop Grumman, Atlanta, Georgia.
- Pierannunzi C, Hu SS, Balluz L. A systematic overview of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011. BMC Med Res Methodol 2013;13(1):49. CrossRef PubMed
- Glasgow RE, Kaplan RM, Ockene JK, Fisher EB, Emmons KM. Patient-reported measures of psychosocial points and well being habits must be added to digital well being data. Health Aff (Millwood) 2012;31(three):497–504. CrossRef PubMed
- Krist AH, Phillips SM, Sabo RT, Balasubramanian BA, Heurtin-Roberts S, Ory MG, et al. ; MOHR Study Group. Adoption, attain, implementation, and upkeep of a behavioral and psychological well being evaluation in main care. Ann Fam Med 2014;12(6):525–33. CrossRef PubMed
- Crilly JF, Keefe RH, Volpe F. Use of digital applied sciences to advertise neighborhood and private well being for people unconnected to well being care techniques. Am J Public Health 2011;101(7):1163–7. CrossRef PubMed
- Cronin KA, Miglioretti DL, Krapcho M, Yu B, Geller BM, Carney PA, et al. Bias related to self-report of prior screening mammography. Cancer Epidemiol Biomarkers Prev 2009;18(6):1699–705. CrossRef PubMed
- Connor Gorber S, Tremblay M, Moher D, Gorber B. A comparability of direct vs. self-report measures for assessing peak, weight and physique mass index: a scientific overview. Obes Rev 2007;eight(four):307–26. CrossRef PubMed
- Rolnick SJ, Parker ED, Nordin JD, Hedblom BD, Wei F, Kerby T, et al. Self-report in comparison with digital medical file throughout eight grownup vaccines: do outcomes differ by demographic elements? Vaccine 2013;31(37):3928–35. CrossRef PubMed
- Khoury MJ, Lam TK, Ioannidis JP, Hartge P, Spitz MR, Buring JE, et al. Transforming epidemiology for 21st century drugs and public well being. Cancer Epidemiol Biomarkers Prev 2013;22(four):508–16. CrossRef PubMed
- Gittelman S, Lange V, Gotway Crawford CA, Okoro CA, Lieb E, Dhingra SS, et al. A new supply of knowledge for public well being surveillance: Facebook likes. J Med Internet Res 2015;17(four):e98. CrossRef PubMed
- Vogel J, Brown JS, Land T, Platt R, Klompas M. MDPHnet: safe, distributed sharing of digital well being file knowledge for public well being surveillance, analysis, and planning. Am J Public Health 2014;104(12):2265–70. CrossRef PubMed
- Smolinski MS, Crawley AW, Baltrusaitis Ok, Chunara R, Olsen JM, Wójcik O, et al. Flu Near You: crowdsourced symptom reporting spanning 2 influenza seasons. Am J Public Health 2015;105(10):2124–30. CrossRef PubMed
- Smart Chicago Collaborative. Foodborne Chicago; 2017. http://www.smartchicagocollaborative.org/people/staff-consultants/. Accessed November 20, 2017.
- Lurio J, Morrison FP, Pichardo M, Berg R, Buck MD, Wu W, et al. Using digital well being file alerts to offer public well being situational consciousness to clinicians. J Am Med Inform Assoc 2010;17(2):217–9. CrossRef PubMed
- Moody-Thomas S, Nasuti L, Yi Y, Celestin MD Jr, Horswell R, Land TG. Effect of techniques change and use of digital well being data on stop charges amongst tobacco customers in a public hospital system. Am J Public Health 2015;105(S2, Suppl 2):e1–7. CrossRef PubMed
- Kern LM, Barrón Y, Dhopeshwarkar RV, Edwards A, Kaushal R; HITEC Investigators. Electronic well being data and ambulatory high quality of care. J Gen Intern Med 2013;28(four):496–503. CrossRef PubMed
- Murphy DR, Laxmisan A, Reis BA, Thomas EJ, Esquivel A, Forjuoh SN, et al. Electronic well being file-primarily based triggers to detect potential delays in most cancers prognosis. BMJ Qual Saf 2014;23(1):eight–16. CrossRef PubMed
- Office of the National Coordinator for Health Information Technology (ONC), Department of Health and Human Services. 2014 Edition Release 2 Electronic Health Record (EHR) certification standards and the ONC HIT Certification Program; regulatory flexibilities, enhancements, and enhanced well being info alternate. Final rule. Fed Regist 2014;79(176):54429–80. PubMed
- Gustafsson PE, San Sebastian M, Janlert U, Theorell T, Westerlund H, Hammarström A. Life-course accumulation of neighborhood drawback and allostatic load: empirical integration of three social determinants of well being frameworks. Am J Public Health 2014;104(5):904–10. CrossRef PubMed
- Cohen J. A coefficient of settlement for nominal scales. Educ Psychol Meas 1960;20(1):37–46. CrossRef
- Cicchetti DV, Feinstein AR. High settlement however low kappa: II. Resolving the paradoxes. J Clin Epidemiol 1990;43(6):551–eight. CrossRef PubMed
- Lin LI-Ok. A concordance correlation coefficient to judge reproducibility. Biometrics 1989;45(1):255–68. CrossRef PubMed
- Lin LI-Ok. A word on the concordance correlation coefficient. Biometrics 2000;56(1):324–5.
- McBride GB. A proposal for energy-of-settlement standards for Lin’s Concordance Correlation Coefficient; 2005. NIWA Client Report: HAM2005-Zero62. Hamilton (NZ): National Institute of Water and Atmospheric Research Ltd. https://www.medcalc.org/download/pdf/McBride2005.pdf. Accessed November 20, 2017.
- Hill EM, Turner EL, Martin RM, Donovan JL. “Let’s get the best quality research we can”: public consciousness and acceptance of consent to make use of current knowledge in well being analysis: a scientific overview and qualitative research. BMC Med Res Methodol 2013;13(1):72. CrossRef PubMed
- Rodriguez HP, Glenn BA, Olmos TT, Krist AH, Shimada SL, Kessler R, et al. Real-world implementation and outcomes of well being habits and psychological well being evaluation. J Am Board Fam Med 2014;27(three):356–66. CrossRef PubMed
- Tisnado DM, Adams JL, Liu H, Damberg CL, Chen WP, Hu FA, et al. What is the concordance between the medical file and affected person self-report as knowledge sources for ambulatory care? Med Care 2006;44(2):132–40. CrossRef PubMed
- Singleterry J, Jump Z, DiGiulio A, Babb S, Sneegas Ok, MacNeil A, et al. State Medicaid protection for tobacco cessation therapies and boundaries to protection — United States, 2014–2015. MMWR Morb Mortal Wkly Rep 2015;64(42):1194–9. CrossRef PubMed
- Sallis R, Franklin B, Joy L, Ross R, Sabgir D, Stone J. Strategies for selling bodily exercise in medical apply. Prog Cardiovasc Dis 2015;57(four):375–86. CrossRef PubMed
- Krist AH, Glenn BA, Glasgow RE, Balasubramanian BA, Chambers DA, Fernandez ME, et al. ; MOHR Study Group. Designing a sound randomized pragmatic main care implementation trial: the my very own well being report (MOHR) undertaking. Implement Sci 2013;eight(1):73. CrossRef PubMed
- Estabrooks PA, Boyle M, Emmons KM, Glasgow RE, Hesse BW, Kaplan RM, et al. Harmonized affected person-reported knowledge parts in the digital well being file: supporting significant use by main care motion on well being behaviors and key psychosocial elements. J Am Med Inform Assoc 2012;19(four):575–82. CrossRef PubMed