Health Disparities Forum

Part 1:

1.    Choose a paper describing the development or validation of a measure of relevance in health disparities research (please give the full citation and/or upload the paper if that's possible). 

Diamond L¹, Chung S, Ferguson W, Gonzalez J, Jacobs EA, Gany F. Relationship between self-assessed and tested non-English-language proficiency among primary care providers. Med Care. 2014 May;52(5):435-8. doi: 10.1097/MLR.0000000000000102.

2.    What was the definition of the construct?

This paper sought to assess the level of language skill for clinicians who identified as “language concordant” with limited English proficiency patients. Clinician non-English language proficiency was assessed by two means: oral proficiency interviews (using the Clinician Cultural and Linguistic Assessment) and self-assessment (using the Interagency Language Roundtable, or ILR, scale).

3.    How did the authors provide evidence on the validity of the measure? Could you think of additional approaches to validating the measure?

For the self-assessment tool (ILR), the authors limited discussion regarding evidence on the validity of the measure, stating only “Other organizations, such as the American Council on the Teaching of Foreign Languages (ACTFL), have adapted the ILR scale for their own proficiency guidelines but it has not been widely adopted within health-care.” In some ways, this paper provided some validation of the tool: The authors compared clinician self-assessment using the ILR with a validated interview (see below). Another approach might be to have the clinicians complete multiple self-assessments, including the ILR and a separate validated tool, and then comparing the results between the two.

For the oral proficiency interview, more detailed information was included: “After ILR completion, clinicians were invited to take the CCLA, the only validated oral proficiency interview designed to assess clinicians’ ability to communicate directly with LEP patients in their preferred language. Other oral proficiency interviews focus either on non-medical settings or on a person's ability to function as an interpreter.²⁹ The CCLA has been validated in 17 languages, including those we tested.” It would improve the validity of the measure by identifying the content, or a description of the steps used to create the tool, as well as how scoring was advised and performed.

4. How did the authors provide evidence on the reliability of the measure? Could you think of additional approaches to evaluating the reliability of the measure?

The authors did not include any significant discussion on reliability of the ILR (comments were limited to those quoted in question 3). One way to determine reliability would be to identify internal consistency within the assessment tool itself. Cronbach alpha scores could be applied to these questions.

One way to assess the interview for reliability might be to have several known national expert speakers of each language test using the same tool, thus providing a means of using interrater reliability.

5. Describe the implications of a lack of measurement validity or reliability for future research applications. 

Without a measurement of validity or reliability, it would be difficult to use the assessment methods described to apply toward future studies; one could not apply much confidence in the results of these assessments or even use them to generalize findings from the data without feeling comfortable with the level of validity and reliability. On a more micro level (although not research-focused), given the importance of having appropriate and clear clinician-patient communication, and the expense of administering the validated oral interview tool, it would be helpful to be able to trust the results of a self-assessment tool for language proficiency.

Part 2:

1.    Find a paper describing a health disparity (please give the full citation or, even better, upload the paper so everyone else can look at it too) 

Barwise A¹, Jaramillo C², Novotny P³, Wieland ML⁴, Thongprayoon C⁵, Gajic O⁶, Wilson ME⁷. Differences in Code Status and End-of-Life Decision Making in Patients With Limited English Proficiency in the Intensive Care Unit. Mayo Clin Proc. 2018 Sep;93(9):1271-1281. doi: 10.1016/j.mayocp.2018.04.021. Epub 2018 Aug 9.

2.    Summarize the construct and measurement of the dimension of disparity (e.g., racial inequalities?, SES inequalities?) and the outcome measured (e.g., self-rated health).

The authors evaluated the effect of limited English proficiency on end-of-life care. Specifically, the primary outcomes described were: “characteristics of decision making for life support, code status, and aggressiveness of treatment²⁶ and included code status on ICU admission, code status on ICU discharge, change in code status during ICU stay, use of life support (invasive mechanical ventilation, noninvasive mechanical ventilation, dialysis, vasopressors, and cardiopulmonary resuscitation), presence of advance directives, and implementation of a standardized institutional comfort measures only order set.”

3.    What is the evidence for the validity and reliability of the measures?

Language proficiency was defined using any language other than English as the patient’s primary language, found retrospectively in the electronic medical record. The validity of this was substantiated with the following comment and three citations: “consistent with the definition used in several previous publications.” The validity was further supported by manual chart abstraction (compared with electronic query) of 100 of the records.

Reliability of the measure was not discussed.

4.    What is the reference category used for the disparity measure (ie, who is the comparison group)? Why does this reference category make sense (or not) for this research question?

The reference category was English-proficient patients. This category makes sense to measure, as the majority of information and discussion around end-of-life care – as well as some defaults – target white, Christian or non-denominational English-proficient patients.

5.    How is the disparity quantified or measured?  Is this an absolute or relative measure or are both provided?  Describe which type of measure you would prefer for this research area, or, if both, why. 

English language proficiency was quantified using primary language status from the electronic medical record. This was an absolute measure. When looking at the outcomes listed in question 2, the majority of data were presented as relative, which feels appropriate given the discrepancy in population sizes (LEP n=779; no limited English proficiency n=26,744).