Part 1:
1. Choose a paper describing the development or validation of a measure of relevance in health disparities research (please give the full citation and/or upload the paper if that's possible).
*The file is too large to upload:
Koita et al. “Development and implementation of a pediatric adverse childhood experiences (ACEs) and other determinants of health questionnaire in the pediatric medical home: A pilot study.” PLoS One. 2018 Dec 12;13(12):e0208088. doi: 10.1371/journal.pone.0208088. eCollection 2018.
2. What was the definition of the construct?
This paper developed a tool (questionnaire) to screen for ACEs (adverse childhood experiences) in pediatric clinics. ACEs are defined as “stressful or traumatic events experienced before the age of 18 years.”
3. How did the authors provide evidence on the validity of the measure? Could you think of additional approaches to validating the measure?
The authors provided evidenced on the validity in several ways. First, a working group was formed incorporating key stakeholders and allies in the development of this tool. Next, the group interviewed a multitude of providers and researchers who had previously utilized or modified an ACEs tool. Then, a thorough review of the literature was performed in order to evaluate the evidence surrounding the tools that were already in existence. For the pilot questionnaire used in this study, the authors described where changes were made to items used in prior tools, in some cases expanding a category, and in others, additions were made (such as the domain including social determinants), all based on review of the literature.
In regards to validating the tool itself, the questionnaire was piloted in a sample of families from a few pediatric centers in the area. They were met by a coordinator at the clinic, and after completion of the questionnaire, participants underwent a cognitive interview to assess comfort and clarity around completion of the questionnaire. Feedback was solicited when met with confusion. The sessions were audio and video recorded as well for additional review. Then, rapid iterative cycles were used to incorporate feedback and improve on the tool and these changes were incorporated into subsequent screenings.
Other ideas for validating the tool would include partnership with a external site, as different regions of the country may have different needs when using this tool. However, since this was a pilot study, this may have been outside the scope initially and may be more appropriate in a subsequent study.
4. How did the authors provide evidence on the reliability of the measure? Could you think of additional approaches to evaluating the reliability of the measure?
The reliability of the measure was demonstrated through ongoing use of the questionnaire with weekly rapid iterations that were incorporated prior to subsequent screenings. Their results demonstrated very little confusion from participants, and although there was discomfort expressed, no participant wished to stop the questionnaire and many offered their opinions about how important they thought the screening was.
Another method to test the reliability could have involved each participant completing the questionnaire on a few different modalities to see if the answers were different. For example, the study mentions they provided the questionnaire on paper, on a tablet, or by interview, and this was randomly decided. All participants could have completed the tool by two of these, rather than one, to ensure reliability within participants. Again, this may be outside of the scope of a pilot study, given the participant number is usually low and it may be difficult to get participation by asking for repeated measures.
5. Describe the implications of a lack of measurement validity or reliability for future research applications.
Both validity and reliability are essential to research. Validity ensures your tool or measurement was not only robustly and accurately defined, but it also ensures that the results from the study are either without bias or with as little bias as possible. The less bias you have, the more you can trust that the results that you found were the truth, which is what we seek in designing and interpreting research studies. Further, without internal validity, it is not possible to generalize your results to the total population at large, which ultimately is essential for furthering medical knowledge, and in this context, to eliminate health disparities.
Reliability is equally important, as the measurements you are taking should demonstrate that they are reproducible. Without this, you are introducing measurement bias into the study, which again affects the overall results and how they are interpreted. If you develop a tool, and another research attempts to utilize this tool in their own study, the results should be similar. If they are not, the results cannot be trusted.
Part 2:
1. Find a paper describing a health disparity (please give the full citation or, even better, upload the paper so everyone else can look at it too) .
*The file is too large to upload:
Sarath Raju, Emily P. Brigham, Laura M. Paulin, Nirupama Putcha, Aparna Balasubramanian, Nadia N. Hansel, Meredith C. McCormack. (2020) The Burden of Rural Chronic Obstructive Pulmonary Disease: Analyses from the National Health and Nutrition Examination Survey. American Journal of Respiratory and Critical Care Medicine 201:4, 488-491.
2. Summarize the construct and measurement of the dimension of disparity (e.g., racial inequalities?, SES inequalities?) and the outcome measured (e.g., self-rated health).
The construct here evaluates the disparities seen in patients with COPD in rural communities. Rural was measured using the NCHS definition, which categorizes residences at the county level to large metro central (inner city), large metro fringe (suburban), medium metro, small metro, micropolitan and noncore, where micropolitan and noncore are both rural. The outcome measured was a diagnosis of COPD by self-report.
3. What is the evidence for the validity and reliability of the measures?
The evidence of validity and reliability, while not explicitly demonstrated in this paper since it is utilizing a previously developed tool by the NCHS, has been demonstrated in other papers. This definition is quite reliable, as county lines are quite stable over time, so the definitions of these variables should be widely applicable. The validity does have flaws, however. This definition was not developed for health research, though it is used by the CDC. County level data can be problematic, as it tends to be subject to underbounding and overbounding of rural areas. For example, the Grand Canyon is designated as urban, because population density adjacent to it. Because of this, counties that may have both urban and rural areas are not captured, so this definition tends to make areas appear more homogenous than they really are. This has implications for the results that are shown in the studies, and may mask disparities.
4. What is the reference category used for the disparity measure (ie, who is the comparison group)? Why does this reference category make sense (or not) for this research question?
Urban, or large metro central, is used as the reference category. I think this makes sense for this research question in particular, because the aim was to highlight differences between urban and rural residents. In some cases, large metro central areas may have their own disparities, so using fringe areas (suburbs), which have been shown in many studies to be less disadvantaged, may be more appropriate to use for as a reference category.
5. How is the disparity quantified or measured? Is this an absolute or relative measure or are both provided? Describe which type of measure you would prefer for this research area, or, if both, why.
As far as I can tell, the rural health disparity is quantified as an absolute measure, designating participants as rural or urban based on the population density at the county level. I think this is appropriate for this research area, because this doesn’t represent a change over time.
Part 3:
1. Read someone else's response to part 1 above (identifying a construct) and comment, specifically noting whether you can see any additional implications of measurement quality for future research or whether you agree with those noted by your classmate.
Please see the post by Jerrine Morris.