Choose at least 3 distinct data sources (e.g., ARIC, HRS, death certificate data, NHS, etc), and give an example of a research question (e.g., a hypothesis about the effect of a specific exposure on a specific outcome) you consider the study exceptionally strong to address. For each, provide an example of a research question you consider the design very weak to address. Explain why the data source is strong or weak for each question. Do not just discuss the questions addressed in the readings, think of new questions, preferably things you might be interested in. This is not supposed to be a commentary related to the substantive questions in the readings: the goal is to focus on the pros and cons of various data sources. For hypotheses each study would not be well equipped to address, if possible describe another study that could address the hypothesis.
To clarify about the assignment this week: I want you to come up with new research questions, not revisit the questions discussed in the readings. The readings are just intended to describe the study design, i.e., the target population, the sampling strategy, the data quality and source, which data elements were measured and how well they would have been measured in the data source, the extent to which elements to help you draw causal inference (e.g., temporal order, covariate control) would be available in each study, etc.
Data sources: ARIC, HRS, and NHS
Question: Among people who drink alcohol at unhealthy levels, do those who predominantly drink liquor have a higher risk of developing atherosclerosis than those predominantly drinking beer or wine?
A related question is whether among those who drink at light-moderate levels, those predominantly drinking wine or beer have a lower risk of developing atherosclerosis than those predominantly drinking liquor.
Background: The association between alcohol consumption and cardiovascular outcomes remains relatively controversial. Most alcohol researchers believe that light to moderate drinking (e.g., 1-2 drinks per day) is protective against a range of bad cardiovascular outcomes including acute coronary syndromes and stroke. Such researchers would also tend to believe that heavier than "just light to moderate drinking", often called unhealthy drinking (e.g., AUDIT C>/=4 for men and >/=3 for women), may increase the risk of bad health outcomes. In general, (at least in the main stream), researchers believe that such effects of alcohol on health are moderated by "how much you drink". Personally, I suspect that what is important may be: not "how much you drink", but "what you drink".
Hypothesis: Drinks such as beer and wine contain anti-inflammatory chemicals (mostly of the polyphenol type). These non-alcohol constituents protect drinkers of wine and beer against inflammation and associated health consequences like atherosclerosis. Drinks such as liquor which are made through distillation (aka purification) processes are depleted of these protective chemicals; the alcohol in such drinks thus causes unhindered inflammation leading to harmful effects. Co-incidentally, light to moderate drinkers predominantly drink beer or wine, while heavy drinkers predominantly drink liquor, suggesting that the effects observed by previous studies may in fact be due to the alcohol type rather than the amount of alcohol consumed.
Data sources to address this question:
ARIC, HRS, and NHS, are all potential data sources. I suspect that the measurement of the outcomes is good for all 3. However, the predictors (total alcohol volumes, drinking patterns, and beverage-specific alcohol volumes) may not be well measured. The choice of which data source to use may depend on who obtained the most complete data on the volumes of specific alcoholic beverages. Any of them would be a good data source if they have complete beverage-specific alcohol consumption data. However, any of them would be a bad data source if they collected alcohol consumption data only as "standard drinks". The NHS would be weaker than the other two because it has data on only women. According to some previous studies women tend to predominantly drink wine and beer, and volume-wise, they tend to be light to moderate drinkers, compared to men. The NHS would thus be enriched with wine/beer drinkers and light/moderate drinkers.
Stephen: neither the predictors nor the outcomes would be measured in HRS.
The outcome is not measured in NHS (think of the design - they never see people in person), although NHS probably has the best dietary information. NHS would also be a concern not only because it's all women, but it's all female nurses, so a particular SES stratum. Since dietary behaviors, such as alcohol use, is highly patterned by socioeconomics, you might worry about this.
It is possible that you could do this in ARIC, which at least has the outcome, and includes some dietary information (but i'm not sure if they distinguish between types of alcohol). But what are the limitation of ARIC?
If beverage-specific alcohol information is available, I am not sure what the other limitation of ARIC would be. I think they interviewed patients annually and examined them every 3 years, both of which should be adequate for predictor and outcome evaluations.
I think that even if "atheroscelosis"-specific outcomes are not available (e.g., in NHS), it may be possible to use outcomes further downstream e.g., stroke or death.
True, the NHS is bad for this. However, to a certain extent, I am not sure why the recruitment of a particular SES stratum would not simply be considered a restriction similar to what you might do to control for confounding since SES is a confounder of this question. What is important may then be whether there is sufficient variability in alcohol type preference at higher SES, which I think is the case.
Data Source #1: ARIC
Question that would be well equipped to handle: Obviously the questions that the ARIC study is strongly tailored to answer are those related to atherosclerotic disease. The combination of the cohorts and community surveillance allows the authors to put forth a strong argument that their cohorts and the conclusions they draw from the cohorts are representative of the community that they are drawn for. A non-atherosclerotic question that could be looked at would be looking at pulmonary function and blood pressure changes from the two time points that were recorded and looking at the factors that are associated with improved functioning vs. decreased functioning. Obviously this is not what the study was designed for but the measurements that were made are valuable and alternative applications should be looked into.
Question that it would not be well equipped to handle: What are the risk factors for a rare atherosclerotic outcome such as cholesterol emboli syndrome?
- Cholesterol emboli syndrome, although related to atherosclerotic disease, is a manifestation that happens infrequently. Thus, you would not expect that many cases in your cohorts. A study that would be better suited to study pericarditis is a case-control study that specifically targets patients with this specific syndrome.
Data Source #2: HRS
Question that it would be well quipped to handle: How does wealth and behaviors impact the prevalence of hypertension, diabetes, cancer, heart disease, lung disease etc. in the US?
The HRS is a very rich dataset - that is a longitudinal survey of a representative sample of Americans over age 50. The banks article, looks at how wealth and behaviors impact overall mortality and uses the ELSA database to evaluate the same thing in England. Rather than looking at overall mortality, this data could be readily be applied to determine the impact of the same predictor variables on specific outcomes.
Question that it would not be well quipped to handle: Assessing causality of diseases that might have been the byproducts of deleterious health behaviors initiated before the age of 50. An example is what are the factors that cause lung disease or liver disease?
- It doesn't seem like the HRS has the granularity in terms of examining past smoking or drinking habits before the age of 50 when participants are recruited that would be crucial in assessing the causal factors of lung disease or liver disease. That is, it matters how much you drank and smoked before the age of 50. The answer to this question would be better assessed by a study specifically geared towards either of these disease sets with a more robust set of questions to assess risk factors.
Data Source 3: NHS
Question that it is well equipped to handle: What is the relationship of lifestyle and risk of different types of chronic disease?
- It is good for this question in that we have long term follow up of individuals over time. Thus we can see how different lifestyle factors either short or long in duration may impact certain diseases influenced by behavior over time such as cancer.
Question that it is not well equipped to handle: As with all of these studies, it wouldn't be good at handling question re: rare disease outcomes. This would be again best be handled by case control studies. It is also limited by generalizability since these are all nurses and groups that are not Caucasian are not very well represented. Thus for example it might not be that suited for evaluating subgroup analyses of the relationship of lifestyle and risk of different types of chronic disease. This would be better handled by making sure that you get a better balanced cohort.
James: good point that ARIC won't work for rare outcomes, but could be used for common outcomes and even secondary outcomes that they happened to measure. But it also has limitations implicit in the sample design.
You have the strengths and limitations of HRS- although in general detailed outcome measurement is a challenge in HRS because for the first ~14 years of the study they did no in -person assessments - so all outcomes were self- or proxy-reported.
Re NHS: what do you mean by "lifestyle"? Because these are all nurses, and the vast majority are white non-Latina, it would be difficult, for example, to evaluate the impact of working vs not working, being married vs never-married, or the impact of behavioral patterns that are highly socioeconomically or racial/ethnically stratified. You might think that even a simple issue like medication adherence would operate quite differently among nurses than others.
Note that NHS has ~120,000 women followed for (at this point) 40 years. So it can be used for much rarer outcomes than either ARIC or HRS. Hemorrhagic stroke might be feasible for example, whereas in most cohorts it would be tough.
National Health Interview Survey (NHIS)
Good question to address:
Do changes in health care policy affecting access to care impact cancer screening uptake?
The biggest downside to cross-sectional studies is not being able to easily establish temporality. This is because cross-sectional studies only measure individuals at a single point in time, with no follow-up, which requires measuring prevalence of exposure and outcome. The question above can use repeated cross-sectional measures, before and after a health care policy was implemented, to determine if there is an association between changes in health care policy and screening uptake. This question would not require follow-up of the same individuals over time, since each cross-section is representative of the US population.
Bad question to address:
Does mammography screening detect breast cancer at an earlier stage?
As mentioned above, we are only working with prevalence of exposure and outcome in NHIS. Even if a question was available asking if mammography led to detection of cancer, we cannot be entirely sure of temporality. Additionally, we cannot follow the same person across waves of NHIS, so it would be very difficult to answer this question.
Breast Cancer Surveillance Consortium (BCSC)
Good question to address:
Does routine mammography screening improve detection of breast cancer in women over age 65?
Since the BCSC collects data on any mammogram a woman receives and links this to pathology data and cancer registries, the data has the capacity to follow individuals from initial screening to cancer detection. Furthermore, since it measures multiple mammograms, it is possible to measure “routine” screening, as opposed to ever screening or recent screening.
Bad question to address:
Do women not undergoing mammography screening have lower breast cancer survival?
This data set cannot measure women who do not screen, as the only participants are those who complete intake forms when coming in for a mammogram. This could potentially leave out a large percentage of the lower-income or uninsured populations.
Kaiser HMO Data:
Good Question to address:
Does open lung biopsy lead to greater readmission rates within 30 days than closed or minimally invasive lung biopsy?
A major benefit of claims data or health insurance-based data is longitudinal documentation of all interactions with the medical environment. Since Kaiser HMO enrollees undergo all of their care at Kaiser facilities (or so we hope), there is a significant amount of data on their health care interactions. This particular research question could be answered because we could use the date of the procedure and billing codes to determine when an individual had a lung biopsy, and what type it would be. We could then document how many of these individuals were readmitted to the hospital with complications (based on ICD-10 codes) within 30 days of their biopsy-related discharge.
Bad Question to address:
Does health insurance type impact cancer screening follow-up?
Since this is health insurance-based data, there is not going to be major differences across health plans that could allow researchers to answer this question. Using insurance-based data can restrict the generalizability of the findings, because the individuals who enroll in a certain type of insurance may have sociodemographic factors that are different from the general population.
Josh:
Great example for NHIS. Distinguish between local, state, or federal policy. To evaluate local or state policies, you need information on location of residence at the relevant geographic resolution.
Also (for you and others) distinguish between evaluating whether there has been a change in uptake of screening since implementation of the policy (easy to evaluate) and evaluating whether the policy caused an uptake of screening (quite hard to evaluate).
Re Kaiser: you need to ask why, within the Kaiser system, one patient receives an open lung biopsy and another patient does not. It is critical to understand the rules - this is unlikely to be "random" within Kaiser, unless they intentionally randomize in an effort to evaluate.
(excuse the tardiness, had delayed departure from a conference in austin, texas)
Data source: Nurses Health Study
Data type: Longitudinal population representative closed cohort
Strong RQ: Does levels of major stressors/cumulative stress lead to differences in breast cancer incidence/mortality? The NHS would be well designed to answer a longitudinal exposure of cumulative stress with sufficient follow-up of breast cancer incidence/mortality. Because it is a closed cohort, it would not be susceptible to survivor bias.
Weak RQ: How do levels of cortisol lead to differences in breast cancer incidence/mortality? A more biologic question may not be able to be answered in a survey based longitudinal cohort unless a subsample had saliva levels obtained. Moreover, cortisol requires more refined diurnal collection, and thus could not be practically performed outside a smaller cohort with this measurement prospectively defined. However, it would likely be unable to answer the mortality question, and more equipped to answer the incidence question.
Data source: Health and Retirement Study
Data type: Longitudinal population representative cohort
Strong RQ: How does absolute SES levels predict incidence and mortality of breast cancer? This study would be well designed to ascertain a spectrum SES to the long term outcome of breast cancer mortality. However, it would be susceptible to selection bias for perhaps underrepresenting low SES for those who actually respond to the questionairre, as well as to ethnic/cultural groups because I presume the survey is in english only?
Weak RQ: How does SES levels predict incidence and mortality of breast cancer in women under age of 50? The survey is limited to only adults > 50 years old. This could be assessed cross-sectionally using claims data, but the exposure information for SES would not be as well enriched.
Data source: Breast Cancer Surveillance Consortium
Data type: Representative population cohort (network of 8 regional mammography registries linking to SEER)
Strong RQ: How does mammography breast density categories reporting change after revision in 2014? The BCSC would be able to best provide a representative national sample of mammography reporting practices — both private and academic across a range of geographical locations.
Weak RQ: What are variations in screen-detected breast cancer within the state of California? The registry can only provide data on regions that participate (in CA, that is only the bay area), and thus it is limited in providing geospatial data of mammography that perhaps Medicare claims data would provide (albeit only for elder populations of >65 years old).
Vignesh: nice examples. Some small points: the representativeness of NHS is questionable, since the sampling frame was married nurses in 10 (12?) states in 1975 (approximately).
Closed cohorts do not eliminate the potential for survival bias. I suppose they eliminate some particularly egregious processes that contribute to survival bias, but you can still have survival bias in a closed cohort.
Re the NHS and biological measures: your concern is completely legitimate. One strategy investigators on this type of cohort use is to collect (often via mail) biological samples and bank them. The reasoning with Nurses is that this group of people would be unusually qualified to collect the samples. Often such samples are stored and used later in a nested case control study, when a particular biomarker becomes of interest or the assays for the biomarker become feasible.
Data Source #1: NHS
Example Research Question #1 (good): Do women who predominantly work night/evening shifts have higher risks of CVD?
Exposure #1: working on rotating night shifts
Outcome #1: CVD incidence between T1 and T2
Example Research Question #1 (not so good): Do men who predominantly work night/evening shifts have higher risks of CVD?
Why the data source is strong or weak (#1): NHS only samples women. This cohort looks like it only uses questionnaires, so any measurement that requires an objective measure would not be ideal for these data.
Data Source #2: ARIC
Example Research Question #2 (strong): Does neighborhood socioeconomic level effect levels of evidence-based care for chronic kidney disease?
Exposure #2: belonging to or living in an adverse social environment
Outcome #2: guideline-concordant levels of care for chronic kidney disease
Example Research Question #2 (weak): Are there racial/ethnic disparities (beyond white vs. blacks) in levels of guideline concordant care for chronic kidney disease?
Why the data source is strong or weak (#2): Only originally sampled men/women white/back, so additional minority groups might be under-represented. Also invitation/volunteer bias will influence who attends the clinical follow-up (i.e. language barriers)
Data Source #3: NHANES
Example Research Question #3 (strong): What is the prevalence of chronic kidney disease awareness among individuals with hypertension, undiagnosed hypertension, pre-hypertension, or no hypertension?
Exposure #3: Hypertension, undiagnosed hypertension, pre-hypertension, or no hypertension.
Outcome #3: CKD defined by estimated GFR 15 to 59 ml/min per 1.73 m2 or albumin-creatinine ratio ≥30 mg/g & patient awareness (via questionnaire).
Example Research Question #3 (weak): What effect does energy intake/energy expenditure have on various chronic illnesses/conditions?
Why the data source is strong or weak (#3): NHANES has a complicated, multistage sampling method, but still uses self-reported measure of diet and physical activity, which has been shown to be unreliable in estimating energy expenditure/intake.
I'm not totally sure I am thinking about this the right way but here is my attempt at completing the assignment.
Data Source 1: Linked Birth / Infant Death Records Data Summary
Is prenatal care effective in reducing SIDS?
The research design of this data set is week in assessing this question. Although prenatal care is assessed before infant death, there are many issues with ascertainment of this predictor. There may be differences in prenatal care that would affect this association. Content and quality of prenatal care likely vary between locations and over time. Furthermore, the ability to conclude not/stated, not on certificate could lead to significant narrowing of this predictor which may or may not be more present in certain prenatal group categories. Furthermore it is impossible to track all the potential confounders of the relationship between prenatal care and SIDS. For example, maternal wealth and nationality may be related to type and access to prenatal care and also associated with SIDS in that co-sleeping and smoking (known risk factors for SIDS) might have different prevalence in these groups. Also the outcome of SIDS may not be adequately obtained simply from ICD scoring. There is sometimes error between autopsy results and assigned diagnosis.
Data Source 2:
Did the new “healthy plate” Dietary guidelines for Americans reduce obesity in the US children under age 18?
This research design is moderately strong. The “healthy plate” replaced the “food pyramid” in 2010. A measure of obesity prevalence in 2009 versus 2015 NHANES report might give adequate time to see the effect of this intervention on US children. NHANES is a representative sample of the US population and sub-analyses could be done in specific subgroups. Many important covariates are included in the NHANES data set including measures of diabetes, physical activity, family questionnaire which includes many important demographic variables, also specific questions about diet behavior and nutrition. Insurance and medical conditions are also included.
Data Source 3: NHS
Does divorce shorten your lifespan?
NHS is a moderately week design to answer this question. Respondents are asked at outset if they have ever been divorced and on repeat surveys about subsequent divorce. This is subject to bias as it requires the subject to report divorce which could be something that is not remembered (low functional status), or lied about. There are many covariates that might affect the outcome of death which can be measured as well
Danielle:
I agree that the linked birth-death files are not going to provide a good source to evaluate the impact of prenatal care on SIDS. Actually this is a pretty tough research question - where could you examine this question?
Your question about the impact of the food pyramid is also tough because although it is easy to evaluate national trends in dietary behaviors, it is difficult to attribute them to the introduction of the food pyramid per se. This study is vulnerable to what Cook & Campbell called bias due to history.
Nurses Health Study –
Good: Do different measures of obesity (e.g., body mass index, waist-hip ratio, waist circumference) and longitudinal changes in these measures over time differ in their effects on breast cancer subtypes? This would be an ideal question ask in the NHS because they have repeated measurements on the same women of a variety of body composition measures. They also have a substantial number of women so you would have enough outcomes to look at subtypes (most likely) of breast cancer. There has been evidence that different measures of body composition differ in their effects on cardiovascaular disease, and subtypes of breast cancer are hypothesized to have different etiologies, particularly with respect to mechanisms by which obesity and adiposity affects risk.
Bad: Nurses would not be optimal to look at the extension of the question above to look at different measures of obesity and their impacts on breast cancer subtypes, and whether this effect differs by race. Because the NHS is predominantly Caucasian, the sample size might be too small to look at differences among, for example, African American women who have triple-negative breast cancer, a rarer but very aggressive subtype that is hypothesized to have a different relationship with adiposity than other subtypes. This may be best examined using a case-control design in an equally large but more diverse cohort, or one that over-samples for African American women, like the Carolina Breast Cancer Study.
Kaiser --
Good: Is higher Indigenous ancestry among Latina women associated with poorer breast cancer outcomes?
This is a good question to ask in the Kaiser cohort because Kaiser is an HMO system where everyone gets largely similar access to care. Because the effect of higher indigenous ancestry is probably mediated by social factors including education, social capital, language, income, and perhaps more downstream from those factors, access to care, which is difficult to fully capture using questionnaire data, using Kaiser helps to control for this by restriction, because all women in the system should have similar access to screening, treatment and follow-up care.
Bad: Does more frequent mammography / shorter mammographic screening interval reduce interval cancer rates? Does it increase false positives?
Kaiser would not be an ideal population to use for this question as Kaiser and the doctors within Kaiser recommend the same screening interval for women based on their reimbursement policies. For a woman of average risk, this would be 2 years. Of course, higher risk women can get shorter screening intervals in Kaiser but there is no variability among those women—you must be at high risk to get increased frequency of screening under the Kaiser plan. You would need a population that is more heterogeneous with respect to screening interval to look at this question, where both high and low risk women get screening at 1 year, 2 year, 3+ year screening intervals.
Health and Retirement Study –
Good: Does the effect of obesity / body mass index on post-menopausal breast cancer incidence differ by education or income (as markers of wealth)? The HRS would be a good cohort to answer this question as they have detailed demographic and behavioral data in addition to complex data on wealth, income and education. I think this is an interesting question as the mechanisms by which obesity increases cancer risk may be exacerbated by stress associated with the income inequality or social factors. In the HRS, current information is available
Bad: What is the effect of lifecourse obesity on postmenopausal breast cancer and how is this altered by social status at different periods in life? The HRS would not be well-equipped to answer this question as people are enrolled starting at age 50 and therefore detailed information may be difficult to collect on early life exposures, and if collected, may be subject to measurement error due to poor recall.
Natalie - These are all very good examples. The Kaiser/ancestry question is interesting. Kaiser as a whole does not include much socioeconomic information, but I like the argument that you can rule out strict access-based mechanisms.
ICN2: Improved Care Network for IBD Databse
The ICN database is drawn from voluntary data entry from sites belonging to this consortium. Data is entered either by research coordinators or nursing staff, and in some instances pulled directly from compatible health records, such as EPIC. It provides demographic information and details about disease location, laboratory assessment, physical exam findings, medications and nutritional outcomes. A question it would be equipped to handle: what are risk factors for being placed on a biologic medication at diagnosis of pediatric Crohn disease. A question it is not well equipped to handle is do children who are cared for at an ICN center have better nutritional status than other children diagnosed with IBD
KIDS: The Kids' Inpatient Database provides information on a national scaled however it provides more generic information that is often pulled from medical records as opposed to being voluntarily entered. It includes discharge status, demographics, hospital characteristics, and more detail about cost of medical treatment. It would be well equipped to compare differences in length of stay between children with IBD who are admitted to teaching facility/academic center as opposed to children with IBD admitted to community hospitals. However, it is not well equipped to look at risk factors leading to increased length of hospital stay for children with a diagnosis of IBD. This database does not provided much detail about patients diagnosis, it does provide comorbidities but no finer detail IBD-specific variables such as nutritional status, medication history, disease location, etc.
NHANES National Youth Fitness Survey:
This survey was performed in 2012, thus it would be well equipped to answer to answer a question about what was going on in 2012 in particular. Perhaps, children <10 have better aerobic fitness than children 10 and older. However, it is not well equipped to answer questions that require repeated measures or data.
Death certificates
Good: How does accidental trauma differ by race and gender? Death certificates would be a great way to study accidental trauma, which leads to death. Accidental trauma is easily assessed and is not often confused with disease states such as CV-related death versus malignancies. My group specifically looks at bicycle related trauma. Bicycle related fatality has actually been quantified by using a death certificates but this is of people who died while on the "road."
Bad: Has prostate cancer deaths decreased since the advent of screening (or vice versa... has prostate cancer deaths increased since we have stopped screening). As the article mentioned, malignancies are underreported in death reports. Most prostate cancer is also indolent and rarely causes systemic disease. However, aggressive forms of prostate cancer can be deadly, but it is challenging to determine the cause of death, even in cancer patients.
ARIC-like designs
Good: Is BPH associated with men who have vessel disease? You could even do any study that looks at a chronic condition like BPH and vessel disease with the ARIC design. I think this design is best for chronic disease states that change over time.
Bad: How is the management of BPH different in adult patients with developmental delay (who might not report urinary symptoms as early as other men)? In one county of the ARIC study, they chose people who were registered to vote or eligible for jury duty or selected based off addresses. Adult patients with developmental delays may not fit this criteria and thus be underrepresented in a study like this.
HRS/WLS
Good: Do more men and women report bisexual activity after DOMA was deemed unconstitutional (i.e. more LGBT-friendly America?). Repeated cross-sectional analysis would be helpful to determine if more people report bisexual activity. We don't really care whether or not it is the same people reporting each time, but rather that the proportion of people reporting bisexual activity increases/decreases/stays the same throughout time.
Bad: Does treatment with sildenafil (Viagra) reduce the incidence of ED after radiation therapy for localized prostate cancer? In a scenario such as this it is important to know who is taking the medication and who develops ED after XRT. Repeated cross-sectional studies will not provide the strongest support to a causal effect. A cohort study of the same individuals or (even better) an RCT could answer this question.
Thomas: These are nice examples. I think it is sometimes difficult to discern whether an accidental death was actually a suicide, although suicide by bicycle one assumes is quite rare.
HRS to my knowledge asks nothing about sexual activity, sexual orientation, or sexual identity. So, it would be tough to study this in HRS. I'm not sure about WLS. In fact one surprise about HRS is that there are very few same sex couples. They enroll new cohorts every 6 years as people "age-in" to eligibility. It would be interesting to see if this has changed in the most recent enrollment cohort (2010).
California Cancer Registry (CCR)
Strength
Is neighborhood deprivation associated with increased risk for in situ and invasive breast cancer?
The CCR contains information on breast cancer (both in situ and invasive) incidence in the state of California. The CCR also has an SES indicator at the block and tract level which allows for assessment of neighborhood SES and breast cancer incidence.
Weakness
Does individual-income independently predict breast cancer risk in women who live in deprived neighborhoods?
Beyond the neighborhood SES indicator, the CCR does not allow for the assessment of other socio-economic factors and breast cancer incidence.
Can’t assess cancer incidence and individual education, income, etc.
American Communities Survey (ACS)-US Census
Strength
Are African Americans more likely to live in poor neighborhoods despite their individual income? Or Is there an association between neighborhood average income and neighborhood racial/ethnic composition?
The census has information on income, education, race, and other social conditions that allow allows for the characterization of neighborhoods. It is useful for assessing whether or not social disparities exist between racial/ethnic groups in the United States.
Weakness
Have cancer screening rates in underserved racial/ethnic minorities improved since the implementation of the the Affordable Care Act?
The census does not provide information on cancer screening outcomes. In order to determine whether or not insurance coverage (which is provided by the census) influences cancer screening you need to overlay the census data with screening data which may be obtained from ACS or CDC.
Surveillance, Epidemiology, and End Results (SEER)
Strength
Is breast cancer incidence increasing or decreasing between the racial/ethnic groups?
SEER contains information about breast cancer incidence and mortality dating back to 1975. This makes it easy to assess whether the cancer incidence and/or mortality gap (across the various cancers) is widening or closing between the five racial/ethnic groups (White, Black, Asian/Pacific Islander, American Indian/Alaska Native, Hispanic).
Weakness
Does poverty explain the breast cancer mortality gap between African American and Caucasian women?
Beyond basic demographic information (i.e. age, race/ethnicity, and sex) that allow us to determine if racial/ethnic disparities exist, SEER does not contain other socio-demographic information to determine if social and economic conditions are associated with cancer disparities.
Ekland,
If you don't have the individual level SES variables in the CCR, how can you distinguish the effects of individual from neighborhood SES? When they link w/ census tract SES, do they conceptually think it's a proxy for individual SES indicator or a place level measure?
Very interesting to think about using the Census for health research. I agree with your first idea. In the old census there were about 3 questions that were nominally health measures (self-reported limitations). There has been hardly any research using these to my knowledge. I do not actually know how valid the measures are, but presumably they took them from SF-36 or something like that. I don't know if there are more items on the ACS.
How is the design of SEER different than CCR?
US Linked Birth/Infant Death Data Set
This would be a good data set to evaluate the effect of WIC on reducing infant mortality, since one of the items is whether or not the mother is a WIC recipient. If there were a particular time at which WIC was scaled up for example, you could maybe do an IV analysis to test for a change in the risk of infant death before and after the scale-up. You could calculate predicted values for the extent to which access to WIC receipt among mothers giving birth increased in the scale-up year, and then regress infant death on these predicted values.
This data set would be problematic to evaluate the effect of smoking (or a smoking campaign or policy change) on infant mortality, since 26% of the births had missing data on the maternal smoking item.
Death Records
Maybe we could take advantage of the poor accuracy of verbal autopsies, and their reliance on reports from household members, to study change in HIV stigma. This assumes that in a high-stigma environment, household members would avoid attributing the death to AIDS, or describing symptoms that might lead to this conclusion. So, maybe we want to evaluate the effects of a campaign to reduce HIV stigma. We could measure change in the discrepancy between HIV prevalence (as measured by DHS) and verbal autopsies attributing death to AIDS, as a proxy for change in HIV stigma. We would need to control for treatment utilization, and use fixed effects for communities to account for poor standardization of verbal autopsy methods across locations.
A questionable study would be one in which we used verbal autopsies to obtain the malaria mortality rate in 25 sites, and compare across sites. First, we have essentially no evidence of the accuracy of verbal autopsy reports, given the challenges of doing autopsy validation studies in developing countries. Secondly, the lack of standardization of verbal autopsy protocols limits comparability across sites.
ARIC
This study began in 1986, the year the Anti-Drug Abuse Act created mandatory prison sentences for low-level drug possession. Given the timing, and the variation in the racial and SES make-up of the communities studied, it could be used to investigate change in cardiovascular risk factors in communities affected by mass incarceration. To do this, you’d need to include additional data sets to characterize the effects of the policy on each study community, eg increased law enforcement presence and arrest rates.
You would probably not want to use this data set to test effects of the policy on cardiovascular events (as opposed to more intermediary risk factors), because from what I can tell the study only ran for six years. I would expect that cardiovascular events that occurred within this timeframe probably began to develop many years prior.
Three Data Sets, Good (+) and Not as Good (-) Research Questions.
NHANES
(+) Is the 1964 civil rights change associated with improved health outcomes?
Since NHANES started in the 1960s, you could look at this particular event which, purportedly banned racial discrimination in healthcare, and see if this date associated with an increase in health care seeking behavior/utilization because of easier access to care.
(-) Is race or ethnicity associated with differential care of chronic disease, such as diabetes?
NHANES is apparently pretty diverse, and follows chronic dz, but its challenging to temporally relate incident chronic dz with medical care received, as lack of medical care in first place might have led to the disease.
CARDIA:
(+) Is X genotype associated with development of cardiac disease?
CARDIA looks at development of heart disease amongst healthy population. Depending on serum availability one could look at a particular gene trait and see if it is associated with development of cardiac dz. This would be best done as case control.
(-) Is race associated with consumption of vegetables?
CARDIA has strong dietary data apparently. A weakness of this study is that it may lack 1) generalizability to areas outside of Kaiser, and 2)socioeconomic granularity that could inform any observed variations. A better study design would be national survey explores dietary patterns.
ARIC
(+) Is atherosclerosis associated with development of diabetes? Does this association vary by race, sex, place?
ARIC studies atherosclerosis history in a representative sample. We know diabetes accelerates atherosclerosis, but the extent to which this is associated with race could be identified.
(-) Is race associated with intensity of care delivered in those suffering from a heart attack?
This wouldn't work for the Jackson group, but essentially you could look at # of procedures or surgeries associated with MI. This would be better served by a focusing on one health system, and doing chart abstraction to see the number of procedures. Volunteer bias of this population would potentially hurt this study question. Another challenge would be how sick the patient was, which would have an effect on level of care. Also, you would have to pull some additional information: payer status, # of visits with PCP.
Alyssa,
These are good examples. I love the clever idea to use verbal autopsy underreporting as a measure of stigma.
Re ARIC, one of the special features of ARIC is that nearly all African Americans were enrolled in one location (which had no whites). So race and place are inextricably confounded in this data set. To the extent that you think the enforcement may have had differential effect on AA communities and this was important to study, this could be a problem.
Data source: California Health Interview Survey
Good Question: What are sociodemographic and sexual risk factors associated with HIV testing in ethnic minorities?
The dataset provides great way to determine correlates of HIV testing behavior because it collects a variety of data on sociodemographics, behaviors, health status, health care access, employment and other relevant factors from a representative sample of individuals in the state (including undocumented persons). Despite being cross-sectional, the insights gained from this dataset can help inform public health interventions, healthcare policies, and resource allocation.
Bad Question: Is the availability of needle-exchange programs associated with higher prevalence of HIV testing among injection drug users?
Although it may be possible to match HIV testing data collected in CHIS with the availability of needle-exchange sites by zip code, the dataset may be limited by the availability of information on this hard to reach population. CHIS samples participants using random digit dialing. Injection drug users are generally isolated and marginalized, and may not have a dedicated telephone from which they can be accessed. This question may be better answered using sampling techniques tailored for hidden populations – like time location sampling or respondent driven sampling.
Data source: Centers for Medicare & Medicaid Services
Good question: The association between primary care follow-up and re-hospitalization within 30 days among patients admitted for heart failure.
This dataset provides a great opportunity for capturing re-hospitalization among persons enrolled in Medicare and Medicaid who have been admitted for heart failure. Since Medicare/Medicaid patients can be tracked through billing claims, the study can determine whether a patient received a primary care follow-up and if they were re-admitted to the hospital within 30 days of discharge. A limitation of this approach may be that persons who have private insurance may not be represented (since CMS is the payer of last resort, I can imagine that some individuals with great private insurance wouldn’t have to go through CMS).
Bad question: 30-day post discharge quality of life and functionality among heart failure patients enrolled in Medicare/Medicaid.
It may be possible to use the CMS data to identify a random subset of the population that can be contacted to assess their quality of life and functionality, but it would be difficult to determine patient quality of life and functionality based solely on insurance claims data.
Data source: SFGH/SFDPH data
Good question: What is the 30-day survival rate of persons with who present to the ED with a penetrating injury to the chest and underwent an emergent thoracotomy?
Admittedly, I don’t know how easy it would be to extract data from SFGH’s electronic health records…in the remote possibility that it is, I can imagine that it would be a great dataset to answer this question. All traumas in San Francisco and northern San Mateo county are brought to SFGH so the dataset would capture all emergent thoracotomies performed related to this type of injury in the area. Mortality can then be tied to the patient using death certificates or from hospital records.
Bad question: Is “opt-out” HIV testing in the emergency department associated with better sexual risk outcomes among men who have sex with men?
This dataset is not suitable for this question for several reasons: 1) it would be difficult to capture who opted out of HIV testing versus the provider forgetting to order an HIV test; 2) it would be difficult to control for confounders (e.g. current risk); 3) it would be difficult to establish who the control group is; and 4) it would be difficult to determine the outcomes (e.g. getting sexual risk data, like diagnosis of STI 6 months later, may not be feasible).
Carlo: great examples.
For CMS, I think most payers try to frame the policy so Medicare pays first- so even though it's supposed to be payer of last resort, it's used by the vast majority of eligible people. There is an issue that some people have HMO medicare, so services are not billed item by item, just per capita. These people may well systematically differ from those with FFS medicare plans, but I do not know the selection process there. It could be a problem for this question.
Re mortality after penetrating injury: it may not be straightforward to link w/ death records because you would need the medical record to be fully de-identified. The issue of de-identification derails many efforts at innovative and useful analyses. I do not know if SFGH automatically pulls in such records, but if not, it would be a tough study.
NHS
Good question: Incidence and risk factors for cancer among smokers in the NHS cohort
The NHS cohort study design provides an opportunity to determine if there is a causal association between smoking and lung cancer, because it offers temporality, dose trends, biological plausibility and specificity – assuming there were no other exposures to lung cancer causing carcinogens like asbestos. Incident rates of lung cancer would be compared between smokers and non-smokers while adjusting for the other covariates including known confounders like alcohol, age and family cancer history.
This would require verifying lung cancer diagnoses in medical records and death certificates plus corroboration with physician/family reports.
Bad question for study design: What is the effect of diabetes on the incidence of pancreatic cancer
Considering that that diabetic patients constitute a small percentage of the study population and that pancreatic cancer is a rare outcome with a narrow diagnostic window (maybe missed/undiagnosed) outcome events may be too few and study wouldn’t powered to detect any effects. A case – control study improves efficiency may be used to examine the temporal link between being diabetic and risk of pancreatic cancer in a design that deploys incident sampling design – in which case the odds ratio would approximate the IRR.
ISGS
Good question for ISGS design: Is there a link between level of exercising and homocysteine levels among stroke patients enrolled in the ischemic Stroke Genetics study
Study design – since the reading suggests that this was a secondary data analysis of the ISGS, conducting a study seeking to show a link between serum homocysteine and level of exercise(or any of the stroke outcomes in paper) - hypothesis generating would be appropriate to the design. Study design – exposure and outcome available at same time, limits quality of question ask. If link present, then using homocysteine over reported level of exercise would be preferred since homocysteine was better quantified/measured than past exercise history in the ISGS, it would be more reliable/valid measure
Bad question for type of design: What is the incidence and risk factors associated with new neurological disability among stroke patients enrolled in the ISGS.
It would be difficult to distinguish new disability due to a recurrence or worsening of a previous stroke or sequelae simply by basing on a follow-up telephone call
ARIC
Good Research question for study design: What is the Incidence of stroke among post MI survivors with atrial fibrillation?
The ARIC study is a prospective cohort study that rigorously collected data on myocardial infarction events that had occurred in the past at baseline and during follow-up. The same ECGs used to diagnose MIs could be examined for other abnormal ECG rhythm patterns – particularly those consistent with atrial fibrillation - corroborated with hospital records for verification. Also the study being prospective, biological plausibility of atrial fibrillation causing stroke and the possibility of establishing temporality of cause – effect, atrial fibrillation precedes stroke make question appropriate.
Bad study question: What is the risk for diastolic ventricular non-compliance (ill defined description and complication of heart disease) among post MI survivors. Outcome is also a sequelae of other long standing stress like hypertension other than an infarction, so non specific cause and may have preceded MI event plus it is ill defined, likely mis-measured plus rare outcome – not appropriate for cohort design