Week 1

Week 1

by Sandeep Brar -

1) Provide an example of 4 threats to validity that you have encountered in your research, drawing one from each of the domains Cook and Campbell delineate (statistical conclusion validity, internal validity, construct validity, and external validity).

Statistical conclusion validity:

Shadish, Cook and Campbell argue that a conclusion about covariation may be inaccurate if either variable is measured unreliably. In my data project examining the effect of angiotensin converting enzyme inhibitors (ACE-I) or angiotensin II receptor blocker (ARB) on further episodes of hospitalized acute kidney injury, there is measurement error of the exposure. Patients self-reported medications at the baseline study visit and this was not confirmed with electronic health record or prescription data. There is also likely measurement error of the outcome given that there are multiple different definitions of acute kidney injury. One definition is a 50% relative or 0.3 mg/dl absolute increase in creatinine from outpatient baseline to peak inpatient vs. a 50% relative change from minimum inpatient to maximum inpatient creatinine. The former definition is likely very sensitive and may be capturing cases of chronic kidney disease progression.  

Internal validity:

One of the threats to internal validity is selection. The authors explain that at the start of an experiment, the average person receiving one experimental condition already differs from the average person receiving another condition, and that this difference might account for any result observed after the experiment ends. This is a concern in my data project where physicians are more likely to prescribe ACE-I/ARB to healthier patients. Therefore, the improved outcomes seen with these medications in AKI survivors may be due to healthy user bias.

Construct validity:

A threat to construct validity is inadequate explication of constructs where the construct is identified at too specific of a level. In my data project, the underlying question is how ACE-I or ARB medications after the risk of acute kidney injury. However, one often needs frequent (daily) creatinine measurements to identify acute kidney injury. Given that it would be very difficult to do this for outpatients, my data project is on the risk of hospitalized acute kidney injury and ACE-I or ARB use. Patients who are admitted to hospital typically have daily bloodwork including creatinine measurements. It is likely that participants in ASSESS-AKI are experiencing subclinical, undetected acute kidney injury in the community, but this data is not captured.

External validity:

Interaction of the causal relationship over treatment variations is a threat to external validity. In my data project, any use of ACE-I or ARB at the baseline study visit was classified as ACE-I/ARB use. However, there are many different brands of ACE-I and ARB medications with varying doses. Furthermore, the ASSESS-AKI study population may have been using less nephrotoxic medications such as diuretics or non-steroid anti-inflammatory drugs which are known to increase the risk of acute kidney injury. These results may therefore not translate to a different population, for instance those with heart failure where diuretic use is common. Combining diuretics with ACE-I or ARB in a heart failure population may lead to more acute kidney injury events, in comparison to my data project which showed decrease acute kidney injury with ACE-I or ARB use.  

 

2) For any data set you frequently use, look up the sample design and describe it. 

ASSESS-AKI is a parallel, matched, prospective cohort design of adult participants with and without acute kidney injury. Adult patients with acute kidney injury were identified during the index hospitalization and screened for initial eligibility. Hospitalized adult patients who did not appear to suffer an acute kidney injury episode (controls) were matched in a 1:1 acute kidney injury:non-acute kidney injury ratio, with each non-acute kidney injury subject individually matched to their corresponding acute kidney injury subject on center and presence of baseline chronic kidney (eGFR <60 ml/min/1.73 m2).

For my data project, I used the unmatched cohort and stratified the cohort by the presence or absence of acute kidney injury at the index hospitalization. There was no clustering or weighting of observations.

Re: Week 1

by Erika Meza-Luman -

1) Provide an example of 4 threats to validity that you have encountered in your research, drawing one from each of the domains Cook and Campbell delineate (statistical conclusion validity, internal validity, construct validity, and external validity).

Statistical conclusion validity: I previously collaborated on the evaluation of a mental health service intervention that partnered community-based organizations (CBOs) with mental health providers (MHPs) to connect CBO clients with mental health services and make referrals as needed via a warm-handoff. The community-based organizations all served similar populations and offered similar services to the clients (i.e. job training programs) but partnered with different mental health institutions. However, a threat to validity was unreliability of treatment implementation because the level of engagement in developing these partnerships between CBOs and MHPs varied from site to site and from person to person within each site.

Internal validity: In a cross-sectional study assessing the effect of depressive symptoms on alcohol consumption a threat to internal validity is ambiguous temporal precedence. Given the cross-sectional nature of the data and no additional variables on alcohol consumption or depressive symptoms duration, it is unclear whether drinking problems led to depressive symptoms or vice versa.

Construct validity: In a study that included neighborhood audits in Baltimore, “Physical disorder” was based on items which included abandoned buildings, broken glass and litter that were measured on a frequency likert-scale (none, very little, some and a lot). However, in an urban setting like Baltimore, these response options may be interpreted differently based on different reference points that data collectors may be using for what would be considered “some” vs. “a lot” of broken glass or litter. Furthermore, this only captured features that were visible at the time of the audit and may be an inadequate analysis for the broader “physical disorder” construct. Therefore, inadequate explication of constructs may have been a threat to construct validity if the definitions for each of the response options had not been piloted, revised and redefined multiple times before using the tool.

External validity: Biologic Disease-Modifying Antirheumatic Drugs (bDMARDs) target parts of the immune system that trigger inflammation that causes joint and tissue damage and are sometimes prescribed to patients with Rheumatoid Arthritis (RA). Although the treatment effect of biologic DMARDs on achieving RA remission has been well established in middle-aged RA patients, observational studies on the effect of bDMARDs in elderly RA patients have been inconclusive. This could be an example of context-dependent mediation since there are additional factors that could be mediating the effect in the elderly population.

2) For any data set you frequently use, look up the sample design and describe it. 

I am currently working with data from the Mexican Health and Aging Study (MHAS) a national longitudinal study representative of Mexican adults born before 1951. MHAS participants were selected based on a multistage area probability sample distributed in all 32 states of the country in urban and rural areas. Households in the six states with high rates of out-migration to the US (accounting for 40% of all migrants to US) were oversampled while the rest were selected with probability proportionate-to-size.