Laura Koth
HW1 Epi Methods III
1) Provide an example of 4 threats to validity that you have encountered in your research, drawing one from each of the domains Cook and Campbell delineate (statistical conclusion validity, internal validity, construct validity, and external validity).
Statistical conclusion validity: My technician is currently performing data analysis on a set of data obtained using a method called flow cytometry. In this method, the possibility for measurement error is great due to processing and analyzing samples in batches on different days using an instrument that can vary in terms of laser intensity from day to day. Therefore, we have to constantly be paying attention to all of our controls we use with each batch of experiments ranging from various beads we run, to the same internal reference controls, and despite all these attempts, we still found that we are getting a range of data that is making it challenging to have confidence in finding a statistically significant difference between our disease and control patient samples. Another problem I face is that fact that the effect size of the targets I am measuring may be small in magnitude (even though that can translate into important clinical outcomes) and do this measurement error, in addition to the small sample size I have, makes it difficult if not impossible to confidently answer the question of interest.
Internal validity: One of the big problems I face in my studies is that the cause of the disease I study is completely unknown. Therefore, the fundamental premise of observing exposure A which led to disease B is not possible for my hypotheses. For example, interferon gamma is a cytokine which is associated and thought to cause the inflammation in sarcoidosis, but perhaps it is not causing the inflammation but is a result of whatever is causing the sarcoidosis.
Construct validity: I also collect surveys completed by patients that intend to capture symptom experiences. I think this type of research has to deal with this concept of is the survey capturing what we really are hoping to study, for example symptom experiences related to fatigue, depression, or shortness of breath? I think that is what inadequate explication of constructs might be getting at?
External validity: this is a big problem for my studies in my cohort because of the issues with selection bias, so that generalizing findings from my cohort may not hold to other people with sarcoidosis who live in different parts of the country or world.
2) For any data set you frequently use, look up the sample design and describe it. My data set consists of patient reported surveys and clinical data collected from a group of patients with pulmonary and systemic sarcoidosis. My cohort was developed through recruitment using online ads and ads in local clinics around the Bay Area. Thus, my data set suffers from selection bias because it required interested participants to reach out to us. We did not randomly call or send letters to the population in the Bay Area. Because I did not send surveys to representative populations as described in Korn et al. it is not clear to me after reading the Korn paper whether any of these concepts would apply to the type of data set that I have. I actually am not sure if there is a word to describe the “sample design” for the data obtained from participants in my cohort. It isn’t a case control study. I did not match cases and controls either. Perhaps it is a prevalent disease sample design??