1) Provide an example of 4 threats to validity that you have encountered in your research, drawing one from each of the domains Cook and Campbell delineate (statistical conclusion validity, internal validity, construct validity, and external validity).
My example involves a cross-sectional study of Ebola virus disease survivors in the Democratic Republic of the Congo (DRC). We aimed to understand the long-term clinical and psychosocial sequelae of Ebola virus disease (EVD) by comparing survivors against contacts. The study population was a cohort of survivors from an Ebola outbreak that occurred more than 20 years ago (1995).
Statistical conclusion validity: The small sample size has been a major statistical threat. For the psychosocial sequelae, we used continuous outcomes and this may have partially been why we were able to detect a difference between survivor and contact groups. In contrast with clinical sequelae, we used binary outcomes and found an association with some health outcomes such as uveitis assuming a p-value cutoff of 0.05. However, we were also multiple hypothesis testing so if we were to adjust the alpha, we would not consider any of the findings statistically significant. As a result, our conclusions must be tempered.
Internal validity: Working in a place like DRC created challenges that threaten internal validity. The country language is French and there are many local languages, so we had to rely on translation and training to ensure that there was high quality data collection. There were several interviewers collecting data and while there were attempts of quality control, it is possible that quality of data collection varied by interviewer.
Construct validity: Some of the outcomes such as depressive and anxiety symptoms as well as stigma were social measurements that had potential threats to their construct validity. Ebola stigma is an example because there are several psychometric domains to the social process and we didn't ask questions that captured certain domains of stigma such as internalized stigma.
External validity: Studying a cohort of Ebola survivors more than 20 years after an Ebola outbreak is very specific and doesn't lend itself well to be generalizable to Ebola survivors from other outbreaks. However, the study does give a sense of what life could be like for Ebola survivors of recent outbreaks in West Africa and DRC.
2) For any data set you frequently use, look up the sample design and describe it.
In addition to this DRC study of Ebola survivors, I study the natural history of asymptomatic or unrecognized, symptomatic EVD in Liberia. This dataset is one that I will be frequently using. It is an observational, longitudinal cohort of Ebola survivors and contacts. The study enrolled 1134 Ebola survivors and 2400 contacts. Among this cohort, I was able to identify contacts who may have an asymptomatic or unrecognized, symptomatic Ebola virus infection. All of these participants are being followed for a 5-year period and attend study visits every 6 months at which time they undergo a questionnaire, clinical exam, and blood draw. These data will be used in longitudinal analyses to test hypotheses about the transmission and sequelae of asymptomatic or unrecognized EVD.