1) Provide an example of 4 threats to validity that you have encountered in your research, drawing one from each of the domains Cook and Campbell delineate (statistical conclusion validity, internal validity, construct validity, and external validity).
Statistical conclusion validity: Work with African American men with prostate cancer under active surveillance. Recruiting an underrepresented group with at a very specific stage of their illness has resulted in low Ns for the study. While a power analysis can provide an ideal target N for sufficient power reaching that goal is another hurdle. In addition, the treatment under study is exercise, which is done at home by the participants. While there is a standardize treatment and follow up (calls, Fitbit tracking) to ensure compliance issues of unreliability of treatment implementation can still arise. As the treatment is not conducted with in a lab setting the participants may not be executing the treatment as recommended.
Internal validity: Selection of participants is a treat to internal validity within this study as the population is extremely difficult to recruit. Thus the ones that are recruited were able to navigate the barriers that it difficult for African American men to participate within research studies. While it is not possible to know all the factors that differ as we do not know what we do not have some factors such as being within the UCSF medical system or having a doctor that is familiar with the study are factors that can be controlled for.
Construct: In a study examining impact of stereotype threat of Black identity the both treatment and outcome were measured the same way for all participants, depending on survey responses. Creating a treat to construct validity due to monomethod bias. While measures were validated and checking not to be leading a difference was observed when the study was administered by an interviewer instead of a survey.
External validity: With the previous study we examined the effect in two extreme Black students in psychology and African studies. These were the target populations for the study but those recruited into the study differ from their peers as the incentive was extra credit so they might have wanted the points or needed them. In addition, Black students that do not sit in either of these extremes such as those in criminal justice which is not culturally affirming but has a high representation of Black students may not see the same effect. This introduces interaction of causal relationship with units threat to external validity similar we can also see threat of causal relationship with setting.
2) For any data set you frequently use, look up the sample design and describe it.
Currently working with TRUEnth data N=218, men with prostate cancer at any stage, age from multiple recruitment sites around the US. Men need to have access to internet to do surveys and phone for one time contact with coordinator for on boarding into survey platform. Surveys measure dietary, exercise habits plus other lifestyle factors in additional to prostate cancer diagnosis for baselines measures. After which tailored diet and exercise plans are made for a 12-week course in a cohort study.