1) Provide an example of 4 threats to validity that you have encountered in your research, drawing one from each of the domains Cook and Campbell delineate (statistical conclusion validity, internal validity, construct validity, and external validity).
· Statistical conclusion validity (1): When studying rare diseases (such as Parkinson’s disease and rarer ones like dystonia, a movement disorder) low power is a chronic problem that permeates many observational and some interventional studies. Currently, one of my studies entails evaluation of pregnancy and delivery complications among women with two inherited causes of dystonia. The outcome of obstetric complications was obtained through a cross-sectional survey and 129 women reported at least one pregnancy (total of 310 pregnancies). While this is a very large number for women with inherited dystonia, it represents a small number of pregnancies overall to study number of complications. The results for both genetic causes together show a greater instance of complications among those mutation carriers, but when studied separately (further decreasing the sample size), while on the same direction, the confidence intervals become very wide (and the results are no longer significant) for one of the genetic forms studied.
· Statistical conclusion validity (2): Violation of the assumption of independently distributed errors à In our neurophysiology lab, we perform neuronal recordings from deep brain electrodes implanted during surgeries for Parkinson’s disease (and other movement disorders). These recordings from neurons are recorded for both sides of the brain for the same person. Statistical analyses are performed then to link specific brain recordings to measurements of movements (like rating scales performed by investigators) or measurements of behaviors/symptoms (via questionnaires). Recordings from one hemi-brain are usually analyzed independently, without taking into consideration the likely correlations between the recordings and outcomes from both sides of the brain from the same individual.
· Internal validity: I suspect that regression to the mean may explain some treatment effects in some of our observational studies of deep brain stimulation surgery for movement disorders. The subjects are selected into treatment because they have extreme values -otherwise, they would not qualify for surgical treatment-. Usually, one measurement is taken before surgery and one or several are taken postoperatively at determined time periods (such as 3, 6 and 12 months). It is possible that the extreme value (rating the severity of motor symptoms) would have naturally lowered to some degree naturally without the treatment intervention; this would be particularly important to note when there is no comparison group, such as in observational predictive studies.
· Construct validity: In a paper published in Brain in 2019, the authors tested the hypothesis that a reduction of brain connectivity in a particular area was associated with a higher risk for impulsive-compulsive behavior in Parkinson’s disease. Eighty subjects underwent functional MRI to evaluate brain connectivity as well as a questionnaire (QUIP-RS) to quantify the severity of impulsivity/compulsivity in Parkinson’s disease. In addition to likely problems of temporality and reverse causation (what was first, the lack of connectivity or the behavior, or are both the consequence of something else), the authors defined presence/absence of impulsivity/compulsivity based on a cut-off point in the questionnaire. I believe this may be an example of inadequate explanation of construct, since this may have been a too crude of a measure of complex behaviors in this population.
· External validity: Interaction of the causal relationship with units (The effect found with certain kinds of units might not hold if other kinds of units were used)à We are currently a site for two similar large multicenter randomized controlled trials for two similar and novel therapies aimed to slow down the progression of Parkinson’s disease (a progressive neurodegenerative disorder). The treatments in question are monthly infusions of two different types of antibodies against misfolded alpha-synuclein (a component of the pathological hallmark of this disease). The trials are recruiting only very recently diagnosed patients (symptoms <3 years) not yet on any symptomatic therapy. One important research question if these trials show a difference in progression for the treatment arm is to whether the findings would be generalizable to more advanced Parkinson’s disease patients.
2) For any data set you frequently use, look up the sample design and describe it.
The Parkinson’s Progression Markers Initiative (PPMI) is an observational, multi-center study funded by the Michael J Fox Foundation that actively collects clinical and imaging data, and biologic samples from several cohorts, that is available for download free to scientists, with the goal of establishing markers of disease progression in Parkinson’s disease.
The dataset contains several cohorts: a) De Novo PD patients: subjects with a diagnosis of PD for <=2 years who are not taking PD medications; b) Control subjects: people without PD at least 30 years of age who do not have any first degree relatives with PD; c) Prodromal subjects: subjects with PD who have a diagnosis of hyposmia (lack of sense of smell, a known prodromal factor for PD) or REM sleep behavior disorder (also a known predictor of PD); d) Genetic cohort: subjects with and without PD who have a genetic mutation in LRRK2, GBA or SNCA genes.
I am uncertain about the specific sampling of these cohorts. I believe subjects with PD and their spouses (without relatives with PD) were approached at the recruiting clinical centers and recruited for the studies. Anyone can request genetic testing (and counseling) through the PPMI website and be recruited for the genetic cohort, links to this study are also available through 23&me (direct to consumer genetic testing company). Subjects with hyposmia and REM sleep behavior disorder were approached and recruited through advertisement and by approaching sleep disorder clinics.